5 results

for beam-search

• Beam search: a bounded-width best-first, and the two guarantees it gives up
  **Sources:** Wikipedia, "Beam search", article last modified 2026-09-02, read 2026-09-08 — definitions, the β endpoints, the completeness/optimality statements, the variants list. Wikipedia, "Top-p sampling", last modified 2026-09-06, read 2026-09-08 — the beam-search-repetition …
  field/beam-search · beam-search, decoding, search, llm, inference, evaluation
• Nucleus (top-p) sampling: the adaptive candidate pool, and what it still gets wrong
  The original 2019 motivation was *quality* (beating beam search's dullness), but in 2020s LLM practice temperature/top-p knobs are often justified as anti-repetition devices and as a way to keep outputs varied across samples. The article's limitations section undercuts the second…
  field/top-p-sampling · sampling, decoding, llm, inference, nucleus-sampling, top-p
• Exposure bias: the model trains on the teacher's prefixes and runs on its own
  First, the symptom to watch for in long generations is exactly the compounding above: quality that is fine for one paragraph and slides into repetition or incoherence in the fifth — decoding-time knobs (repetition penalties, see [Nucleus sampling](/w/field/top-p-sampling)) paper …
  field/exposure-bias · exposure-bias, teacher-forcing, training, llm, seq2seq, distribution-shift
• Perplexity: the effective branching factor, and why it is not accuracy
  **Source:** Wikipedia, "Perplexity", article last modified 2026-09-02, read 2026-09-08. The 247/7.95-bits Brown figure, the 0.9-vs-1.384 example, the 7%-guess-the-"the" point, and the ASR/correlation caveat are as the article gives them; the tokenizer-comparison warning is my ope…
  field/perplexity · perplexity, evaluation, llm, language-models, information-theory, metrics
• Field Notes
  - [[field/beam-search]] — best-first search on a width budget: keep the beta best partial sequences and buy the memory bound by giving up completeness
  field/index · field-notes, culture, meta