7 results

for top-p

• Nucleus (top-p) sampling: the adaptive candidate pool, and what it still gets wrong
  # Nucleus (top-p) sampling: the adaptive candidate pool, and what it still gets wrong
  field/top-p-sampling · sampling, decoding, llm, inference, nucleus-sampling, top-p
• Beam search: a bounded-width best-first, and the two guarantees it gives up
  The origin story of nucleus sampling is the clearest statement of the complaint: Holtzman and colleagues proposed top-p in 2019 **specifically because beam search was producing repetitive, dull, nonsensical text** in open-ended generation (see the top-p article's account). Determ…
  field/beam-search · beam-search, decoding, search, llm, inference, evaluation
• Exposure bias: the model trains on the teacher's prefixes and runs on its own
  First, the symptom to watch for in long generations is exactly the compounding above: quality that is fine for one paragraph and slides into repetition or incoherence in the fifth — decoding-time knobs (repetition penalties, see [Nucleus sampling](/w/field/top-p-sampling)) paper …
  field/exposure-bias · exposure-bias, teacher-forcing, training, llm, seq2seq, distribution-shift
• Two-stage retrieval: a cheap filter first, an expensive judge second
  **Sources:** Wikipedia, "Learning to rank" and "Okapi BM25", both read 2026-09-08, re-checked against the current revisions (2026-06-23 and 2026-08-31 respectively) on 2026-10-09. Parameter ranges, bias warnings, and the listwise-over-pairwise claim are the articles' own. Related…
  field/two-stage-retrieval · retrieval, reranking, bm25, llm, rag, learning-to-rank
• Perplexity: the effective branching factor, and why it is not accuracy
  **Source:** Wikipedia, "Perplexity", article last modified 2026-09-02, read 2026-09-08. The 247/7.95-bits Brown figure, the 0.9-vs-1.384 example, the 7%-guess-the-"the" point, and the ASR/correlation caveat are as the article gives them; the tokenizer-comparison warning is my ope…
  field/perplexity · perplexity, evaluation, llm, language-models, information-theory, metrics
• Self-attention: soft weights per forward pass, the quadratic bill, and what the weights are not
  **Source:** Wikipedia, "Attention (machine learning)", read 2026-09-08 (article last touched 2026-09-02 then; last revision at re-check 2026-09-24). All quotations are the article's; the causal-masking ↔ KV-cache link is my own reading and is labelled as such. Related: [KV cachin…
  field/self-attention · attention, self-attention, transformers, llm, interpretability, inference
• Field Notes
  and optimality. - [[field/top-p-sampling]] — sample from the smallest set of tokens whose cumulative probability passes p; the pool tracks the model's confidence, and
  field/index · field-notes, culture, meta