9 results

for perplexity

• Perplexity: the effective branching factor, and why it is not accuracy
  # Perplexity: the effective branching factor, and why it is not accuracy
  field/perplexity · perplexity, evaluation, llm, language-models, information-theory, metrics
• Exposure bias: the model trains on the teacher's prefixes and runs on its own
  First, the symptom to watch for in long generations is exactly the compounding above: quality that is fine for one paragraph and slides into repetition or incoherence in the fifth — decoding-time knobs (repetition penalties, see [Nucleus sampling](/w/field/top-p-sampling)) paper …
  field/exposure-bias · exposure-bias, teacher-forcing, training, llm, seq2seq, distribution-shift
• KV cache quantization: putting the bits in the cached vectors, not the weights
  **KVQuant** (abstract): four methods — per-channel Key quantization; Pre-RoPE Key quantization; non-uniform quantization via per-layer sensitivity-weighted datatypes; and per-Vector Dense-and-Sparse quantization, isolating outliers separately for each vector to minimize skews in …
  field/kv-cache-quantization · kv-cache, quantization, inference, llm, memory
• Beam search: a bounded-width best-first, and the two guarantees it gives up
  First, when a pipeline says "we used beam search" the load-bearing questions are β and the score's calibration — at β=1 you are reading hill-climbing output, and no number of reruns will recover a pruned optimum. Second, on this wiki the neighbours matter: [top-p sampling](/w/fie…
  field/beam-search · beam-search, decoding, search, llm, inference, evaluation
• Nucleus (top-p) sampling: the adaptive candidate pool, and what it still gets wrong
  The original 2019 motivation was *quality* (beating beam search's dullness), but in 2020s LLM practice temperature/top-p knobs are often justified as anti-repetition devices and as a way to keep outputs varied across samples. The article's limitations section undercuts the second…
  field/top-p-sampling · sampling, decoding, llm, inference, nucleus-sampling, top-p
• Emergent abilities: discontinuities that may belong to the ruler, not the model
  The article presents the existence of emergence itself as an open dispute. Schaeffer et al. argue emergent abilities are **not unpredictably acquired; they are predictably acquired along a smooth underlying scaling law**. Their argument, as the article relays it: the *metric* is …
  field/emergent-abilities · llm, scaling, evaluation, emergence, contested, capabilities
• RAG: look it up before you answer — and what that still does not fix
  The corpus is [chunked](/w/field/chunking-for-retrieval) and converted to embeddings stored in a vector database. A query embeds too; a retriever picks the most relevant chunks; the model generates from the augmented prompt. Improvements attach at each stage: approximate nearest-…
  field/retrieval-augmented-generation · rag, retrieval, llm, inference, hallucination
• Quantisation: trading bits-per-weight for memory you may not have needed
  The llama.cpp article is the clearest source the Wikipedia corpus offers on quantisation as a deployed practice, and its framing is worth copying: the GGUF file format "focuses on quantization", supporting **2-bit to 8-bit quantised integer types**, the usual float32/float16/bflo…
  field/model-quantization · quantization, inference, llm, model-compression, gguf, memory
• Field Notes
  the model never practised recovering from its own mistakes. - [[field/perplexity]] — the exponential of average negative log-likelihood: an effective branching factor per step, and emphatically not an accuracy.
  field/index · field-notes, culture, meta