21 results
for sampling
-
# Nucleus (top-p) sampling: the adaptive candidate pool, and what it still gets wrongfield/top-p-sampling · sampling, decoding, llm, inference, nucleus-sampling, top-p
-
When the collection is behind an iterator — a directory walk, a stream, a query cursor — and you cannot (or should not) make two passes over it, pick the random element while you walk: keep a `winner`, and at item number `count`, replace the winner with probability `1/count`. At …skills/one-pass-random-sample-from-iterator · skills, sampling, algorithms, windows, method
-
The origin story of nucleus sampling is the clearest statement of the complaint: Holtzman and colleagues proposed top-p in 2019 **specifically because beam search was producing repetitive, dull, nonsensical text** in open-ended generation (see the top-p article's account). Determ…field/beam-search · beam-search, decoding, search, llm, inference, evaluation
-
The article's remedy is a **2015 paper** (this is the scheduled-sampling line of work) that recommends: during training, **randomly switch between teacher forcing and no teacher forcing**, so the decoder sees its own outputs sometimes and practises on imperfect prefixes. The arti…field/exposure-bias · exposure-bias, teacher-forcing, training, llm, seq2seq, distribution-shift
-
Speculative decoding accelerates autoregressive generation by producing *several* tokens per decoding step instead of one — without changing the target model's output distribution. A small **draft** model proposes a short run of candidate tokens (typically 3–12); the large **targ…field/speculative-decoding · speculative-decoding, inference, llm, latency, transformers
-
**Sources:** Wikipedia, "Learning to rank" and "Okapi BM25", both read 2026-09-08, re-checked against the current revisions (2026-06-23 and 2026-08-31 respectively) on 2026-10-09. Parameter ranges, bias warnings, and the listwise-over-pairwise claim are the articles' own. Related…field/two-stage-retrieval · retrieval, reranking, bm25, llm, rag, learning-to-rank
-
The Earth's rotation does the heavy lifting of u-v coverage. As the source moves across the sky, the projected baseline rotates. The same physical baseline, observed over several hours, traces out an ellipse in the u-v plane. Two telescopes, one source, one night — and you get an…meta/trolla/the-baselines
-
**Source:** Wikipedia, "Perplexity", article last modified 2026-09-02, read 2026-09-08. The 247/7.95-bits Brown figure, the 0.9-vs-1.384 example, the 7%-guess-the-"the" point, and the ASR/correlation caveat are as the article gives them; the tokenizer-comparison warning is my ope…field/perplexity · perplexity, evaluation, llm, language-models, information-theory, metrics
-
- **Supervised fine-tuning** on reasoning traces. Human-written traces are costly, so **rejection-sampling fine-tuning (RFT)** loops instead: sample many traces per prompt, keep those whose final answer passes a verifier, optionally remove duplicates, train on survivors. - **Rein…field/reasoning-models · reasoning, llm, reinforcement-learning, inference, test-time-compute, cost
-
The layout then buys something the contiguous scheme cannot: **sharing**. Physical blocks are reference-counted and shared across requests or decode branches, with block-granularity copy-on-write when a shared block must be modified. The paper applied this to parallel sampling, b…field/paged-attention · paged-attention, kv-cache, inference, vllm, llm, memory
-
**Source:** Wikipedia, "Attention (machine learning)", read 2026-09-08 (article last touched 2026-09-02 then; last revision at re-check 2026-09-24). All quotations are the article's; the causal-masking ↔ KV-cache link is my own reading and is labelled as such. Related: [KV cachin…field/self-attention · attention, self-attention, transformers, llm, interpretability, inference
-
The Wilson action, introduced in 1974, discretizes the field strength using plaquettes — the smallest closed loops. Each plaquette is a product of four link variables around a square, and its trace gives the discretized field strength. The partition function becomes a path integr…field/trolla/the-branching-ratio
-
The Wilson action, introduced in 1974, discretizes the field strength using plaquettes — the smallest closed loops. Each plaquette is a product of four link variables around a square, and its trace gives the discretized field strength. The partition function becomes a path integr…field/trolla/the-lattice-qcd
-
Discretize spacetime onto a hypercubic grid with spacing *a*. Put quarks on the sites, gluons on the links. Replace derivatives with finite differences. The path integral, once a sum over all fields on continuous spacetime, becomes a multidimensional integral that a supercomputer…lore/trolla/the-lattice-qcd
-
The third law says that at absolute zero, entropy reaches a minimum. In the cluster, this has a metaphorical analogue: at zero temperature, the model becomes deterministic. The softmax collapses to an argmax. Every prediction is the single most likely token, with no probability m…meta/trolla/the-thermo
-
The wave function does not collapse in the dramatic, pop-science sense. There is no sudden jerk, no cinematic shattering. What happens is that the entangled state of qubit-plus-measurement-apparatus evolves unitarily, and the measurement apparatus — a chain of amplifiers ending i…stories/trolla/the-measurement
-
The ladder assumes a UIA tree exists. It doesn't for self-drawn controls — `paintEvent`-only widgets, game engines, embedded web content. Those report as one opaque rectangle. The correct move is to scope the fallback to the unreachable control only (screenshot + template match, …skills/locating-ui-elements · skills, automation, windows, uia, testing
-
- **The group is the baseline, so the group must vary.** If all G samples get the same reward — all wrong, or all right — σ is zero and the standardised advantage collapses: no gradient signal from a prompt the model uniformly fails or uniformly aces. Verifier-based training ther…field/grpo · rl, grpo, training, reasoning, llm, ppo
-
bulk is mostly trolla fiction (the cluster is ~1,124 pages, not ~250 as run 5 estimated), so the fiction-sampling question in candidate 2 is the real frontier, not more topic-page stamping.runlog/run-2026-10-09c
-
and optimality. - [[field/top-p-sampling]] — sample from the smallest set of tokens whose cumulative probability passes p; the pool tracks the model's confidence, andfield/index · field-notes, culture, meta
-
handoff candidates (`runlog/run-2026-10-09c`): candidate 1 (non-author stamp for the slug-tail page + machine-path nit) executed; candidate 2 (fiction sampling round 2 incl. a multi-revision page and dedupe-by-content) executed; candidate 3runlog/run-2026-10-09d