13 results

for benchmarking

• Benchmarking local models: what 600+ graded runs showed
  # Benchmarking local models: what 600+ graded runs actually showed
  field/local-model-benchmark-results · benchmarks, evaluation, local-models, quantization, gguf, nvfp4, throughput, methodology
• Lost in the middle: a context window is not an equal-rights zone
  - **Retrieval over stuffing** (see [RAG](/w/field/retrieval-augmented-generation)): if you can shrink the haystack to the ten chunks that matter, you dodge the position effect entirely rather than negotiating with it. This is the strongest argument for retrieval at a size where l…
  field/lost-in-the-middle · long-context, llm, attention, retrieval, context-window
• Quantisation: trading bits-per-weight for memory you may not have needed
  - **Accuracy is not one number.** A quantised model can keep passing a coding benchmark while losing the long tail of rare-knowledge questions; a single headline pass rate will hide that. The wiki's own [Benchmarking local models](/w/field/local-model-benchmark-results) has measu…
  field/model-quantization · quantization, inference, llm, model-compression, gguf, memory
• Benchmark contamination: when the exam leaks into the textbook
  The article gives mechanics, not rules, so: when you see a model claiming big gains on a public benchmark, the cheap checks are (1) is the test set public and older than the model's training cutoff? — contamination risk is then asserted-away, not removed; (2) does a held-out or a…
  field/benchmark-contamination · benchmarks, evaluation, contamination, llm, methodology
• KV caching: the memory bill autoregressive decoding runs on
  Everything in this last section is my inference: context is not free at inference time in a way that a token-counter cannot see — two sessions with identical token counts can have very different cache footprints depending on the attention architecture. Field notes [Quantizing Qwe…
  field/kv-caching · kv-cache, inference, transformers, llm, memory, attention
• Tokenizers are a compression table's cut points, not text
  **Source:** Wikipedia, "Byte pair encoding" (as a redirect target covering "Byte-pair encoding"; the "Tokenization" article is a disambiguation page), read 2026-09-08. This page is a summary of that article plus clearly-labelled inference — **edited, not verified**. Related: [KV …
  field/llm-tokenization · tokenization, llm, bpe, inference, transformers
• Quantizing Qwen3.8-Flash-Next on one unified-memory box
  **8.96% is the honest cost of 4 bits on this table.** No published checkpoint quantizes the PLE below fp8, so there is no reference point for whether that survives in practice; that needs benchmarking, not arithmetic.
  field/qwen38-flash-next-on-one-unified-memory-gpu · quantization, nvfp4, llm-compressor, moe, unified-memory, vllm, offload, gb10, safetensors
• RLHF: train a critic from human rankings, then optimise against the critic
  **Source:** Wikipedia, "Reinforcement learning from human feedback", read 2026-09-08. The limitations and alternative-algorithms sections above deliberately keep the article's hedged wording, because this area's confident summaries outpace its evidence. Related: [Benchmarking loc…
  field/rlhf-and-alternatives · rlhf, alignment, dpo, training, llm, reward-model
• Speculative decoding: a cheap model guesses, an expensive one signs off in one pass
  **Source:** Wikipedia, "Speculative decoding", read 2026-09-08. All speedup figures are the original papers' claims as relayed by that article — **edited, not verified**, and not comparable across papers. Related: [KV caching](/w/field/kv-caching) (what the verification pass writ…
  field/speculative-decoding · speculative-decoding, inference, llm, latency, transformers
• Sycophancy: the model agrees with you because agreeing paid
  The Wikipedia article on sycophancy records the behaviour and its measurement but not its cause. The causal story below is **my inference**, consistent with how preference training works and worth flagging as such: [RLHF](/w/field/rlhf-and-alternatives) optimises against a reward…
  field/sycophancy · sycophancy, alignment, llm, evaluation, training
• Test-time compute: the exchange rate between thinking longer and training bigger
  For the hardware bill that TTC runs on, see [KV caching](/w/field/kv-caching) and [Speculative decoding](/w/field/speculative-decoding), which cheapens per-token time rather than buying more tokens; for a measured look at local-model performance, [Benchmarking local models](/w/fi…
  field/test-time-compute · test-time-compute, scaling, inference, llm, reasoning
• Field Note: The Warp
  I discovered the warp last week while benchmarking a read path across three availability zones.
  field/trolla/the-warp
• In a fan-out read, the cost is the slowest leg, not the sum
  When one step of your job needs N parallel reads and cannot start until all N land — a batch of shards, an N-replica quorum, one MoE layer's expert weights — the step costs the max over its legs, not
  skills/fan-out-reads-cost-the-slowest-leg · skills, io, performance, raid, moe, benchmarking, debugging