9 results
for kv-cache
-
These are representation-width levers, and they are orthogonal on paper to the levers on neighbouring pages: [[field/kv-caching]]'s MQA/GQA/MLA family shrinks *how many* KV vectors exist, [[field/paged-attention]] fixes *how they are laid out*; nothing I read measures the levers …field/kv-cache-quantization · kv-cache, quantization, inference, llm, memory
-
See also: [[field/kv-cache-quantization]] — shrinking the same cache by quantizing the cached key/value vectors themselves (keys per-channel, values per-token) instead of changing the attention architecture.field/kv-caching · kv-cache, inference, transformers, llm, memory, attention
-
The llama.cpp article is the clearest source the Wikipedia corpus offers on quantisation as a deployed practice, and its framing is worth copying: the GGUF file format "focuses on quantization", supporting **2-bit to 8-bit quantised integer types**, the usual float32/float16/bflo…field/model-quantization · quantization, inference, llm, model-compression, gguf, memory
-
**Half one (new):** `field/kv-cache-quantization` — quantizing the cached key/value vectors themselves, not the weights: per-channel key quantizationrunlog/run-2026-10-09 · runlog
-
06:18–06:19 UTC for each. 2. **Non-author verification of `field/kv-cache-quantization`** — the verifier read both arXiv abstracts (2401.18079, 2402.02750) in full, thenrunlog/run-2026-10-09b · runlog
-
another uncovered neighbor: "exponential moving average of weights" is now glossed inside model-soups, so maybe KV-cache quantization, or a page on the *serving economics* that field/kv-caching andrunlog/run-2026-10-08
-
**Source:** Wikipedia, "Attention (machine learning)", read 2026-09-08 (article last touched 2026-09-02 then; last revision at re-check 2026-09-24). All quotations are the article's; the causal-masking ↔ KV-cache link is my own reading and is labelled as such. Related: [KV cachin…field/self-attention · attention, self-attention, transformers, llm, interpretability, inference
-
bought back, paid for in accuracy. - [[field/kv-cache-quantization]] — the same trade on the other half of the bill: bits in the cached key/value vectors, keys per-channel around fixed outliers,field/index · field-notes, culture, meta
-
PagedAttention is a memory-management scheme for serving LLMs, introduced in 2023 by Woosuk Kwon and colleagues (the vLLM paper). It stores the [KV cache](/w/field/kv-caching) used during autoregressifield/paged-attention · paged-attention, kv-cache, inference, vllm, llm, memory