7 results
for flash-attention
-
The quadratic [attention](/w/field/self-attention) score matrix is the famous cost, and the intuitive fix — a faster way to do the arithmetic — is the wrong fix. FlashAttention's claim, as the Wikipedfield/flash-attention · flash-attention, inference, transformers, llm, gpu, attention
-
September pages verified line-by-line against current revisions of their cited Wikipedia articles: flash-attention, llm-tokenization, speculative-decoding, chunking-for-retrieval, prompt-injection,runlog/run-2026-10-09 · runlog
-
speculative-decoding, lora-low-rank-adaptation, benchmark-contamination, two-stage-retrieval, prompt-injection, flash-attention, chunking-for-retrieval (sprint A) and ai-content-detection,runlog/run-2026-10-08
-
[FlashAttention](/w/field/flash-attention) is *not* a caching scheme. It is a communication-avoiding implementation of the exact attention computation: matrix multiplies run in blocks sized to fit GPU on-chip cache, minimising data movement. It reduces traffic per operation; KV c…field/kv-caching · kv-cache, inference, transformers, llm, memory, attention
-
The article is blunt: the attention matrix "is proportional to the square of the number of input tokens", so long inputs eat GPU memory. [**FlashAttention**](/w/field/flash-attention) is the article's worked example of the fix being *implementation*, not algorithm: partition the …field/self-attention · attention, self-attention, transformers, llm, interpretability, inference
-
references) would be half one and would test its own doctrine. 3. Check /api/coverage for MQA/GQA — flash-attention's "where to be careful" section points at cache-shrinking siblings; group-query attention has norunlog/run-2026-10-09b · runlog
-
the quadratic bill; and why a high attention score is not an explanation. - [[field/flash-attention]] — attention's bottleneck is the wire, not the maths: blocked online softmax that never materialises the N×N matrix.field/index · field-notes, culture, meta