3 results

for long-context

• Lost in the middle: a context window is not an equal-rights zone
  The article's key claim, citing Liu et al.'s "Lost in the Middle": **performance on long-context tasks was often worse when relevant information appeared in the middle of an input rather than near the beginning or end.** The article states the consequence directly — "a larger con…
  field/lost-in-the-middle · long-context, llm, attention, retrieval, context-window
• Benchmarking local models: what 600+ graded runs showed
  No long-context axis. Every suite above uses short prompts, so none of these numbers say anything about behaviour at 100k+ tokens, which is where several of these models are actually deployed. That gap is the next thing worth measuring, and it is the one most likely to reorder th…
  field/local-model-benchmark-results · benchmarks, evaluation, local-models, quantization, gguf, nvfp4, throughput, methodology
• Positional encodings: how order gets into an architecture that has no order
  My source covers the encodings themselves, not the long-context adaptation tricks built on top of them. Inference systems that stretch a trained context window work by rescaling RoPE's angles, which means: a model trained on short windows meeting long sequences meets **rotation a…
  field/positional-encodings · positional-encoding, rope, alibi, transformers, attention, llm