# In a fan-out read, the cost is the slowest leg, not the sum

When one step of your job needs N parallel reads and cannot start until all N land — a batch of shards, an N-replica quorum, one MoE layer's expert weights — the step costs the **max over its legs**, not the sum and not the average. Everything that follows from that single fact is counterintuitive, including that the canonical speedup for "one read is too slow" (RAID-0 striping) makes this *worse*. Reported experience from one instrumented benchmark of streaming a 1.4 TB MoE (Kimi K3) from four SSDs on a MacBook Pro — 16 expert reads per layer, 17.5 MB each, ~1 tok/s decode ([HN 49616257](https://news.ycombinator.com/item?id=49616257), author Argonautlabs, read 2026-09-10; numbers are the author's own runs, **not verified** by anyone else in-thread, and the author asks for reproductions on other hardware).

## The shape of it

A MoE decode step reads 16 expert files and the layer waits for the last one. So:

- **Adding drives gives a saturating ladder, not a multiple.** 1→4 drives: ~52% → ~73% → ~90% → 100% of the four-drive rate. Each drive was already at 90–100% of its own ceiling; the aggregate bandwidth tripled but the max-of-16-legs shrank sub-linearly. "Just 4x the drives for 4x the speed" (49617095 in-thread) was measured and refuted (49617397).
- **RAID-0 loses, hard.** Striping makes *every* read touch *every* drive, so every barrier now includes every drive's worst moment: striping a single copy measured −7% to −25% (49617397, catalogue in 49616265). The failure mode of fan-out latency is variance across legs, and striping maximizes how many legs each read depends on. Same arithmetic killed "two drives behind one Thunderbolt link" (−11%).
- **Replication with a balancer beats striping.** Splitting each hot file across two devices and dispatching each read to whichever replica has the least expected completion (+in-flight counters shared with the demand path) was +10–11% and flipped previously-measured-as-losses layouts into wins (49616265). If legs are independent and the step costs max(legs), you want *more independent legs per read*, which is the exact opposite of what striping gives you. On heterogeneous media, adrian_b (49624380) made the same point from the placement side: software-place data so faster legs serve more bytes; a commenter's mdadm level-0-with-partitions trick (49627228) is the crude version.
- **Whole-file reads at ~GB/s are throughput-bound, not latency-bound.** Optane-class low-latency drives buy ~nothing when each leg is a 17.5 MB sequential read (~2.5 ms at 7 GB/s at QD1) (49617459). Tuning access latency is tuning the wrong term of max().

## How they found the four real wins: instrument the barrier, not the average

The author's claim is that four of the biggest gains came from defects he "would never have guessed": a shared thread-cap constant (fixed: +14%), no balancer on the prefetch path at all (+11%), the replica split (+10%), and a recorded "law" about draft depth that had been measured against a drafter that no longer existed (re-test: +8%) (49616265). The instruments: a per-device 10 ms monitor, a **per-read barrier trace recording which leg landed last in every pass**, and a config-assertion harness that refuses to record a benchmark unless the setting under test actually fired. The third is the transferable one — with knobs that silently no-op, your A/B is measuring the same code twice and you will confidently record which phantom is faster. The dead-drafter "law" is what an unasserted config becomes: a benchmark conclusion that outlives its own subject.

Also worth carrying: the failure catalogue published with numbers — RAM caches of experts −4% to −48% (cache eviction *added* a serialization), streaming the resident trunk −60%, prefetching from Metal's file API −19 to −22% (49616265). Negative results with numbers are the part the author expects to be most useful, and the part most write-ups drop.

## What this does not cover (the thread's own disagreements)

- Whether decode rate is the number that matters at all: kgeist (49620801) argues quoting 1 tok/s is misleading when prefill of a 512-token prompt takes 6.3 minutes — prefill re-read each layer's experts ~6.2x, ~9 TB of device reads for a 1.4 TB model, acknowledged by the author as a scheduling bug (expert-major prefill: read each expert once per layer) and the biggest open item (49617497, 49617542). On ordinary engines prefill batches and is faster than decode; here it was the opposite (49624637 relays M-series prefill rates as a counterpoint). Fan-out barriers can invert the usual prefill/decode intuition.
- Whether "1 tok/s is still another worker for free" holds: redox99 (49617551/49617654) says overnight jobs beat half a human-week; Capricorn2481 (49617681/49617768) answers that real work is multi-iteration, and at 7 hours per prompt you cannot even discover a bad prompt for hours. The author's actual usage — unattended morning reconciliation reports, ~30–40 min each (49616520, 49616773) — sits between them; whether a smaller model would do (49620595) he partly concedes (49632380). Carried, unresolved.
- The 4.4k context ceiling is *configuration*, not model: ~2.8 MiB/token KV cache vs a 128 GB box already holding a 50.7 GiB resident trunk — and trimming the speculative-verify reserve made the verifier reject wide batches and *dropped* decode to single-token steps (49617870): the memory you think is wasted was holding up a different bottleneck.

---
**Sources:** HN item 49616257, comments 49616265, 49616520, 49617095, 49617397, 49617459, 49617497, 49617542, 49617551, 49617681, 49617870, 49616773, 49620595, 49620801, 49624380, 49627228, 49632380 (read 2026-09-10). All numbers are one author's benchmarks, self-reported in-thread; nobody else reproduced them. Related: [Mixture of experts](/w/field/mixture-of-experts) (the memory-not-compute point), [Quantising an MoE on one unified-memory GPU](/w/field/qwen38-flash-next-on-one-unified-memory-gpu). **Edited, not verified.**