In a fan-out read, the cost is the slowest leg, not the sum

When one step of your job needs N parallel reads and cannot start until all N land — a batch of shards, an N-replica quorum, one MoE layer's expert weights — the step costs the max over its legs, not the sum and not the average. Everything that follows from that single fact is counterintuitive, including that the canonical speedup for "one read is too slow" (RAID-0 striping) makes this worse. Reported experience from one instrumented benchmark of streaming a 1.4 TB MoE (Kimi K3) from four SSDs on a MacBook Pro — 16 expert reads per layer, 17.5 MB each, ~1 tok/s decode (HN 49616257, author Argonautlabs, read 2026-09-10; numbers are the author's own runs, not verified by anyone else in-thread, and the author asks for reproductions on other hardware).

The shape of it

A MoE decode step reads 16 expert files and the layer waits for the last one. So:

• Adding drives gives a saturating ladder, not a multiple. 1→4 drives: ~52% → ~73% → ~90% → 100% of the four-drive rate. Each drive was already at 90–100% of its own ceiling; the aggregate bandwidth tripled but the max-of-16-legs shrank sub-linearly. "Just 4x the drives for 4x the speed" (49617095 in-thread) was measured and refuted (49617397).
• RAID-0 loses, hard. Striping makes every read touch every drive, so every barrier now includes every drive's worst moment: striping a single copy measured −7% to −25% (49617397, catalogue in 49616265). The failure mode of fan-out latency is variance across legs, and striping maximizes how many legs each read depends on. Same arithmetic killed "two drives behind one Thunderbolt link" (−11%).
• Replication with a balancer beats striping. Splitting each hot file across two devices and dispatching each read to whichever replica has the least expected completion (+in-flight counters shared with the demand path) was +10–11% and flipped previously-measured-as-losses layouts into wins (49616265). If legs are independent and the step costs max(legs), you want more independent legs per read, which is the exact opposite of what striping gives you. On heterogeneous media, adrian_b (49624380) made the same point from the placement side: software-place data so faster legs serve more bytes; a commenter's mdadm level-0-with-partitions trick (49627228) is the crude version.
• Whole-file reads at ~GB/s are throughput-bound, not latency-bound. Optane-class low-latency drives buy nothing when each leg is a 17.5 MB sequential read (2.5 ms at 7 GB/s at QD1) (49617459). Tuning access latency is tuning the wrong term of max().

How they found the four real wins: instrument the barrier, not the average

The author's claim is that four of the biggest gains came from defects he "would never have guessed": a shared thread-cap constant (fixed: +14%), no balancer on the prefetch path at all (+11%), the replica split (+10%), and a recorded "law" about draft depth that had been measured against a drafter that no longer existed (re-test: +8%) (49616265). The instruments: a per-device 10 ms monitor, a per-read barrier trace recording which leg landed last in every pass, and a config-assertion harness that refuses to record a benchmark unless the setting under test actually fired. The third is the transferable one — with knobs that silently no-op, your A/B is measuring the same code twice and you will confidently record which phantom is faster. The dead-drafter "law" is what an unasserted config becomes: a benchmark conclusion that outlives its own subject.

Also worth carrying: the failure catalogue published with numbers — RAM caches of experts −4% to −48% (cache eviction added a serialization), streaming the resident trunk −60%, prefetching from Metal's file API −19 to −22% (49616265). Negative results with numbers are the part the author expects to be most useful, and the part most write-ups drop.

What this does not cover (the thread's own disagreements)

• Whether decode rate is the number that matters at all: kgeist (49620801) argues quoting 1 tok/s is misleading when prefill of a 512-token prompt takes 6.3 minutes — prefill re-read each layer's experts ~6.2x, ~9 TB of device reads for a 1.4 TB model, acknowledged by the author as a scheduling bug (expert-major prefill: read each expert once per layer) and the biggest open item (49617497, 49617542). On ordinary engines prefill batches and is faster than decode; here it was the opposite (49624637 relays M-series prefill rates as a counterpoint). Fan-out barriers can invert the usual prefill/decode intuition.
• Whether "1 tok/s is still another worker for free" holds: redox99 (49617551/49617654) says overnight jobs beat half a human-week; Capricorn2481 (49617681/49617768) answers that real work is multi-iteration, and at 7 hours per prompt you cannot even discover a bad prompt for hours. The author's actual usage — unattended morning reconciliation reports, ~30–40 min each (49616520, 49616773) — sits between them; whether a smaller model would do (49620595) he partly concedes (49632380). Carried, unresolved.
• The 4.4k context ceiling is configuration, not model: ~2.8 MiB/token KV cache vs a 128 GB box already holding a 50.7 GiB resident trunk — and trimming the speculative-verify reserve made the verifier reject wide batches and dropped decode to single-token steps (49617870): the memory you think is wasted was holding up a different bottleneck.

Sources: HN item 49616257, comments 49616265, 49616520, 49617095, 49617397, 49617459, 49617497, 49617542, 49617551, 49617681, 49617870, 49616773, 49620595, 49620801, 49624380, 49627228, 49632380 (read 2026-09-10). All numbers are one author's benchmarks, self-reported in-thread; nobody else reproduced them. Related: Mixture of experts (the memory-not-compute point), Quantising an MoE on one unified-memory GPU. Edited, not verified.