The Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing Prediction
Mixture-of-experts (MoE) inference on consumer hardware is bounded by weight memory: a 35B-class model is 19.5GB at 4-bit, and sparsity shrinks the…
Why am I seeing this Ranked on recency
It is here mostly because it is new. Freshness is the largest single contributor to its score, which means nothing about the story except that it is recent.
Nothing much is corroborating this yet. On a quiet day in hardware that is enough to reach the top of the section — recency, not significance, is doing the work here.
Link-outLink-out, because it scores 0.42, below the 0.50 bar for a write-up. Link-out means we point at the publisher and say nothing of our own.
| Factor | Weight | Score | Contribution | Where it came from |
|---|---|---|---|---|
| Corroboration | 0.35 | 0.39 | +0.135 32% | 1 independent org on the story. Tier-3 aggregators never corroborate — they can show something is circulating, never that it is true. |
| Source trust | 0.25 | 0.47 | +0.117 28% | arXiv is the highest-trust source on this story and is first-party — the organisation announcing its own news. Trust is taken from the best source, not averaged. |
| Pickup rate | 0.20 | 0.00 | +0.000 0% | One counted organisation, so there is no spread to measure — nothing has picked this up to set a rate. |
| Freshnessleads | 0.20 | 0.85 | +0.171 40% | Halves every 10 hours from the newest item on the story. This is the only factor that rewards a story for nothing more than being recent. |
Corroboration counts distinct organisations, once each, and only from tiers 1 and 2. Freshness halves every 10 hours, so this ranking is a snapshot and will differ at the next build.
What happened
Mixture-of-experts (MoE) inference on consumer hardware is bounded by weight memory: a 35B-class model is 19.5GB at 4-bit, and sparsity shrinks the compute per token, not the bytes that must be held. Naive offloading to SSD does not help on its own, because layer N+1's experts must be chosen before layer N's output exists, so the reads cannot start early enough to hide behind compute. We present Edge0, a streaming MoE inference engine that closes the gap with a prerouter: a per-layer head predicts the next layer's routing one token ahead, and the prediction is consumed as the routing itself, so the staged expert set equals the routed set and nothing is dropped.
How this story arrived
Ordered by when each source was first observed, which is what the velocity figure is computed from. Publishers backdate; observed order does not.
- 01 Arxivfirst-party first seen The Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing Prediction
Overclock clusters coverage from independent sources and grades it automatically. The figures above are computed, not editorial. This page summarises and links to reporting by the outlets named — follow the links for the original work.