The Information reckons xAI founder Elon Musk is only getting just more than 10 per cent out of its installed Nvidia capacity, thanks to a software stack that is not keeping up.
xAI runs about 550,000 Nvidia GPUs, a mix of H100s and H200s, spread across its Memphis and Colossus clusters with plenty of liquid cooling in the mix. They might be a generation behind Blackwell, but half a million accelerators is still a ridiculous pile of silicon.
The problem is utilisation, with the report putting xAI at 11 per cent, or roughly 60,000 GPUs doing real work while the rest sit around burning money. As GPU counts climb from thousands into the hundreds of thousands, idle time snowballs and the cracks in distributed training start showing fast.
At that scale, the pipeline and analysis stages leave expensive kit waiting for data, synchronisation or the next job to arrive. It is not unique to xAI either, because efficiency at huge scale is a nasty engineering slog across the AI industry.
Some outfits have it more nailed down, with Meta and Google pushing utilisation beyond 40 per cent, with figures cited at 43 per cent and 46 per cent. The usual range sits around 35 to 45 per cent, which still means a lot of hardware spends its day doing nothing useful.
xAI wants to lift utilisation to 50 per cent, with the fixes expected to land in infrastructure and software stack tuning rather than another round of shiny new GPUs. There is talk it could rent out chunks of its fleet, while it shifts future workloads towards systems better suited to agentic AI.
Musk is still pushing the TeraFab idea, building in-house silicon under its “AI” family label and leaning on Intel’s 14A process for future xAI, SpaceX and other projects, assuming the code stops tripping over its shoelaces.
