Nvidia has unveiled NVHBM, a custom memory wheeze designed to make future AI GPUs faster and less power-hungry.
According to Nvidia, NVHBM extends NVLink Fusion and targets future XPUs running agentic and physical AI workloads. The idea is to shift the memory controller off the XPU and shove it into the HBM base die, because apparently even memory controllers now need better real estate.
Nvidia claims the approach will deliver up to 30 per cent more memory bandwidth and up to 15 per cent lower HBM power use than standard HBM4e. It could free up to 25 per cent more compute die area on the main chip, which is handy when AI models are ballooning into trillion-parameter absurdity.
“Compared with the JEDEC HBM4e standard, this design reduces PHY and support area by up to 67 per cent. The narrower interface also simplifies interposer routing, providing up to 80% more usable silicon across the entire layout.”
In human, that means less chip area wasted on memory plumbing and more room for the expensive compute bits that Nvidia likes selling by the shedload. The narrower interface should make interposer routing less of a nightmare.
The idea is not completely out of nowhere. Qualcomm has been pushing a similar trick with HBC and LPDDR DRAM stacks, while Samsung, SK Hynix and Micron have been making similar noises about HBM base dies.
Nvidia wants a standard NVHBM implementation available from multiple memory suppliers. That should reduce the engineering effort typically required to integrate and qualify custom memory across different vendors.
Amazon Annapurna Labs vice president Nafea Bshara said: “NVHBM represents a new architectural approach to advancing high-bandwidth memory performance and efficiency,” and “We look forward to this technology collaboration to benefit future AWS infrastructure designs.”
Amazon’s Annapurna Labs will be one of the first partners to use Nvidia’s NVHBM, tying it into NVLink scale-up architecture for AI workloads. Future AWS Trainium chips, starting with Trainium4, will use NVLink Fusion to connect Nvidia GPUs with Amazon chips under a common rack-scale architecture.
Nvidia has already talked up custom HBM for its Feynman GPUs, so 2028 looks like the likely first stop for NVHBM. By then, the AI memory wall may have become less of a wall and more of a very expensive toll booth.
