The automaker claims it has the fastest AI training machine in the world, but then we have come to take Tesla’s comments with a grain of salt over the years and in this case it is not exactly as if he switched on a lump of hardware.
For years now, Tesla has been teasing the development of a new supercomputer in-house optimised for neural net video training. Tesla is handling video data from its fleet of over a million vehicles, which it uses to train its neural nets.
The two year project involved the creation of Dojo, an exaflop supercomputer which can handle all this data and make it into pretty information. If Tesla is right about the computer’s power it will be the most powerful supercomputer in the world so we are not sure we are ready to believe that yet, particularly as the computer has not been built yet.
However there is some interesting tech under Dojo’s bonnet. Ganesh Venkataramanan, Tesla's senior director of Autopilot hardware and the leader of the Dojo project, led the presentation. The engineer started by unveiling Dojo's D1 chip, which is using 7 nanometer technology and delivers what is described as breakthrough bandwidth and compute performance.
Tesla designed the chip to "seamlessly connect without any glue to each other" and the automaker took advantage of that by connecting 500,000 nodes together. It adds the interface, power, and thermal management, and it results in what it calls a training tile.
The result is a 9 PFlops training tile with 36TB per second of bandwidth in a less than one cubic foot format. But now it still has to form a compute cluster using those training tiles in order to truly build the first Dojo supercomputer. Tesla hasn't put that system together yet, but CEO Elon Musk claimed that it will be operational next year.