Google races marvell on ai chip duel

Google is in talks with semiconductor giant Marvell Technology to develop two new AI chips, a strategic move to turbocharge its inference capabilities – essentially, the real-world application of trained AI models.

A new era for ai performance

This isn’t about theoretical breakthroughs; it’s about raw speed and efficiency. Google’s aiming to drastically improve the performance of its AI models in generating actionable insights from data, moving beyond the training phase. Frankly, it’s a race to deliver tangible value faster.

The first chip will be a memory-optimized design, working in tandem with Google’s Tensor Processing Units (TPUs) – already a cornerstone of their AI infrastructure. The second? A bespoke TPU specifically engineered for inference tasks. This is a calculated expansion of their existing partnerships with Broadcom, MediaTek, and TSMC.

Last year, Google’s TPUs, accelerated AI processors designed for model training, hit the seventh generation with Ironwood, available since November. Ironwood boasts a tenfold performance increase over v5p and over four times the performance-per-chip for both training and inference compared to v6e. Think 9,216 chips crammed into a ‘superpod’ – a massive AI supercomputer connected via Google’s ICI network, moving data at a staggering 9.6Tb/s. That’s enough bandwidth to generate a 1.77 Petabyte pool of high-bandwidth memory, far surpassing traditional RAM.

But Google isn’t alone in recognizing the importance of inference. Microsoft, for instance, recently unveiled Maia 200, an AI accelerator promising three times the FP4 performance of Amazon’s Trainium and superior FP8 performance to Google’s 7th generation TPU. It’s a clear indication that the competition is heating up.

The bottom line? This pursuit of optimized inference isn’t a trend; it’s a fundamental shift. Google’s commitment to this Technology suggests a future where AI doesn’t just learn – it acts, and it does so with unprecedented speed and scale.