Microsoft Maia 300: How the Hyperscalers' Custom-Chip Race Will Change Inference Costs

For a long time, Nvidia was the only name that mattered in AI cost debates. But the big cloud providers are leaning more and more on their own chips, especially for running models (inference). Maia 300, which Microsoft unveiled at the OCP Global Summit on October 12, is the newest and most ambitious example of that trend.

What we know about Maia 300

• It is the third generation of Microsoft's in-house AI accelerator, designed primarily for inference.
• Microsoft says it produces 40% more tokens per megawatt than leading GPU systems, but did not say which systems it compared against.
• A full rack of 96 accelerators delivers up to seven times the output of the previous-generation Maia 200.
• Production starts in 2027, with deployments expected at gigawatt scale — potentially more than a million chips that year.
• Reports say the chip carries 288 GB of HBM4 memory and will be built on TSMC's N2P process; these technical details have not been independently verified.

Not for rent

Maia 300 won't be offered to Azure customers as a separate product; Microsoft will use it as the main inference infrastructure for Copilot and its other services. Nvidia remains a key partner for model training. As of the announcement, no independent company had tested the chip, so the performance claims are, for now, the vendor's own numbers.

What it means for enterprises

• Per-token costs will diverge further across providers: those running their own chips can offer similar quality at different prices.
• Prices will move: as new hardware comes online, inference pricing will shift — today's cheapest option may not be the cheapest in six months.
• Lock-in risk grows: being tied to one provider's infrastructure can mean missing out on these price advantages.
• Measure for yourself: instead of vendor metrics like 'tokens per megawatt', track cost and latency per request on your own workloads.

Where remot fits

As the hardware race speeds up, inference pricing and performance will shift between providers more often. Because remot puts models behind a single API, it can route each request to the best provider based on live cost, latency and quality data, measures the real cost per request, and lets you shift traffic when a provider gets cheaper — without code changes. That way you capture the price gains of the chip race without tying yourself to a single provider.