Microsoft has unveiled Maia 300, an inference accelerator that the company says will make its own silicon the primary engine behind Copilot, Azure OpenAI and its in-house models from 2027. The headline claims: 40% more tokens per megawatt than leading GPU-based systems and up to seven times the rack-scale token throughput of its predecessor, Maia 200 — numbers that, if they hold, would redirect a meaningful share of Microsoft’s AI spending away from Nvidia.
- Efficiency-first design: eight HBM4 stacks, 288 GB of capacity and 18 TB/s of bandwidth per accelerator, built on TSMC’s N2P process with a chiplet layout.
- Rack-scale ambition: 96 accelerators in a 167 kW rack, linked by an Ethernet fabric that spans up to 12,288 chips in a single superpod.
- Roadmap: multi-gigawatt Maia rollouts begin in 2027, with Maia becoming Microsoft’s primary inference platform.
- Verification pending: analysts warn the efficiency figures remain vendor claims until reproducible benchmarks appear.
Inside the rack: memory bandwidth takes centre stage
Maia 300’s architecture is unapologetically memory-centric. Microsoft’s technical note frames it as the second generation of its data-centre inference silicon, while launch coverage counts it as the third Azure Maia accelerator; either way it succeeds Maia 200, which entered production in January 2026 and now runs Copilot workloads, with GitHub Copilot and Microsoft Foundry due to follow by the end of the year. Against that chip, Microsoft cites 36% more compute FLOPs, 2.5x the HBM bandwidth, 33% more memory capacity and roughly double the bandwidth per watt. Each accelerator carries eight HBM4 stacks holding 288 GB at 18 TB/s, with compute on TSMC N2P and memory-controller and I/O functions split onto separate dies. At 950 W per chip, the reference rack packs 96 accelerators into 24 four-way trays for a 167 kW envelope, and a two-tier Ethernet fabric joins up to 128 racks into a 12,288-accelerator superpod without a separate scale-out network. Enhanced MXFP4 and sub-3-bit formats trim the bytes moved per token, while memory sits in a NUMA-like layout so software keeps data near the compute using it.
Power, not silicon, is the binding constraint
Analysts are already arguing over those efficiency figures. Stephen Sopko, practice lead for semiconductor and deep tech at HyperFrame Research, calls power the right scoreboard, but warns the metric is only as trustworthy as the assumptions behind it.
“It’s the right scoreboard, because power, not silicon, is now the binding constraint on AI buildouts. But the number is only as good as its assumptions.”
Sopko notes that tokens-per-megawatt comparisons hinge on the GPU baseline, the models in use, latency targets and whether power is measured at rack or facility level, and that Microsoft has not published reproducible results. Matt Kimball of Moor Insights & Strategy is less fixated on the precise percentage: vendors use different methodologies, and even a modest gain compounds at data-centre scale. His read is that much of Maia’s advantage may come from packing more accelerators into a fixed power envelope rather than from an intrinsically faster chip. He is also clear that Maia is not a bid to push Nvidia out of Azure, describing the design as vertical integration at its best, with silicon, memory, networking and software tuned together for Microsoft’s own services. At the stated 167 kW per rack, one gigawatt would accommodate roughly 575,000 accelerators before facility overhead — an illustrative calculation that shows the scale Microsoft is chasing.
What Maia means for Nvidia and for Azure customers
Maia 300 targets inference, not training, so Nvidia remains the default for building frontier models; the difference is leverage. Microsoft has committed about $80 billion to AI infrastructure across 2025-2027, and if deployment targets are met the fleet could exceed one million Maia accelerators — against roughly 400,000 Nvidia H100 units sold in 2024. Amazon’s Trainium and Inferentia and Google’s TPUs follow the same playbook of reducing, rather than eliminating, dependence on merchant silicon. The cost pressure is real: Copilot’s economics depend on inference staying affordable at hundreds of millions of users, and Jim Kleewein, technical fellow for Microsoft 365, has described Copilot and Cowork summarising, composing and completing multi-step tasks on Maia 200, with responsiveness data feeding back into the new design. Azure customers do not rent Maia instances, however — they consume the services it powers, and analysts want evidence that efficiency gains reach them through pricing or capacity.
Maia 300 is a negotiating instrument as much as a chip. It will not displace Nvidia in training any time soon, and every efficiency figure still originates with Microsoft, with no independent benchmark published. What it does give the company is a credible internal alternative for the fastest-growing slice of AI compute and a lever for resetting GPU terms. If the multi-gigawatt rollout lands on schedule, the more consequential shift will be in how the industry prices inference, not just in how Microsoft fills its data centres.