MS to scale up its in-house AI chip rollout to multi-GW levels starting next year, boosting AI inference efficiency

[Financial News] Microsoft (MS) will expand the supply of its in-house artificial intelligence (AI) accelerator Maia to multi-gigawatt (GW) levels starting next year. The move is aimed at meeting surging inference demand as AI evolves beyond answering questions into agents that perform complex tasks. MS plans to improve the power efficiency and data-processing capabilities of its next-generation AI chips, strengthening its competitiveness in enterprise AI infrastructure as well as in its own services, including Copilot.■ Maia 200 enters full-scale use, delivering up to 1.7 times the power efficiency of GPUsMS said on the 13th that it would expand deployment of Maia, its family of in-house AI accelerators for its Azure cloud platform, to multi-GW levels starting in 2027. The company plans to use its self-developed AI accelerators as inference infrastructure for major AI services.
The second-generation product, Maia 200, is currently in operation and is being used in major services such as MS Copilot and Teams. It runs OpenAI models, MS's own MAI models, and leading open-source AI models. By year-end, it is also expected to be deployed for GitHub Copilot and Microsoft Foundry, the company's enterprise AI development platform.
Maia 200 has already been deployed in the United States, Europe and Australia. According to MS, when processing AI functions in live services such as Teams and Copilot, it handles up to 1.7 times as many tokens per watt as leading GPU-based infrastructure. This means it can process more AI requests using the same amount of power.
Expanding its in-house AI accelerators also aligns with MS's strategy of optimizing AI infrastructure as an integrated whole, from semiconductors to software and services. MS supports a range of AI services by running not only its own AI models but also OpenAI and open-source models on Maia. The aim is to optimize infrastructure for the needs of each service without being tied to a particular model.
■ Next-generation Maia 300 targets the AI agent marketBuilding on this operational experience, MS plans to further improve the inference efficiency of its next-generation product, Maia 300. Maia 300 is a third-generation accelerator optimized for AI agents that analyze lengthy information or carry out tasks in multiple steps. It was designed with the understanding that data movement and memory use increase as AI retains longer conversations and performs more complex tasks.
MS expects Maia 300 to process 40% more tokens per megawatt (MW) than leading GPU-based systems. A rack-scale system equipped with 96 accelerators is designed to deliver up to seven times the token throughput of the previous generation, Maia 200. These figures are target performance levels presented by MS, not results from actual operation.
To achieve this, Maia 300 focuses not only on compute performance but also on minimizing data movement. Software controls data flows to reduce unnecessary transfers and make efficient use of compute, memory and network resources.
The hardware also incorporates the latest technologies. Maia 300 is equipped with eight fourth-generation high-bandwidth memory (HBM4) units and provides memory bandwidth of up to 18 terabytes (TB) per second within a 950-watt thermal design power (TDP) envelope. It uses a chiplet architecture that separates compute, memory control and input/output functions, with its compute chiplets manufactured using Taiwan-based TSMC's 2-nanometer-class N2P process.
The rack-scale system integrates 96 accelerators within a power limit of 167 kilowatts (kW). The entire rack has 27.6 TB of HBM capacity and memory bandwidth of 1.7 petabytes (PB) per second. MS expects this to enable more AI requests to be handled simultaneously within the power and space constraints of data centers.
Maia 300 is scheduled to be deployed in live service environments starting in 2027. MS plans to share further details on its AI infrastructure strategy and progress in adopting related technologies at Microsoft Ignite 2026, which will be held in San Francisco, United States, this November.


[email protected] Choi Hye-rim Reporter