Embarking on a strategic journey to break free from GPU shortages, Microsoft dives into AI chips by designing Microsoft custom AI chip, revealing two groundbreaking solutions at its 2023 Ignite conference.
The GPU Conundrum
The backbone of many AI models, including ChatGPT and GPT-4 Turbo, is GPUs’ parallel computation capabilities. However, with a global shortage leaving Nvidia’s flagship AI cards sold out until 2024, Microsoft chooses to chart its way.
Microsoft’s In-House Solutions
Revealing the Azure Maia 100 AI Accelerator and the Azure Cobalt 100 CPU at Ignite, Microsoft introduces two cutting-edge chips. While the Maia 100 is custom-designed for training and running AI models, the Cobalt 100 is designed for general-purpose workloads.
Scott Guthrie’s Vision
Scott Guthrie, Microsoft Cloud and AI Group EVP emphasizes the company’s commitment to building infrastructure supporting AI innovation. He states, “At the scale we operate, we need to optimize and integrate every layer of the infrastructure stack.”
Hardware of Microsoft Custom AI Chip
Both chips are set to roll out in Azure data centers early next year, initially powering Microsoft AI services such as Copilot and Azure OpenAI Service. Microsoft assures that these chips are not one-offs, with second-generation Maia and Cobalt hardware already in development.
Maia 100
The Maia 100, a 5-nanometer chip housing 105 billion transistors, is covered in secrecy regarding its architecture. Microsoft claims it is specifically engineered for the Azure hardware stack, promising maximum utilization and powering extensive internal AI workloads.
Cobalt 100
Details about Cobalt 100 are more explicit—it’s an energy-efficient, 128-core chip built on an Arm Neoverse CSS architecture. Microsoft aims to deliver greater efficiency and performance in cloud-native offerings.
The Origin of Microsoft Custom AI Chip
Microsoft’s charge into this new field with Microsoft custom AI chip is not a sudden revelation. The company initiated this journey in 2019 under the project codenamed Athena. Reports in 2020 hinted at Microsoft designing ARM-based chips for data centers and consumer hardware. OpenAI, Microsoft’s close AI partner, played a role in shaping the design of Maia 100. The collaboration builds on their history of co-designing Azure’s AI infrastructure and signifies a deeper integration of AI innovations.
Navigating Challenges
Microsoft faced challenges in implementing these microsoft custom AI chips, necessitating the creation of custom data center server racks and a unique liquid-based cooling solution for Maia 100. The larger physical package and higher power consumption demanded a fresh approach to infrastructure.
Motivation Behind the Move
Microsoft’s pursuit of custom AI chips stems from the shortage and indispensability of GPUs, disrupting AI progress and incurring high costs. By bringing chip design in-house, Microsoft aims to enhance competitiveness, control costs, and overcome dependency on external vendors like Micron AI chips.
Future Prospects
As Microsoft dives into the silicon realm, it contains confidence in providing customers with optimal experiences. The company sees these innovations as instrumental in shaping the future of AI workloads on Azure, dealing with diverse customer needs. In a landscape dominated by GPU shortages, Microsoft’s venture into its Microsoft custom AI chip could reshape the dynamics of AI model development and deployment, offering a glimpse into a future where control and innovation converge.
