Microsoft AI Diagnostic System Outperforms Generalist Physicians in Complex Case Study

Microsoft’s new AI diagnostic system, MAI-DxO, correctly identified complex medical conditions in a simulation more than twice as often as generalist physicians, a result the company describes as a significant step toward medical superintelligence. Published in Science on Thursday, the study indicates that the system, which coordinates multiple AI agents to mimic a panel of doctors, reached an 83% accuracy rate on difficult cases sourced from the New England Journal of Medicine. In contrast, primary care doctors and general internists working individually with reference materials were correct in only 36% of those instances. While these cases are typically handled by specialists or teams, the disparity highlights the potential for AI to close the gap in expert medical reasoning.

The system’s performance relies on an orchestration framework that allows various AI models to debate diagnoses and order tests. Microsoft tested this framework using models from OpenAI, Google, Anthropic, Meta, and DeepSeek, finding that the best results were achieved with OpenAI’s GPT-5 and GPT-5.5. However, the study’s data suggests that much of the improvement stems from the power of these underlying models rather than the orchestration itself. For instance, GPT-5 alone scored 78.9% on complex cases at a cost of approximately $8,400 per case, while adding the DxO layer reduced the cost by about $3,000 but improved accuracy by only four points.

The benefits of the orchestration layer are more pronounced when using less capable models. For example, Gemini 2.5 Flash saw its accuracy jump from 46.4% to nearly 69% when paired with DxO. Despite these gains, the gap between weaker and stronger models remained larger than the boost provided by the system. Microsoft notes that accuracy improvements naturally diminish as models approach theoretical limits, and the orchestration is designed to optimize both cost and precision.

The company is now moving from simulation to real-world validation. Through a partnership with Beth Israel Deaconess Medical Center in Boston, Microsoft will evaluate DxO across different hospital settings, including emergency departments and urgent care. Patients will receive standard treatment while consenting to parallel engagement with the AI, allowing researchers to compare DxO’s recommendations against human physician decisions. Dominic King, vice president of Health at Microsoft AI, noted that this will be one of the largest studies of its kind, providing evidence on the system’s scalability.

Mustafa Suleyman, CEO of Microsoft AI, views this work as foundational infrastructure for a broader push into healthcare. He stated that the company is collaborating with the Mayo Clinic to train a specific foundation model for healthcare, with release planned for next year. Suleyman anticipates that healthcare models will become an order of magnitude more capable within the next two to three years, driven by a flywheel of real-world usage and iteration similar to what occurred in coding and chat applications.

This announcement places Microsoft in direct competition with Google and OpenAI, both of which have recently claimed AI capabilities that match or exceed human doctors. Google, for instance, published a study in The Lancet earlier this month evaluating its AMIE diagnostic AI in real-world urgent care visits. Microsoft’s strategy involves expanding its existing portfolio of dictation and radiology tools, aiming to become the premier provider of clinical decision support products. Suleyman believes that by 2027, the company will be on the verge of breakthroughs in processing medical records, allowing for highly accurate predictions of patient outcomes.