On tough diagnostic cases pulled from the New England Journal of Medicine, Microsoft's AI system nailed the correct diagnosis 83% of the time. Generalist doctors working through the same case files managed only 36% — less than half. The numbers didn't come from a company blog post this time; they're in Science, one of the most demanding peer-reviewed journals in the world.
The system is called MAI-DxO, and it isn't a single model but an orchestrator that pits several AI systems against each other like a panel of doctors arguing over a chart. One proposes a diagnosis, another checks it against the symptoms, a third pushes for an extra test before the group settles on an answer. The lineup included OpenAI's GPT-5 and GPT-5.5, Google's Gemini 2.5 Flash, plus models from Anthropic, Meta and DeepSeek.
The gains varied a lot by model. GPT-5.5 scored 72% on its own and jumped to 87% once wrapped in the orchestrator, while trimming roughly $3,000 in unnecessary testing per case. Gemini 2.5 Flash climbed from 46.4% to 69%. GPT-5, by contrast, gained only about four percentage points — a sign the orchestration trick doesn't help every model equally.
Microsoft AI chief Mustafa Suleyman called the result a step toward 'medical superintelligence' — AI reasoning that could outperform even the strongest human-led health systems. That's a big claim, and the study itself flags a real limitation: the comparison group was generalist physicians, not the specialists or full medical teams who normally handle cases this complicated. On emergency-room style cases, the AI showed no meaningful edge at all.
For now this is a lab simulation run on written case descriptions, not actual patients. The next test is real: Microsoft has lined up a partnership with Beth Israel Deaconess Medical Center to trial MAI-DxO in genuine urgent-care settings with consenting patients. The gap between guessing a diagnosis from a written history and working a real emergency room could end up a lot wider than the gap between 83% and 36%.


