On October 22, Microsoft AI shared the results of its diagnostic system MAI-DxO (Diagnostic Orchestrator) in a study published in the peer-reviewed journal Science. Mustafa Suleyman, who heads the unit, describes it as a step toward the goal of "medical superintelligence".
What happened?
The system was tested in a simulated setting on complex case reports published in the New England Journal of Medicine. According to TechCrunch and The Verge, MAI-DxO reached the correct diagnosis in about 83 percent of these cases. General practitioners and internists who could consult resources scored 36 percent, and those who used no resources scored 22 percent. On emergency room cases drawn from Beth Israel's patient admission database, the result was 80 percent against an average of 47 percent.
MAI-DxO is not a single model; it is an orchestrator that makes different AI models work together like a team of physicians. Microsoft says this setup also lowers costs compared with a model working alone.
TechCrunch points out an important caveat: according to the study's own data, most of the improvement comes from the underlying models getting stronger, Microsoft's system adds a further gain on top of that, and this extra gain was not seen in emergency room cases. Also, cases this hard normally go to specialists; specialists were not included in the comparison.
Why does it matter?
Being published in a peer-reviewed journal puts the results on firmer ground than company announcements. Still, this is a simulation, not an experiment with real patients. Microsoft is training a foundation model for health with Mayo Clinic and is running a real-world trial at Beth Israel Deaconess Medical Center in Boston: MAI-DxO's recommendations are added to the care of patients who consent. The real question is whether these numbers will hold up in the exam room too.