AegisDx: AI framework for safe diagnostics outperformed GPT-5 in physicians’ evaluation

A research group published a description of AegisDx on arXiv in July 2026 — a framework for safe differential diagnosis based on large language models. In a blind trial of 43 real patient records from Yale New Haven Health System's emergency department, AegisDx received an average safety score of 4.55 out of 5 from physicians compared to 4.31 for GPT-5 — a statistically significant difference (adjusted p = 2.1×10⁻⁴).

Why Ordinary LLM Is Not Enough for Safe Diagnosis

Diagnostic errors remain one of the major threats to patient safety. Modern LLM systems typically approach diagnosis as a "single query" task: a doctor describes symptoms, the model provides an assumption. This approach has no protection against missing life-threatening conditions and does not verify its own reasoning.

AegisDx is structured fundamentally differently. The authors call their approach hypothetico-deductive clinical reasoning: the system first generates a broad list of hypotheses, then successively narrows it, checking each step against the medical evidence base. Key architectural elements:

• Role contracts: each LLM component is strictly responsible for its task
• Structured intermediate conclusions: each step of reasoning is explicitly recorded
• Evidence search interfaces: hypotheses are verified against medical literature in real time
• Verification gates: the final diagnosis is checked before output
• Mandatory must-not-miss screening: myocardial infarction, pulmonary embolism, aortic dissection, meningitis — the system must check them explicitly, accidental omission is impossible

What Three Levels of Testing Showed

The authors compared AegisDx with GPT-oss-120B used directly without additional constraints. For each group of cases, the primary metric was Top-3 accuracy — the correct diagnosis being in the top three predictions, corresponding to real clinical practice.

• JAMA cases: Top-3 accuracy 59.9% versus 52.1%
• NEJM cases: Top-3 accuracy 62.7% versus 51.4%
• Annals of Emergency Medicine: 85.7% versus 68.6%
• Critical must-not-miss diagnoses: AegisDx included at least one such condition in the top-3 in 78.0% of cases versus 52.0% for standalone LLM

Most stringent was the blind test with practicing clinicians: 43 real records from the emergency department were compared blind to GPT-5 conclusions. Participants rated AegisDx higher on an aggregate safety measure — 4.55 versus 4.31 on a five-point scale. Qualitatively, they noted improvement in identifying life-threatening diagnoses and greater transparency in reasoning.

What This Means

The gain from structured framework proved comparable to direct model scaling — and the authors consider this the main finding. Engineering diagnostic AI systems as safe reasoning pipelines can be more productive than chasing raw predictive accuracy — especially in emergency medicine, where diagnostic error costs lives. AegisDx is positioned not as a replacement for doctors, but as a transparent decision-support tool at the bedside.

Frequently Asked Questions

What Language Model Does AegisDx Use?

In experiments, the authors used GPT-oss-120B as the base model for all components. The framework architecture is not tied to any particular LLM and was designed as universal.

What Clinical Material Was Used for Testing?

Literature cases were taken from NEJM, JAMA, and Annals of Emergency Medicine. The blind trial with physicians was conducted on 43 real patient records from Yale New Haven Health System's emergency department.