History
Monthly snapshot of same-LLM roundtrip fidelity across clinical scenarios. Each cell shows the number of issues found when the same LLM encodes and decodes.
Monthly snapshot of same-LLM roundtrip fidelity across clinical scenarios. Each cell shows the number of issues found when the same LLM encodes and decodes.
| Scenario | Claude | Gemini | ChatGPT | Mistral | Claude→ChatGPT | Gemini→Mistral | ChatGPT→Gemini | Mistral→Claude |
|---|---|---|---|---|---|---|---|---|
| Asthma - Albuterol + Montelukast | 8 issues | 6 issues | 13 issues | 9 issues | 9 issues | 0 issues | 6 issues | 7 issues |
| Chronic Kidney Disease - Multiple Medications | 13 issues | 9 issues | 7 issues | 14 issues | 10 issues | 5 issues | 10 issues | 16 issues |
| Complex Appointment Booking - Coordinated Care | 12 issues | 15 issues | 4 issues | 19 issues | 9 issues | 15 issues | failed | 31 issues |
| Dental Extraction - Post-Op Care | 11 issues | 19 issues | 12 issues | 21 issues | 20 issues | 11 issues | 11 issues | 20 issues |
| Hypertension - Lisinopril | 5 issues | 5 issues | 7 issues | 24 issues | 11 issues | 7 issues | 5 issues | 17 issues |
| Pregnancy - Prenatal Care | 6 issues | 5 issues | 7 issues | 27 issues | 11 issues | 5 issues | 4 issues | 18 issues |
| Type 2 Diabetes - Metformin | 14 issues | 5 issues | 9 issues | 12 issues | 14 issues | 6 issues | 5 issues | 17 issues |
| Scenario | Claude | Gemini | ChatGPT | Mistral | Claude→ChatGPT | Gemini→Mistral | ChatGPT→Gemini | Mistral→Claude |
|---|---|---|---|---|---|---|---|---|
| Asthma - Albuterol + Montelukast | 4 issues | 9 issues | 5 issues | 7 issues | 13 issues | 4 issues | 8 issues | 13 issues |
| Chronic Kidney Disease - Multiple Medications | 8 issues | 6 issues | 15 issues | 8 issues | 16 issues | 3 issues | 11 issues | 14 issues |
| Complex Appointment Booking - Coordinated Care | 15 issues | 16 issues | 21 issues | 19 issues | 25 issues | 9 issues | 16 issues | 22 issues |
| Dental Extraction - Post-Op Care | 13 issues | 17 issues | 9 issues | 12 issues | 29 issues | 16 issues | 9 issues | 11 issues |
| Hypertension - Lisinopril | 10 issues | 8 issues | 13 issues | 10 issues | 13 issues | 9 issues | 10 issues | 13 issues |
| Pregnancy - Prenatal Care | 5 issues | 8 issues | 12 issues | 7 issues | 12 issues | 3 issues | 8 issues | 25 issues |
| Type 2 Diabetes - Metformin | 8 issues | 4 issues | 8 issues | 7 issues | 16 issues | 6 issues | 11 issues | 11 issues |