📞

AI Telephone

LLM Interoperability Demo

← Back to home

Round Details

September 30, 2026 at 07:29 AM UTC

Scenario: Asthma - Albuterol + Montelukast

Emily Chen, 28-year-old female, has mild intermittent asthma since childhood. She uses Albuterol 90mcg inhaler, 2 puffs as needed for shortness of breath, typically 2-3 times per week. She also takes Montelukast 10mg daily for maintenance. Her last pulmonary function test showed FEV1 of 92% predicted. No known drug allergies.

How did the LLMs do?

Same LLM Tests

ChatGPT→ChatGPT

Reconstructed text

Emily Chen is a 28-year-old woman with confirmed, active mild intermittent asthma without complications, present since childhood. Her current asthma medications include an albuterol inhaler, 90 mcg per actuation, taken as 2 puffs by inhalation as needed for shortness of breath, typically used 2–3 times per week. She also takes montelukast 10 mg orally once daily for asthma maintenance. Her last pulmonary function test showed a forced expiratory volume in 1 second (FEV1) of 92% of predicted. She has no known drug allergies, as confirmed in the record.

Comparison1 issue

Hallucinated:

  • asthma described as without complications the original does not state this
Mistral→Mistral

Reconstructed text

Emily Chen, a 28-year-old female, has active mild intermittent asthma with onset in childhood. She is prescribed albuterol 90 MCG per actuation inhaler, 2 puffs as needed for shortness of breath, and montelukast 10mg daily. Her FEV1 is 92% of predicted value.

Comparison4 issues

Missing information:

  • albuterol usage frequency of typically 23 times per week
  • no known drug allergies nkda status
  • montelukast indication as maintenance therapy
  • fev1 source as her last pulmonary function test
Claude→Claude

Reconstructed text

Clinical Narrative

Emily Chen is a female patient born in 1997 (approximately 28 years old). She has no known drug allergies, and this has been confirmed.

Her problem list includes an active, confirmed diagnosis of mild intermittent asthma, uncomplicated (SNOMED CT 427679007; ICD-10-CM J45.20), present since childhood. Severity is documented as mild.

She has two active prescriptions for asthma:

  • Albuterol: 90 mcg/actuation metered-dose inhaler (200 actuations per inhaler). She takes 2 puffs by inhalation as needed for shortness of breath. Per her instructions, she typically uses it 2–3 times per week.
  • Montelukast: 10 mg oral tablet, taken once daily for maintenance.

Her most recent pulmonary function test showed an FEV1 of 92% of predicted. This result is final.

No appointments or scheduling information were included in the record.

Comparison5 issues

Hallucinated:

  • birth year 1997 added the original states only the age 28 years old
  • albuterol inhaler package size 200 actuations per inhaler added not in the original
  • asthma diagnosis qualified as uncomplicated and assigned codes snomed ct 427679007 icd10cm j4520 neither appears in the original
  • montelukast dosage form specified as oral tablet the original states only 10mg daily
  • pulmonary function test result designated as final the original does not specify a result status

Cross LLM Tests

ChatGPT→Mistral

Reconstructed text

Emily Chen is a 28-year-old female with a confirmed diagnosis of mild intermittent asthma since childhood. She has no known drug allergies. Her current medications include albuterol 90 mcg per actuation inhaler, used as 2 puffs as needed for shortness of breath (typically 2–3 times per week), and montelukast 10 mg taken daily for asthma maintenance. Her most recent pulmonary function test showed an FEV1 of 92% of predicted.

Comparison

Complete

No discrepancies found by any verifier.

Claude→ChatGPT

Reconstructed text

Emily Chen is a female born in 1997 with active, confirmed mild intermittent asthma, uncomplicated, documented on her problem list with onset in childhood. Her active asthma prescriptions include an albuterol 90 mcg/actuation metered-dose inhaler containing 200 actuations, with instructions to inhale 2 puffs as needed for shortness of breath. She reports typical albuterol use of 2–3 times per week. She is also prescribed montelukast 10 mg by mouth once daily as maintenance therapy for asthma. The finalized result from her most recent pulmonary function test shows an FEV1 of 92% predicted, interpreted as normal. Her allergy record confirms no known drug allergies.

Comparison7 issues

Missing information:

  • the patients stated age of 28 years

Hallucinated:

  • pft interpretation interpreted as normal added original gives only fev1 92 predicted
  • inhaler package size of 200 actuations added not in original
  • asthma qualifier uncomplicated added not in original
  • birth year 1997 added the original provides age 28 but no birth year or reference date
  • asthma described as uncomplicated the original does not specify complication status
  • albuterol inhaler specified as containing 200 actuations the original does not specify the number of actuations
Mistral→Claude

Reconstructed text

Clinical Narrative

Emily Chen is a female patient born in 1995 (approximately 30 years old). She has an active diagnosis of mild intermittent asthma, which began at age 5. She has no known medication allergies.

For her asthma, she has two active prescriptions. The first is an albuterol 90 mcg/inhalation inhaler, 2 puffs as needed for shortness of breath, which she typically uses 2–3 times per week. The second is montelukast sodium 10 mg oral tablet, taken once daily. A finalized examination recorded her forced expiratory volume in one second (FEV1) at 92%.

Comparison7 issues

Missing information:

  • fev1 is no longer described as a percent of predicted the original says 92 predicted the reconstruction says only 92
  • montelukasts role as maintenance therapy is not stated
  • fev1 was 92 of the predicted value the reconstruction omits predicted
  • the fev1 result was from her most recent pulmonary function test the reconstruction refers only to an examination

Hallucinated:

  • age changed from 28 to approximately 30 years old
  • birth year 1995 added which was not in the original
  • asthma onset given as age 5 the original says only since childhood