📞

AI Telephone

LLM Interoperability Demo

← Back to home

Round Details

March 9, 2026 at 12:00 AM UTC

Scenario: Asthma - Albuterol + Montelukast

Emily Chen, 28-year-old female, has mild intermittent asthma since childhood. She uses Albuterol 90mcg inhaler, 2 puffs as needed for shortness of breath, typically 2-3 times per week. She also takes Montelukast 10mg daily for maintenance. Her last pulmonary function test showed FEV1 of 92% predicted. No known drug allergies.

How did the LLMs do?

Same LLM Tests

GeminiGemini

Reconstructed text

Emily Chen, a female born in 1995, has no known drug allergies. She has an active diagnosis of mild intermittent asthma since childhood. A recent pulmonary function test revealed an FEV1 at 92% of predicted. To manage her condition, she is currently prescribed an albuterol 0.09 mg/actuation inhaler (2 puffs as needed for shortness of breath) and montelukast 10 mg oral tablets (10mg daily).

Comparison6 issues

Missing information:

  • albuterol usage frequency of typically 23 times per week
  • montelukast described as being for maintenance
  • patient age of 28 years is not stated

Hallucinated:

  • birth year 1995 added original only states age 28
  • montelukast formulation specified as oral tablets not stated in original
  • birth year 1995 was added which was not stated in the original scenario
ClaudeClaude

Reconstructed text

Emily Chen is a 28-year-old female with a confirmed diagnosis of mild intermittent asthma, uncomplicated, with onset in childhood. The condition is currently active. She has no known allergies.

Pulmonary function testing shows an FEV1 of 92% predicted, which is interpreted as normal.

Her current medication regimen includes two active prescriptions for asthma management:

  1. Albuterol 0.09 MG/ACTUAT Metered Dose Inhaler – 2 puffs as needed for shortness of breath (dyspnea), typically 2–3 times per week, administered via oral (inhalation) route.
  2. Montelukast 10 MG Oral Tablet – 10 mg once daily by mouth for maintenance therapy.

In summary, Ms. Chen has well-controlled mild intermittent asthma since childhood, supported by normal spirometry findings. She is maintained on daily montelukast with a rescue albuterol inhaler used on an as-needed basis. No known drug allergies are documented.

Comparison8 issues

Hallucinated:

  • added uncomplicated descriptor to asthma diagnosis not present in original
  • added currently active status not stated in original
  • allergy status was broadened from no known drug allergies to no known allergies
  • asthma was described as wellcontrolled which was not stated in the original scenario
  • fev1 of 92 predicted was additionally interpreted as normalnormal spirometry findings which was not explicitly stated in the original scenario
  • asthma characterized as uncomplicated
  • fev1 92 predicted is interpreted as normal
  • montelukast specified as oral tablet
MistralMistral

Reconstructed text

Emily Chen is a 28-year-old female (DOB: May 15, 1996) with a confirmed diagnosis of mild intermittent asthma, first identified at 5 years of age. Her most recent spirometry evaluation (May 1, 2024) demonstrated a normal FEV1/FVC ratio of 92%, indicating well-controlled airflow.

For asthma management, she is prescribed:

  • Albuterol 90mcg inhaler2 puffs as needed for shortness of breath, with a maximum of 2 puffs per week.
  • Montelukast 10mg oral tablet10mg daily for maintenance therapy.

She has no known drug allergies on record.

Comparison9 issues

Missing information:

  • albuterol usage frequency of typically 23 times per week
  • fev1 of 92 predicted

Hallucinated:

  • age of asthma onset specified as 5 years of age original only states since childhood
  • fev1 of 92 predicted changed to fev1fvc ratio of 92 these are different pulmonary measurements
  • albuterol usage changed from typically 23 times per week to maximum of 2 puffs per week
  • date of birth may 15 1996 added with no basis in original
  • spirometry evaluation date of may 1 2024 fabricated
  • dob may 15 1996 was added but not present in the original
  • added interpretation indicating wellcontrolled airflow
ChatGPTChatGPT

Reconstructed text

Emily Chen is a female patient born in 1998. Her active, confirmed problem list includes mild intermittent asthma (mild severity; ICD-10-CM J45.20, uncomplicated), with onset in childhood. For asthma management, she has an active medication order for albuterol 90 mcg/actuation inhaler, instructed as inhale 2 puffs by inhalation as needed for shortness of breath, typically used 2–3 times per week. She also has an active medication order for montelukast 10 mg oral tablet, instructed as take 10 mg by mouth once daily for maintenance. A final observation documented FEV1 percent predicted at 92%, interpreted as normal. Allergy documentation indicates an active, confirmed status of no known drug allergies.

Comparison13 issues

Missing information:

  • age explicitly stated as 28 years old
  • fev1 result was from a pulmonary function test reconstruction refers to it as a final observation
  • age 28yearold
  • pulmonary function test

Hallucinated:

  • birth year 1998 fabricated original only states age 28
  • icd10cm code j4520 fabricated not in original
  • fev1 interpretation as normal fabricated original only states 92 predicted without interpretation
  • birth year 1998 was added the original only states that the patient is 28 years old
  • fev1 of 92 predicted was additionally interpreted as normal this interpretation was not stated in the original
  • birth year 1998 added
  • icd10cm j4520 added
  • montelukast formulation specified as oral tablet
  • fev1 result interpreted as normal

Cross LLM Tests

GeminiMistral

Reconstructed text

Emily Chen, a 28-year-old female, has a history of mild intermittent asthma since childhood, which remains active. Her most recent pulmonary function testing demonstrated an FEV1 of 92% predicted, indicating well-preserved lung function.

For asthma management, she is prescribed:

  • Albuterol 90mcg inhaler, with instructions to use 2 puffs as needed for shortness of breath, typically 2–3 times per week.
  • Montelukast 10mg orally once daily for maintenance therapy.

She has no known drug allergies.

Comparison

Complete

No discrepancies found by any verifier.

ChatGPTGemini

Reconstructed text

Emily Chen, a female born in 1998, has no known drug allergies. She has an active, confirmed diagnosis of mild intermittent asthma that has been present since childhood. A recent pulmonary function test revealed a normal FEV1 of 92% predicted. To manage her condition, she is currently prescribed a Montelukast 10 mg oral tablet to be taken daily for maintenance. Additionally, she has an active prescription for an Albuterol 90 mcg inhaler, with instructions to inhale 2 puffs as needed for shortness of breath, typically 2 to 3 times per week.

Comparison6 issues

Missing information:

  • explicit age of 28 years old

Hallucinated:

  • birth year 1998 fabricated not stated in original and may not accurately correspond to age 28
  • montelukast routeform specified as oral tablet not stated in original
  • fev1 characterized as normal not stated in original
  • diagnosis described as active confirmed not stated in original
  • birth year 1998 added
MistralClaude

Reconstructed text

Emily Chen is a 28-year-old female (date of birth: May 15, 1996) with no known drug allergies. She has a confirmed, active diagnosis of mild intermittent asthma, with onset at age 5.

Her current medications include albuterol 90 mcg/actuation inhaler, 2 puffs as needed for shortness of breath, and montelukast 10 mg oral tablet taken once daily for maintenance therapy.

Pulmonary function testing performed on February 15, 2024, showed an FEV1 of 92% predicted, which was interpreted as normal.

Comparison7 issues

Missing information:

  • albuterol usage frequency of typically 23 times per week

Hallucinated:

  • date of birth may 15 1996 added with no basis in the original
  • asthma onset specified as age 5 whereas original states since childhood without a specific age
  • pulmonary function test date of february 15 2024 fabricated original does not specify a date
  • fev1 interpretation of normal added original does not include an interpretation
  • fev1 result was additionally interpreted as normal which was not stated in the original
  • interpretation of pft results as normal added
ClaudeChatGPT

Reconstructed text

Emily Chen is a female patient born in 1997. She has active, confirmed mild intermittent asthma (ICD-10 J45.20: mild intermittent asthma, uncomplicated), with onset in childhood. Her allergy record documents an active, confirmed status of no known drug allergies.

For asthma management, she has an active medication order for albuterol 0.09 mg/actuation metered dose inhaler, prescribed for her asthma, with instructions to use 2 puffs as needed for shortness of breath (dyspnea), typically 2 to 3 times per week. She also has an active medication order for montelukast 10 mg oral tablet for asthma maintenance, dosed at 10 mg once daily.

Pulmonary function testing has been completed with final results. A pulmonary function test panel was reported as final, including FEV1 percent predicted. Her FEV1 % predicted was 92%, interpreted as normal.

Comparison9 issues

Missing information:

  • explicit age of 28 years old

Hallucinated:

  • fev1 92 predicted interpreted as normal no such interpretation is stated in the original
  • birth year 1997 fabricated original only states age 28
  • birth year 1997 was added the original only states that she is 28 years old
  • icd10 code j4520 and the qualifier uncomplicated were added these were not stated in the original
  • birth year 1997 added
  • icd10 code j4520 added for asthma
  • montelukast formulation specified as oral tablet
  • fev1 result interpretation as normal added