📞

AI Telephone

LLM Interoperability Demo

← Back to home

Round Details

February 15, 2026 at 12:00 PM UTC

Scenario: Pregnancy - Prenatal Care

Sarah Johnson, 32-year-old female, G2P1 at 24 weeks gestation with uncomplicated pregnancy. She has a history of gestational diabetes in her previous pregnancy. Current glucose tolerance test is normal. She is taking Prenatal vitamins with folic acid 800mcg daily and Iron supplement 27mg daily. Blood type is O positive, rubella immune. Next ultrasound scheduled for anatomy scan.

How did the LLMs do?

Same LLM Tests

Claude→Claude

Reconstructed text

Sarah Johnson is a 31-year-old female currently in her second pregnancy (G2P1) at 24 weeks gestation. Her pregnancy is confirmed as uncomplicated and clinically active.

Obstetric History: Ms. Johnson has a history of gestational diabetes mellitus during her previous pregnancy, which resolved postpartum. Given this history, a glucose tolerance test was performed during the current pregnancy, with results interpreted as normal.

Prenatal Laboratory Findings:

  • Blood type: O positive (Rh(D) positive)
  • Rubella immunity status: Immune
  • Glucose tolerance test (1-hour post-dose glucose): Normal

Current Medications:

  1. Prenatal vitamins with folic acid 800 mcg, taken once daily
  2. Ferrous sulfate (iron supplement) 27 mg oral tablet, taken once daily

Both medications are actively prescribed in the context of her current pregnancy.

Upcoming Appointments: An obstetric ultrasound anatomy scan has been ordered and is currently booked. This prenatal anatomy scan will assess fetal anatomy via ultrasonography of the gravid uterus.

Summary: Ms. Johnson is progressing through an uncomplicated second pregnancy at 24 weeks gestation. Despite a history of gestational diabetes in her prior pregnancy, her current glucose tolerance testing is normal. Routine prenatal labs including blood typing and rubella immunity are reassuring. She is maintained on standard prenatal supplementation with prenatal vitamins and iron, and she has an anatomy scan ultrasound scheduled as her next appointment.

Comparison5 issues

Hallucinated:

  • added that gestational diabetes resolved postpartum not stated in original
  • iron supplement specified as ferrous sulfate original only says iron supplement
  • glucose tolerance test specified as 1hour postdose glucose not stated in original
  • age changed from 32 to 31
  • age changed from 32yearold original to 31yearold reconstruction
Mistral→Mistral

Reconstructed text

Sarah Johnson, a 33-year-old female with an unknown birthplace, is currently 24 weeks pregnant with an uncomplicated pregnancy. This is her second pregnancy (G2) and she has had one previous delivery (P1). She has a history of gestational diabetes mellitus in a previous pregnancy, but this condition is currently inactive. Her recent glucose tolerance test on October 10, 2023, was normal. Sarah is prescribed prenatal vitamins with folic acid 800 mcg and iron 27 mg, both to be taken as one tablet daily. Her blood type is O positive, and she is immune to rubella. An anatomy ultrasound has been ordered for her. Her pregnancy status was confirmed on October 15, 2023.

Comparison7 issues

Missing information:

  • next ultrasound is scheduled original specifies it is scheduled for an anatomy scan reconstruction only states it was ordered

Hallucinated:

  • age changed from 32 to 33
  • unknown birthplace added not mentioned in original
  • glucose tolerance test date of october 10 2023 fabricated
  • pregnancy status confirmation date added october 15 2023 not in original
  • statement that prior gestational diabetes is currently inactive added original only notes history of gestational diabetes in prior pregnancy
  • medication instruction changedadded both to be taken as one tablet daily original only states prenatal vitamins with folic acid 800 mcg daily and iron 27 mg daily no tablet formulation specified
Gemini→Gemini

Reconstructed text

Sarah Johnson is a female patient born in 1991 who is currently presenting with an active normal pregnancy. Her estimated gestational age is 24 weeks, and her obstetric history is Gravida 2, Para 1 (2 maternal pregnancies, 1 parity). She has a previously resolved condition of gestational diabetes mellitus, which is supported by a normal result on a glucose tolerance test.

Laboratory findings indicate that her blood type is O positive and she is immune to the Rubella virus. Her current active medications include a daily prenatal vitamin with 800mcg of folic acid (taken as 1 pill daily) and an iron supplement (27mg daily). Additionally, there is an active service request planned for a fetal anatomy ultrasound scan.

Comparison8 issues

Missing information:

  • patient age stated as 32 years old
  • specific detail that gestational diabetes occurred in her previous pregnancy

Hallucinated:

  • birth year 1991 fabricated original only states age 32
  • taken as 1 pill daily for prenatal vitamin is fabricated not specified in original
  • year of birth added as 1991 not provided in the original scenario
  • gestational diabetes characterized as a previously resolved condition original only states a history of gestational diabetes in a prior pregnancy and that the current glucose tolerance test is normal without stating resolution
  • birth year 1991 added
  • prenatal vitamin instruction taken as 1 pill added
ChatGPT→ChatGPT

Reconstructed text

Sarah Johnson is a female patient born in 1994. She is currently pregnant with an active, confirmed uncomplicated/normal pregnancy (problem list item; SNOMED CT 237238006 “Normal pregnancy,” text noted as “Uncomplicated pregnancy”). Her obstetric history documents Gravida 2 and Para 1. She also has a confirmed but inactive problem list history of gestational diabetes mellitus (SNOMED CT 11687002), noted to have occurred in a previous pregnancy (G2P1).

At the time of the recorded exam, her estimated gestational age is 24 weeks. Laboratory/screening observations include a glucose tolerance test interpreted as normal, blood type O positive, and rubella immunity present (IgG positive/immune).

Current active medications include prenatal vitamins with folic acid 800 mcg, instructed as 1 dose by mouth daily, and an iron supplement 27 mg, instructed as 27 mg by mouth daily.

An active imaging ServiceRequest has been placed for an obstetric ultrasound (anatomy scan), with the reason linked to the current pregnancy condition. An associated ultrasound appointment is booked, described as a “Scheduled ultrasound appointment for fetal anatomy scan,” with the patient listed as a participant and accepted. (No appointment date/time or location is provided in the data.)

Comparison12 issues

Missing information:

  • patient age explicitly stated as 32 years old
  • age 32yearold

Hallucinated:

  • birth year 1994 fabricated not in original and inconsistent with stated age of 32 which would correspond to 1992 or 1993
  • appointment status described as accepted is fabricated not mentioned in original
  • birth year added born in 1994 original only states age 32 with no dobbirth year
  • gestational diabetes history is described as occurring in a previous pregnancy g2p1 which is internally inconsistent a previous pregnancy would be g1p1 g2p1 reflects current obstetric status and this parenthetical detail is not in the original
  • adds problemliststatus qualifiers and coding details not present in the original eg active confirmed normal pregnancy confirmed but inactive gestational diabetes snomed ct codes
  • adds lab detail rubella immunity present igg positiveimmune original only states rubella immune without igg result
  • adds imaging workflow details not present in the original an active imaging servicerequest an appointment is booked patient participation and status accepted
  • birth year 1994 added
  • rubella igg positive added original only stated rubella immune
  • route by mouth added for prenatal vitamins and iron supplement

Cross LLM Tests

Gemini→Mistral

Reconstructed text

Sarah Johnson is a 32-year-old female patient with an uncomplicated pregnancy at an estimated gestational age of 24 weeks. She is gravida 2, para 1, indicating this is her second pregnancy and she has one previous delivery. Her medical history includes resolved gestational diabetes. Current laboratory results show normal glucose tolerance, blood type O positive, and immunity to rubella. She is prescribed prenatal vitamins with folic acid 800mcg daily and an iron supplement of 27mg daily. A fetal anatomy ultrasound has been ordered as part of her prenatal care.

Comparison3 issues

Missing information:

  • gestational diabetes was specifically noted as occurring in her previous pregnancy

Hallucinated:

  • gestational diabetes characterized as resolved when the original only states it as a history of gestational diabetes in her previous pregnancy
  • ultrasound status changed from next ultrasound scheduled for anatomy scan to a fetal anatomy ultrasound has been ordered
ChatGPT→Gemini

Reconstructed text

Sarah Johnson, a female born in 1994, is currently presenting with an active, uncomplicated pregnancy. She is Gravida 2, Parity 1 (G2P1) and is at an estimated 24 weeks gestation. Her medical history is notable for a resolved history of gestational diabetes mellitus during a previous pregnancy.

Her recent laboratory results include a normal glucose tolerance test. Her blood type is O positive, and she is confirmed to be immune to the Rubella virus. She is currently on active medications, taking a prenatal vitamin with 800 mcg of folic acid daily, as well as a 27 mg iron supplement daily.

Her Obstetrics Provider has placed an order for an obstetric ultrasound anatomy scan to monitor the pregnancy. Consequently, a diagnostic radiology appointment for this ultrasound has been booked and accepted by the patient.

Comparison8 issues

Missing information:

  • age explicitly stated as 32 years old

Hallucinated:

  • detail that the ultrasound appointment has been booked and accepted by the patient is fabricated
  • detail that an obstetrics provider has placed an order for the ultrasound is fabricated original simply states next ultrasound is scheduled
  • birth year of 1994 fabricated original only states age 32
  • birth year added born in 1994 which is not stated in the original scenario
  • states prior gestational diabetes was resolved original only notes a history of gestational diabetes in prior pregnancy without stating resolution status
  • birth year of 1994 added
  • details about an obstetrics provider placing an order a diagnostic radiology appointment being booked and the patient accepting the appointment were fabricated
Claude→ChatGPT

Reconstructed text

Sarah Johnson is a female born in 1993. She is currently pregnant with an uncomplicated pregnancy, documented as gravida 2 para 1 (G2P1) and 24 weeks’ gestation. Estimated gestational age is 24 weeks. Gravida is 2 and parity is 1.

Her obstetric/problem history includes a confirmed history of gestational diabetes mellitus in a previous pregnancy, which is listed as resolved; a note specifies that gestational diabetes occurred during the prior pregnancy. Current glucose tolerance testing is interpreted as normal.

Prenatal laboratory results show blood type O positive (O Rh[D] positive). Rubella IgG indicates she is rubella immune with a normal interpretation.

Active medications include:

  • Prenatal vitamins with folic acid: folic acid 800 mcg by mouth daily (documented as “800 mcg folic acid daily”).
  • Ferrous sulfate 27 mg oral tablet (iron supplement): 27 mg by mouth daily.

An obstetric ultrasound anatomy scan has been ordered (ServiceRequest active) for the indication of her normal pregnancy at 24 weeks’ gestation, with a note stating it is scheduled for 24 weeks. An associated appointment is booked for the “Anatomy scan ultrasound,” based on the ultrasound order; Sarah Johnson is a participant and has accepted the appointment. (No appointment date/time or location is provided in the record.)

Comparison12 issues

Missing information:

  • age explicitly stated as 32 years old

Hallucinated:

  • birth year 1993 fabricated not in original
  • iron supplement specified as ferrous sulfate when original only states iron supplement
  • appointment described as booked and patient has accepted the appointment original only states ultrasound is scheduled
  • ultrasound note states it is scheduled for 24 weeks original does not specify timing only that the next ultrasound is for anatomy scan
  • gestational diabetes described as confirmed and resolved these specific statuses are not stated in the original
  • birth year addedchanged reconstructed states she was born in 1993 original only states she is 32 years old no birth year provided
  • gestational diabetes status altered reconstructed describes prior gestational diabetes as confirmed and resolved original only states a history of gestational diabetes in the previous pregnancy without labeling it confirmed or resolved
  • ultrasound scheduling details fabricated reconstructed states an anatomy scan ultrasound has been ordered as an active servicerequest and that it is specifically scheduled for 24 weeks original only says the next ultrasound is scheduled for an anatomy scan no orderservicerequest status and no explicit scheduled for 24 weeks statement
  • appointment details fabricated reconstructed states an appointment is booked for the anatomy scan and that the patient has accepted the appointment original does not mention an appointment being booked or accepted
  • rubella lab interpretation details added reconstructed specifies rubella igg with a normal interpretation original only states rubella immune without test type or interpretation wording
  • rubella immunity specified as being determined by rubella igg
Mistral→Claude

Reconstructed text

Sarah Johnson is a 34-year-old female (born 1991, patient ID 12345) currently pregnant at 24 weeks gestation as of October 15, 2023, with pregnancy status interpreted as normal.

Obstetric History: Ms. Johnson has a history of gestational diabetes mellitus in a previous pregnancy, with onset documented on January 1, 2020, and abatement on June 30, 2020. This condition is confirmed and currently listed as active, with a clinical note stating "History of gestational diabetes in previous pregnancy."

Laboratory and Screening Results:

  • Blood glucose (October 10, 2023): 90 mg/dL, interpreted as normal.
  • Blood type: O positive.
  • Rubella serology: Immune.

Current Medications:

  1. Prenatal vitamins with folic acid 800 mcg, taken once daily.
  2. Iron 27 mg, taken once daily.

Pending Orders: A transabdominal obstetric ultrasound (CPT 76805) for fetal and maternal evaluation has been ordered and is scheduled for October 20, 2023. This is a standard second-trimester ultrasound for single gestation evaluation with real-time imaging and documentation.

Comparison25 issues

Missing information:

  • g2p1 obstetric history designation
  • ultrasound described as anatomy scan in original
  • gravidapara status g2p1
  • statement that the current pregnancy is uncomplicated
  • result specified as a normal glucose tolerance test gtt in the current pregnancy
  • g2p1

Hallucinated:

  • age changed from 32 to 34
  • birth year 1991 fabricated
  • patient id 12345 fabricated
  • date of october 15 2023 for current gestation fabricated
  • gestational diabetes onset date of january 1 2020 fabricated
  • gestational diabetes listed as active original indicates it was a historical condition from a previous pregnancy
  • glucose tolerance test changed to blood glucose test
  • blood glucose value of 90 mgdl fabricated
  • cpt code 76805 fabricated
  • added birth year born 1991 not in original
  • added patient id 12345 not in original
  • added specific dating 24 weeks gestation as of october 15 2023 date not in original
  • added blood glucose lab 90 mgdl on october 10 2023 original only states normal glucose tolerance test no valuedate provided
  • added gestational diabetes onsetabatement dates january 1 2020 to june 30 2020 not in original
  • gestational diabetes history characterized as confirmed and currently listed as active original describes only a history in a previous pregnancy and current testing is normal
  • ultrasound order details addedchanged transabdominal obstetric ultrasound with cpt 76805 described as standard secondtrimester single gestation evaluation scheduled for october 20 2023 original only states next ultrasound scheduled for anatomy scan without cpttypedate
  • previous gestational diabetes incorrectly listed as currently active
  • blood glucose test specific value 90 mgdl and date october 10 2023 added
  • ultrasound scheduled date october 20 2023 modality transabdominal and cpt code 76805 added