Purpose or Case Report: At our institution, a pediatric hospital within a larger academic center, radiology residents independently interpret pediatric radiology studies performed after hours. These studies are finalized by an attending pediatric radiologist and any discrepancies from the preliminary report to the final report are tracked. We wanted to develop a streamlined approach to better understand the types of discrepancies, whether there were patterns to the types of these discrepancies, and then use this information to build improved and tailored educational material for our residents.

We used MS365 Copilot/ChatGPT-5 to evaluate fully anonymized discrepancy notifications from radiology resident overnight reports. We had this large language model (LLM) categorize the types of discrepancies (musculoskeletal, abdomen, genitourinary, neuro, chest, spine, and others). We used the LLM evaluate for patterns of errors within each of these categories. Based on these patterns, educational material was designed with the use of these LLM.

Over an 8-month period (Jan 2025-August 2025) there were a total of 436 discrepancy cases identified. Most discrepancy cases involved musculoskeletal (41%), chest (27%), and spine (12%) misinterpretations. Within the musculoskeletal category, most misses involved fractures involving the elbow (20%), forearm (22%) and distal tibia (15%). In the chest category, resdients had difficulty distinguishing bronchial wall thickening from pneumonia (64%). To address the identified deficiencies, a pediatric ED lecture series has been added to the resident education. LLMs are being utilized to help design and focus these specific lecture topics. Additional, refined, and tailored didactic topics include elbow fractures, distal tibial fractures, pediatric pneumonia, and spine fractures.

Artificial intelligence and large language models can be usedl to evaluate data from overnight pediatric radiology resident discrepancy reports to help identify areas of deficiency and improve resident education through design and refinement of lecture topics.
Methods & Materials:
Results:
Conclusions: