Purpose or Case Report: Our study aims to test for validity a recently developed scoring system for the assessment of resident proficiency at performing a head ultrasound (HUS) examination in infants.
Methods & Materials: A scoring system, which includes the most important elements in technique and relevant anatomic structures, was designed by 4 senior pediatric radiologists. Face and content validity were obtained by consensus.
Using the scoring system, 10 HUS examinations performed by residents (5 junior and 5 senior) and 10 HUS examinations performed by experienced ultrasound technologists were retrospectively graded by 3 senior pediatric radiologists. To test the scoring system for construct validity, the average scores of the technologists vs. residents and of the junior vs. senior residents were compared using the two-way analysis of variance test.
Results: The maximum score was 36. The mean score was 33.3 for the technologists and 28.6 for the residents. The senior residents scored, on average, higher (31.6) than the juniors (25.6). There is a significant statistical difference between the technologists’ and residents’ scores (p=009) as well as between the junior and senior residents’ scores (p=0.035). The inter-rater variability is moderate (0.5).
Conclusions: The scoring system demonstrates validity for assessing proficiency at performing HUS examinations. The use of in-training evaluation provides the opportunity for a critical appraisal of the resident’s competency and allows for timely corrective action through formative feedback. The scoring system will subsequently be used to evaluate the effects of a newly introduced brain phantom simulation on HUS training.