Purpose or Case Report: At our institution, airway radiographs are routinely checked by the radiologist to ensure diagnostic image quality prior to the technologist completing the examination. These checks interrupt the workflow for both the technologist and radiologist. In this study, we develop and validate a deep learning algorithm to detect non-diagnostic lateral airway radiographs.
Methods & Materials: Lateral airway radiographs scanned in the emergency room patients scanned between 01/01/2000 and 07/01/2019 were retrospectively queried from the PACS. Two radiologists reviewed and classified each radiograph as diagnostic or nondiagnostic. Disagreements were adjudicated by a third radiologist. This classification served as ground truth to train different deep learning models. The radiographs were assigned to training and test datasets using an 80/20 random data split. Data augmentation techniques like image rotation, shearing and translation were used during training. In order to evaluate the performance of the resultant algorithms, 3 technologists and 3 different radiologists classified the images in the test dataset as diagnostic or nondiagnostic. Inter-observer agreement between the technologists and between the radiologists was calculated and their consensus ratings were compared to the ground truth obtained from the original reviewers.
Results: The training set consisted of 615 radiographs (366 diagnostic/249 nondiagnostic) while the test set consisted of 239 radiographs (165 diagnostic/74 nondiagnostic). The best deep learning model (ResNet) achieved sensitivity, specificity, and AUC (area under the curve) of 0.90 (0.01), 0.82 (0.03) and 0.86 (0.02), respectively on the test set. The technologists achieved sensitivity, specificity, and AUC of 0.70 (0.10), 0.77 (0.10) and 0.74 (0.01) and the radiologists achieved sensitivity, specificity and AUC of 0.94 (0.03), 0.64 (0.09), 0.80 (0.03) versus ground truth. Inter-observer agreement for the technologists was fair (kappa=0.36), while that for the radiologists was moderate (kappa=0.59). Kappa value for agreement between the best deep learning model, consensus rating of three technologists, and consensus rating of three radiologists to the ground truth were 0.69, 0.49 and 0.66, respectively.
Conclusions: We report the development and validation of deep learning models that detect non-diagnostic pediatric DR airway/soft tissue neck X-ray exams and have shown that the algorithms perform significantly better than the technologists and as good as the radiologists.