Purpose or Case Report: The ChRonic nonbacterial Osteomyelitis Magnetic Resonance Imaging Scoring (CROMRIS) consensus-based scoring system was adapted to assess specific features of bone and soft tissue inflammation in MRI of patients with CNO in a web-based interface. The objectives were to evaluate the absolute agreement of components of and summary CROMRIS scores at each body site, and the inter-rater reliability across raters.
Methods & Materials: A reliability module with 29 deidentified whole body (WB) MRI scans from children with CNO and normal controls was created. Three pediatric radiologists independently scored these cases directly on the web-based platform after viewing a training atlas and video. Readers scored individual bone units to indicate presence of: bone marrow hyperintensity (HI) on STIR images (score 0-1); soft tissue/periosteal HI (score 0-1), bony expansion (score 0-1), and quantify the signal size of CNO lesion (scores 1-3 defined as <25%, 25-50%, or >50% of estimated volume, respectively). The sum of these parameters for lesions detected on fluid-sensitive sequences was the CROMIS Activity Index (maximum score 722). Pairwise comparisons of the specific parameters and CROMRIS Activity Index were made. Absolute agreement was when the summary score difference between any 2 raters was <=2. For specific variables, (bone marrow HI), signal size, soft tissue/periosteal HI, bony expansion, the absolute agreement was defined as all the raters assigning the exact same score. Interrater reliability for abnormal signal and severity were assessed using free-marginal kappa (k) statistics.
Results: Among all bone sites, the prevalence of abnormalities were the highest at the tibia (59%), feet (41%), femur (38%), and pelvis (34%). The proportion of absolute agreement of CROMRIS Activity Index scores across 3 raters was greater than 70% in most bone sites. Mean kappa of each category of bones was near or >0.5 demonstrating moderate interrater reliability amongst radiologists for most bone sites, except lumbar spine, ulna, patella, mandible, sternum, scapula, and sacroiliac joints, which had mean k scores ranging from -0.04 to 0.21, correlating with sites of lowest prevalence of abnormalities in this module.
Conclusions: The web-based CROMRIS showed moderate agreement of CROMRIS Activity Index total scores among experienced radiologists after self-guided learning of atlas and video. A consensus-based set of scans for reader calibration will be developed to enhance inter-rater reliability for clinical trials.