Purpose or Case Report: To evaluate the peer collaborative improvement (PCI) process used in our pediatric radiology department since January 2016, differences in sequential surveys and temporal change in types of submissions were assessed.
Methods & Materials: Study was HIPAA compliant and exempt from IRB approval. Faculty were anonymously surveyed online in August 2016 and March 2019. Response options were scaled 1 (disagree) to 10 (agree). Response differences were compared for 3 questions that were asked on both surveys via the Wilcoxon-rank test. Free text comments were tabulated and categorized, and compared via Fisher’s exact test. In response to the 2019 survey, the PCI system was modified to improve anonymity. PCI database was queried over three time periods for number and types of submissions: 2016-2017 (16 months following go-live), current 2019 (5 months following modification), and 2018 (same 5-month period, one year prior). Rates of submission types were compared using chi-square test.
Results: In 2019, 33 of 42 faculty responded (79% participation rate, previously 30 of 40, 75%). Comparing 2016 to 2019, there was no statistically significant change in scores of either “Comfort in pointing out errors knowing data not used to judge peers” (2016 vs 2019 mean 8.0 vs 6.7 p=0.071) or faculty “usually receives helpful information in PCI monthly conferences” (2016 vs 2019, 8.4 vs 7.8 p=0.36), though both scores decreased. There was a significant decrease in response to “goal of PCI is improving patient care, not placing blame” (2016 vs 2019, 9.1 vs 7.9 p<0.0014). There were 17 comments in 2019 (51.5%), increased (p<0.0001) from 2016 (n=2, 6.3%). 2019 comments were to improve the punitive aspects of PCI (n=7, 41.2%), ensure improved patient care (n=6, 35.3%), and increase anonymity (n=4, 23.5%). The lowest rated question in 2019 was “The random auditing of cases contributes important learning material" (3.6). There was no significant difference between the types of PCI submissions over the time periods (p = 0.62), with most agree (2016-2017 90.7%, 2018 91.5%, 2019 91.4%), and no change in error types (perception/cognition 5.8%, 5.4%, 5.5%; reporting 1.9%, 1.7%, 1.7%; and other 1.6%, 1.4%, 1.4%).
Conclusions: Faculty have increasing concerns with anonymity and potential punitive effects of PCI submissions. Faculty affirm the value of learning from PCI conferences. Lack of identifiable change in PCI submission types over multiple time periods suggests the types of submissions reflects our error types in practice.