Development of a Method to Measure Clinical Reasoning in Pediatric Residents: The Pediatric Script Concordance Test
Date
2015-09-30
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Context: National medical examinations boards have requested that methods be developed to assess the clinical reasoning competency of medical trainees. The Script Concordance Test (SCT) is an emerging method of assessment that holds promise for the evaluation of clinical reasoning skills.
Objectives: This study had 3 objectives: 1) to examine the validity of SCT scores in accurately discriminating clinical reasoning ability between junior (JR) and senior (SR) pediatric residents and experienced pediatricians serving on the panel of experts (POE), 2) to determine if higher reliability of the SCT method could be achieved by applying specific strategies to the design and construction, and 3) to explore participants’ acceptability of the SCT method.
Methods: A 90-minute SCT containing 24 cases and 137 questions was administered to 91 residents from four pediatric residency training centers. Twenty-one pediatricians served on the POE. All participants completed a post-test survey designed to explore their impressions and attitudes regarding the SCT assessment method.
Results: Overall, there was a difference in performance across the three levels of experience, F = 22.84 (df = 2); p < 0.001. The POE scored higher than both the SR (mean difference 9.15; p < 0.001) and the JR (mean difference = 14.90; p < 0.001) and the SR scored higher than the JR (mean difference = 5.76; p < 0.002). The reliability of the SCT scores (Cronbach’s α) was 0.85. Sub-test data, based exclusively on the new evolving type cases demonstrated similar results. Participants expressed keen interest and engagement in the SCT format of assessment.
Conclusions: The findings of this study contribute to a growing body of literature suggesting that SCT holds promise as a valid, reliable and acceptable method to assess the core competency of clinical reasoning in medicine. The SCT method is based on two core principles we believe are central to the assessment of clinical reasoning: acknowledging uncertainty and the possibility of more than one “right answer.” We propose the SCT may be effectively integrated into formative residency assessment and with increasing exposure, experience and refinement may soon be ready to pilot within summative assessments in pediatric medical education.
Description
Keywords
Medicine and Surgery
Citation
Cooke, S. (2015). Development of a Method to Measure Clinical Reasoning in Pediatric Residents: The Pediatric Script Concordance Test (Doctoral thesis, University of Calgary, Calgary, Canada). Retrieved from https://prism.ucalgary.ca. doi:10.11575/PRISM/25660