Farrell et al: Evaluating validity evidence for 2 instruments developed to assess students' surgical skills in a simulated environment
Veterinary Surgery 5, 2022

🔍 Key Findings

• 39 of 40 checklist items for simulated OVH surgical assessment had good content validity (CVI = 0.81)
• Only 1 of 6 items from the OSATS GRS (respect for tissue) met inclusion criteria (CVI = 0.80)
• Checklist showed strong reliability (G-coefficient = 0.85) for moderate-stakes exams
• Modified OSATS GRS showed acceptable reliability (G-coefficient = 0.79)
• Two raters needed for acceptable reliability in high-stakes exams when using the checklist
• Minimal interrater bias found; variance largely due to interaction among student, rater, and item
• Digital recordings were a reliable method of evaluating surgical performance
• Study supports using checklist over OSATS GRS for assessing preclinical students on simulated models