Purpose: The process of developing checklists to rate clinical performance is essential for ensuring their quality; thus, the authors applied an integrative approach for designing checklists that evaluate clinical performance.
Method: The approach consisted of five predefined steps (taken 2012–2013). Step 1: On the basis of the relevant literature and their clinical experience, the authors drafted a preliminary checklist. Step 2: The authors sent the draft checklist to five experts who reviewed it using an adapted Delphi technique. Step 3: The authors devised three scoring categories for items after pilot testing. Step 4: To ensure the changes made after pilot testing were valid, the checklist was submitted to an additional Delphi review round. Step 5: To weight items needed for accurate performance assessment, 10 pediatricians rated all checklist items in terms of their importance on a scale from 1 (not important) to 5 (essential).
Results: The authors have illustrated their approach using the example of a checklist for a simulation scenario of infant septic shock. The five-step approach resulted in a valid, reliable tool and proved to be an effective method to design evaluation checklists. It resulted in 33 items, most consisting of three scoring categories.
Conclusions: This approach integrates published evidence and the knowledge of domain experts. A robust development process is a necessary prerequisite of valid performance checklists. Establishing a widely recognized standard for developing evaluation checklists will likely support the design of appropriate measurement tools and move the field of performance assessment in health care forward.
Mr. Schmutz is research associate, Industrial Psychology and Human Factors, Department of Psychology, University of Fribourg, Fribourg, Switzerland.
Dr. Eppich is assistant professor of pediatrics and medical education, Northwestern University Feinberg School of Medicine, Chicago, Illinois.
Dr. Hoffmann is senior pediatrician, Dr. von Hauner University Children’s Hospital, Munich, Germany.
Dr. Heimberg is pediatrician, University Children’s Hospital, Tübingen, Germany.
Dr. Manser is associate professor for industrial psychology and human factors, Department of Psychology, University of Fribourg, Fribourg, Switzerland.
Funding/Support: This work was funded by the Swiss National Science Foundation (grant number PP00P1_128616).
Other disclosures: None reported.
Ethical approval: This type of research is exempt from ethics review in Switzerland.
Previous presentations: Oral presentation at the 19th Annual Meeting of the Society in Europe for Simulation Applied to Medicine (SESAM), June 13 to 15, 2013, Paris, France.
Correspondence should be addressed to Mr. Schmutz, University of Fribourg, Department of Psychology, Industrial Psychology and Human Factors, Rue de Faucigny 2, 1700 Fribourg, Switzerland; telephone: (+41) 26-300-7484; e-mail: firstname.lastname@example.org.