Cerebral palsy (CP) is the most common cause of childhood physical disability, with a reported prevalence of 3.5 per 1000 births in the United States.1 Physical therapists provide services to children with CP across the lifespan to promote optimal function and participation while minimizing secondary complications.2 The focus and amount of therapy intervention varies by age and severity.3–5 Determining optimal dose (frequency, intensity, timing, and type) of physical therapy (PT) intervention for individuals with CP is a national priority6 and would support value-based care.
A recent National Institutes of Health executive summary7 identifies using detailed intervention information from the electronic medical record (EMR) as an effective pathway toward improving our understanding of dose and outcomes through practice-based evidence (PBE) designs. PBE designs include care and outcomes from regular clinical care that are used to study associations between patient characteristics, intervention characteristics, and outcomes.8,9 Integrating standard point-of-care forms in the EMR may allow us to combine large amounts of clinical data across institutions to inform dosing questions and improve care. The emerging field of bioinformatics has developed resources for hospital settings to assist clinicians with harvesting data from the EMR for research and improvement endeavors that offer the opportunity to study practice patterns and assess outcomes.10 Even so, prior work4 highlights that detailed documentation about what happens during PT sessions for individuals with CP is often lacking. Consequently, dosing details for individuals with CP at a single session and at aggregated population levels remain largely unknown.
Quality improvement (QI) methods offer an innovative way to change work processes and spread changes quickly.11 Examples include physician compliance with hand hygiene12 and medicine reconciliation.13 QI methods use multiple Plan-Do-Study-Act (PDSA) cycles beginning with small scale changes and include data acquired from daily workflow to alter how work is done.
Using QI methods and a well-designed documentation system for an EMR may provide a strong, sustainable path toward standardizing documentation of dose of intervention and create an infrastructure supporting PBE studies related to dose. The purposes of this report are to describe (1) the QI activities used to develop and monitor a reliable process in the EMR to improve performance of therapist documentation of dose for PT CP treatment sessions and (2) insights gained from this project. The Revised Standards for Quality Improvement Reporting Excellence (SQUIRE 2.0)14 were used to guide the information provided in this report.
Setting and Context
This QI project occurred from January 2017 to February 2018 in a pediatric academic medical center with 8 outpatient locations serving individuals with CP in both urban and suburban environments. At the start of the project, there were 55 licensed PT CP providers (43.67 FTEs) across locations with years of experience ranging from less than 1 to 28 years (median 11.5 years). In January 2017, a core improvement team composed of 4 pediatric physical therapists (the QI team leader plus 3 treating therapists) was established. The QI core team met weekly to review information, evaluate progress, make decisions, and plan next steps. Education on improvement science and personalized mentors supported the implementation, analysis, and reporting of the project. The initiative was within the facility's Institutional Review Board's guidance for QI work that was not considered human subjects research.
The Model for Improvement,11 a framework that promotes increased effectiveness, efficiency, and learning, was used to guide planning and implementation of the team's improvement efforts. In addition to the PDSA cycle, the Model for Improvement is based on 3 questions: (1) what are we trying to accomplish, (2) how will we know that a change is an improvement, and (3) what changes can we make that will result in improvement? Specifically, the Model for Improvement was used to help the team (1) establish a shared vision for what they were trying to accomplish using global and smart aims, (2) formulate a measurement system to identify specific changes resulting in an improvement, and (3) design interventions that could result in the desired improvement. The team developed a key driver diagram to represent their theory of improvement and to guide overall planning of interventions15 (Figure 1). The smart aim, key drivers, and interventions were refined and revised as the project progressed. Using consensus decision-making, potential interventions for each driver were planned and tested using PDSA cycles.16
Prior to starting PDSA cycles, the QI core team reviewed the current process and practices for delivery and documentation of dose in therapy sessions. A systematic method to identify and prioritize existing failures before they occur, Failure Mode Effects Analysis (FMEA) technique,17 was used in the preintervention or baseline period. The FMEA method includes review of functions and failure causes and consequences as part of improvement projects, to design a process to address potential failures. Using this technique, several failures in documentation of dose were identified including (1) clinician lack of awareness of dose, (2) few entries in the existing EMR record on the level of effort of the child, (3) lack of discrete parameters to document dose and to allow for communication or comparison over time, and (4) variability in documentation practices among therapists. A Pareto chart11 represents qualitative data and is used to help focus improvement efforts by charting categories of reasons for failure. The team created a Pareto chart (Figure 2) to determine the general frequency of failure types and consider interventions that could offer the greatest effect for improvement efforts. Change concepts11 were used to brainstorm ideas for potential interventions that could result in an improvement in therapy dose documentation.
Study of the Interventions
Measure Development:. At the start of the project, the team identified a shared global goal to deliver evidence-based therapy intervention to all children with CP and envisioned developing a standardized infrastructure for documenting therapy dose in the EMR. The QI team operationalized “dose” as 4 elements: frequency, intensity, timing, and type of intervention.18 Using examples from the literature, frequency was defined as how often the treatments are planned to occur (eg, every other week, weekly, several times a week, periodic, consultative, and other).19Intensity was defined as the child's effort during the session and measured on a 0- to 6-point Likert scale. Timing was operationalized as the total duration of the session in minutes, and type was defined as both the area of focus or goals of the session and specific interventions delivered during the session.20–23
Prior to initiating improvement interventions, the QI core team analyzed current processes and the QI team lead manually extracted baseline data by retrospective chart review, sampling documentation from 2 treatment sessions of 3 participating therapists each week (September 5, 2016, to February 3, 2017). Within a single session, a possible of 4 dose elements could be documented. An element was counted as present if it was clear from reading the note what the frequency of the intervention, intensity, or level of effort of the child in the session, total time in the session, and what type of intervention was delivered either in a discrete field or in free text. The percentage of dosing elements present at baseline was approximately 78%, calculated by dividing the number of total elements present by the total possible for sessions (eg, 10 sessions × 4 dose elements = 40/40 possible).
Based on the global aim, project vision, and baseline data, the QI core team developed a SMART (Specific, Measurable, Actionable, Realistic, and Timely) aim to increase the percentage of dose elements documented from 78% to 90% by January 2018. From February 6, 2017, until March 10, 2017, manual chart reviews were performed. Beginning March 13, 2017, the EMR system was evolved to facilitate weekly automatic data audits. A divisional registry stored and managed extractions over the course of the study for individuals with CP, as identified through billing codes and problem lists in the EMR and did not include age limits.
In addition, a balancing measure24 was established to evaluate for unintended consequences (eg, incorrect documentation resulting from process changes). Each month (March to December 2017), 2 treatment sessions of therapists using the new documentation system were video recorded and percent agreement between therapists was evaluated overall for intensity, time, and type of intervention by comparing what was documented in the EMR by the treating therapist to what was documented on paper by therapists viewing the recorded sessions. Agreement for treatment frequency was not established due to the inability to view frequency on a recorded session.
Analysis. Progress toward achieving the SMART aim and monitoring of the balancing measure was evaluated through a quantitative time series study design. Statistical process control (SPC) was used to monitor process behaviors over time. SPC is a method using statistical techniques to measure and control for process variation. Control charts are a graphical tool used to visually parse the data into potential cases of common cause and special cause variation. The control limits in SPC represent ±3 standard deviations around the center line and are used to characterize the stability of a system measure. Common cause variation is inherent in a system and typically represents the natural variation expected within a given system for that measure. In contrast, special cause variation is typically represented by a new event in the system that is outside of what is expected as natural variation. Sometimes special cause events are unanticipated and unplanned (eg, a fire alarm that disrupts a therapy session), and sometimes special cause is due to a predictable cause (eg, a change in staffing from an experienced therapist to a novice therapist).
In QI, the goal is often to achieve consistent and stable special cause variation in the direction of the SMART aim (eg, staff training for new standardized documentation procedures). There are rules to define what constitutes a stable change in the measurement signal such that the special cause variation can shift the centerline creating new boundaries for common cause versus special cause variation.25 This study used a P-chart (Figure 3), which is an SPC chart used to plot the proportion of conforming units, in this case percentage of dosing elements present.
Annotated P-charts were updated and shared weekly with the QI core team and eventually the PT staff. The centerline represented the mean with upper and lower control limits in the P-chart. Standard criteria were used to determine whether observed changes in measures were common cause variation or due to a special cause in this case the intervention.11
The QI efforts resulted in an overall improvement from 78% dose documentation to 94% during a 13-month period. The final key driver diagram and the P-chart are represented in Figures 1 and 2, respectively. Initial PDSA testing (January 2017 to June 2017) started with a team of 4 physical therapists, and then spread to 1 therapist at each community location (June 2017 to September 2017) before spreading to all therapists within the division by October 16, 2017. The process was monitored weekly until achievement of the aim (February 26, 2018), which was defined as 12 consecutive data points with the completion, met or exceeded our goal.
An early Pareto analysis (Figure 2) of process failures demonstrated that 83% of the failures were due to missing information about the child's effort during the session. In many cases, the therapist documented “child tolerated well” but there was no way to interpret how that directly related to child's effort. An early emphasis, therefore, focused on improving documentation of the child's effort level toward meeting the session objectives. The team combined each concept with knowledge of their system to develop interventions that were planned and tested through PDSA cycles according to 4 change concepts11: (1) change the work environment, (2) manage variation, (3) exploit variation, and (4) consider people in the same system. During testing, 4 different interventions were trialed with varying numbers of PDSA cycles (Table). Interventions were categorized according to the 4 change concepts.
Summary and Outcomes of Plan-Do-Study-Act Interventions
||Summary and Outcome
||PDSA Cycles, n
||Testing Length, d
||Effect to Key Driver
||Change the work environment, increase awareness and buy-in
||Developed and shared training materials, observed taped session, and watched with initial team to revise training materials for flow sheet use. Adopt
||Therapist knowledge, clinician buy-in, effective engagement
||Manage variation, create transparent documentation flow sheet
||Trial documentation flow sheet and revise. Adopt
||Transparent and reliable documentation system, buy-in, knowledge
||Create note templates that pull in information from EPIC flow sheet. Adopt
||Transparent and reliable documentation system, buy-in, knowledge
||Consider people in the same system
||Spread plan: Trial whole package (training, video, flow sheet, and note templates) with 1 therapist at another satellite, spread to 1 at each location, then all Adopt
||Transparent and reliable documentation system
Abbreviation: PDSA, Plan-Do-Study-Act.
Change the Work Environment
PDSA testing to change the work environment (February 6, 2017, to January 9, 2018) consisted of 5 PDSA cycles and began with educational training of the QI core team and divisional leaders. A didactic lecture was provided reviewing the literature on dosing for children with injured brains,6,26 and used PBE examples20,22,23,27 to focus on the importance of documenting in discrete fields. An expert in PBE was invited and spoke on highlighting benefits of PBE. Following this training, the QI core team viewed a videotaped treatment session and discussed using the National Institute for Neurological Disorders and Stroke (NINDS) PT individual session form.28 Information and knowledge of clinician performance was shared weekly throughout the process. Training was provided to all staff at meetings in June 2017 and October 2017. PDSAs related to changing the work environment continued until January 9, 2018. Training materials and definitions of terms (see Supplemental Digital Content 1 [available at: http://links.lww.com/PPT/A251] and 2 [available at: http://links.lww.com/PPT/A252]) were revised 2 additional times before agreement was reached to adopt the educational module consisting of a training manual, didactic lecture on dosing and PBE, and review of 2 treatment videos followed by practice completing a paper form for each viewed session. These interventions were linked to the drivers of therapist knowledge, clinician buy-in, knowledge of evidence-based treatments, and audit and feedback.
PDSA testing to manage variation (February 9, 2017, to January 9, 2018) consisted of 5 cycles and started with the core team testing the NINDS PT session28 paper form for 2 treatment sessions and providing feedback. The core team's feedback was used to inform the creation of an electronic form in EPIC that included radio buttons and free-text fields aligned with intervention dose elements (frequency, intensity, time, and type; Supplemental Digital Content 1 [http://links.lww.com/PPT/A251]).
Testing of the electronic flow sheet started in the EPIC test environment and, upon team agreement, went into live production on March 13, 2017. A significant improvement in the process was demonstrated with a nonrandom signal of 8 consecutive data points above baseline (indicating special cause variation), which warranted shifting the center line to 97% on April 24, 2017 (Figure 3).
The core team agreed our process was working and spread to another therapist began at one of the community locations on May 13, 2017, and followed with spread to 1 person at each community location from June 2017 to September 2017. The flow sheet was revised twice more before adoption on January 9, 2018. Adopted PDSA cycles provided feedback that resulted in revisions to EPIC flow sheet, clarification of definitions of terms and updates to staff training materials. Revisions to the flow sheet included order changes to improve flow, use of capitalized headers to increase legibility, and the addition of the focus areas of formal assessment and pain/effusion (see Supplemental Digital Content 1 [full EPIC flow sheet and instructions], available at: http://links.lww.com/PPT/A251). Interventions for managing variation were linked to the key drivers of clinician buy-in, transparent and reliable documentation system, and therapist knowledge of dose elements.
Testing to exploit variation (March 27, 2017, to July 3, 2017) included 2 PDSAs. During the FMEA process, QI core team members identified a preference to customize their notes using note templates for different types of sessions. Thus, the team created note templates in EPIC that would use the dosing information from the flow sheet. The flow sheet information was linked with unique note templates (eg, casting visit, locomotor training, orthotic fitting, and intensive strengthening), which allowed for some note customization if additional detail was warranted. Cycles continued during training and spread to other locations, resulting in additional template modifications and templates (eg, orthotic molding/fabrication) before the team agreed to adopt in July 2017. This intervention was linked to the key drivers of transparent and reliable documentation system, clinician buy-in, and therapist knowledge of dose.
Consider People in the Same System
PDSA testing to consider people in the same system (May 2, 2017, to October 16, 2017) included 6 PDSA cycles. The goal was to successfully spread use of the CP treatment documentation flow sheet to all of the division's physical therapists treating children with CP. While all physical therapists work under the same hospital system, each community location of therapists had its unique culture, style of note writing, and documentation practices. In order to assure that the training materials and note templates worked for all therapists, we leveraged existing groupings at each community location by spreading strategically to 1 therapist at each site. The site PTs were designated as the “spread team” serving as a local resource to their respective location's therapists. On September 29, 2017, the remaining physical therapists treating CP (40 PTs) completed training with a 2-week transition to using the flow sheet by October 16, 2017. Staff tracked performance via weekly histograms that included completion percentage for each community location. Beginning the week of October 16, we observed 8 consecutive data points below the centerline resulting in a downward shift to 85% (Figure 3). Following ongoing weekly audits and feedback, we observed an upward shift and established a new baseline for 12 consecutive data points at 94%.
Monitoring the balancing measure (March 2017 to January 2018) consisted of video recording 2 therapy sessions of each therapist on the core and spread team. Each month the core and spread team therapists viewed 2 recorded sessions and completed documentation of a paper version of the flow sheet. Percent agreement between what was documented in the EMR by the video-recorded therapist and on paper by the viewing therapist was calculated overall combining intensity, time, and type with an overall 86% agreement among therapists.
This QI report describes a stepwise approach for successful execution of a project that aimed to improve the documentation infrastructure for PT dose for individuals with CP and resulted in an increase in the dose elements documented at each session weekly from 78% to 94% over a 13-month period. Frequency, intensity, time, and type of PT intervention are documented at each treatment session in discrete fields and are easily harvestable to track dose. Key drivers of improvement included (1) knowledge and awareness of dose, (2) clinician buy-in, (3) effective engagement of child and parent, (4) therapist knowledge of evidence-based treatments, (5) transparent and reliable documentation system, and (6) audit and clinician feedback. This new standardized documentation infrastructure allows for a reliable and efficient way to study dose using PBE methods.
In reflecting on our QI initiative, several lessons and insights were identified that may be transferable to other conditions and institutions. Weekly review of the data with therapists allowed for frequent reflection and revision of the interventions until adoption. As expected with spread, a downward shift in completion rate was observed in October 2017, as all newly trained therapists learned to use the new documentation process. Over time, with weekly audit and feedback, completion increased by January 2018.
Clinicians expressed that using the new documentation prompted them to reflect on what treatment was being delivered and on whether the child's level of effort was matched to the focus of the session. Clinicians also identified that these reflections motivated them to adjust interventions, as they worked with children and caregivers to maximize the level of effort toward achieving the session's goals. Initially, therapists questioned the documentation changes and expressed concern over a standardized note template. Therapists felt customizing each note was necessary for 2 reasons: (1) to know what they did in the child's last session to progress treatment toward goals and (2) to provide adequate information for a covering therapist scenario. The majority were receptive to participating in the changes when presented with ways to customize their note and include more detail if desired.
Therapists initially questioned audits and presented rationale against change, such as need for increased time to document and change of their workflow. After using the system, the majority reported that the new system was efficient and actually shortened documentation time compared with the previous system. Furthermore, therapists reported they could complete a large portion of their note documentation during the session, while previous to implementing the new flow sheet, they often completed all documentation after the session. The frequent feedback and data about the services delivered to children with CP provided impetus for ongoing engagement. This included informal data sharing, formal staff meeting presentations, and positive recognition of team members, as improvements were made and sustained across each location.
As with all studies, there are several limitations related to the generalizability of our findings to other settings and conditions. The QI process for implementing this system with other conditions or with other institutions or EMR may be different. Delays in data reporting for example, waiting at least 7 days for the therapist to complete documentation and implementation of multiple interventions simultaneously, impacted the ability to discern which interventions brought about specific changes. There was the potential for inaccurate documentation of information; however, our balancing measure demonstrated over 80% agreement among 10 therapists on overall intensity, time, and type. A factor that may be unique to our setting is the infrastructure with access to QI resources and a supportive culture surrounding staff ability to participate in QI activities. Other institutions and clinicians may have fewer support and infrastructure, limiting the ability to engage in QI activities. Despite the limitations, a major strength of this work is the system to harvest a large volume of treatment data for children with CP.
Unintended consequences included some patients being seen by orthopedic sports physical therapists. These therapists were not targeted to train in this study, but plans to provide training to this group are underway. Another unintended, but positive, consequence was that therapists identified a desire to use this standardized system for all patients on their caseload, not only those with CP.
QI methods provided the tools and structure to improve workflow and increase consistency in the documentation of PT intervention dose for individuals with CP. EMR flow sheets and training materials may be useful and would facilitate collaboration between institutions to study intervention dose in CP. Improving the consistency of documentation for children with CP offers valuable opportunity to embark on large-scale PBE studies in pediatric PT to study a variety of interventions and outcomes.
We would like to thank Jason Long, PhD, and Julie Badylak, PT, for their technical assistance with data extraction and database creation. Mike Clay, PT, DPT, Chantelle Cunningham, PT, DPT, Molly Thomas, PT, DPT, and Kelly Bonarrigo, PT, DPT, for participating in the core team and testing throughout the process, Mary Gannotti, PT, PhD, and Susan Horn, PhD, for their mentorship and the APPT Research Summit III on Dosing for inspiring these collaborations.
1. Christensen D, Van Naarden Braun K, Doernberg NS, et al. Prevalence of cerebral palsy
, co-occurring autism spectrum disorders, and motor functioning—Autism and Developmental Disabilities Monitoring Ne2rk, USA, 2008. Dev Med Child Neurol. 2014;56(1):59–65.
2. Wright MJ, Wallman L. Cerebral palsy
In: Campbell SK, Palisano RJ, Orlin MN, eds. Physical Therapy for Children. 4th ed. St Louis, MO: Elsevier Saunders; 2012:577–627.
3. Bailes AF, Gannotti M, Fenchel M. Using the electronic medical record to study the association of child and environmental characteristics on the type of physical therapy services delivered to individuals with cerebral palsy
. Physiother Theory Pract. 2017;33(8):644–652.
4. Bailes AF, Succop P. Factors associated with physical therapy services received for individuals with cerebral palsy
in an outpatient pediatric medical setting. Phys Ther. 2012;92(11):1411–1418.
5. Palisano RJ, Begnoche DM, Chiarello LA, Bartlett DJ, McCoy SW, Chang HJ. Amount and focus of physical therapy and occupational therapy for young children with cerebral palsy
. Phys Occup Ther Pediatr. 2012;32(4):368–382.
6. Kolobe TH, Christy JB, Gannotti ME, et al. Research summit III proceedings on dosing in children with an injured brain or cerebral palsy
: executive summary. Phys Ther. 2014;94(7):907–920.
7. Frontera WR, Bean JF, Damiano D, et al. Rehabilitation Research at the National Institutes of Health Moving the Field Forward (Executive Summary). Phys Ther. 2017;97(4):393–403.
8. Horn SD, DeJong G, Deutscher D. Practice-based evidence
research in rehabilitation: an alternative to randomized controlled trials and traditional observational studies. Arch Phys Med Rehabil. 2012;93(8 suppl):S127–S137.
9. Horn SD, DeJong G, Ryser DK, Veazie PJ, Teraoka J. Another look at observational studies in rehabilitation research: going beyond the holy grail of the randomized controlled trial. Arch Phys Med Rehabil. 2005;86(12, suppl 2):S8–S15.
10. Dean BB, Lam J, Natoli JL, Butler Q, Aguilar D, Nordyke RJ. Review: use of electronic medical records for health outcomes research: a literature review. Med Care Res Rev. 2009;66(6):611–638.
11. Langley G, Rd M, Km N, Tw N, Cl N, Lp P. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. 2nd ed. San Francisco, CA: Jossey-Baas; 2009.
12. White CM, Statile AM, Conway PH, et al. Using improvement science methods to improve physician compliance with proper hand hygiene. Pediatrics. 2012;129(4):e1042–e1050.
13. White CM, Schoettker PJ, Conway PH, et al. Utilising improvement science methods to optimise medication reconciliation. BMJ Qual Saf. 2011;20(4):372–380.
14. Ogrinc G, Davies L, Goodman D, Batalden P, Davidoff F, Stevens D. SQUIRE 2.0 (Standards for Quality Improvement Reporting Excellence): revised publication guidelines from a detailed consensus process. J Contin Educ Nurs. 2015;46(11):501–507.
15. Berwick DM. A primer on leading the improvement of systems. BMJ. 1996;312(7031):619–622.
16. Taylor MJ, McNicholas C, Nicolay C, Darzi A, Bell D, Reed JE. Systematic review of the application of the plan–do–study–act method to improve quality in healthcare. BMJ Qual Saf. 2014;23(4):290–298.
17. Cagliano AC, Grimaldi S, Rafele C. A systemic methodology for risk management in healthcare sector. Saf Sci. 2011;49(5):695–708.
18. American College of Sports Medicine. ACSM's Guidelines for Exercise Testing and Prescription. 9th ed. Philadelphia, PA: Lippincott Williams & Wilkins; 2013.
19. Bailes AF, Reder R, Burch C. Development of guidelines for determining frequency of therapy services in a pediatric medical setting. Pediatr Phys Ther. 2008;20(2):194–198.
20. Horn SD, Corrigan JD, Beaulieu CL, et al. Traumatic brain injury patient, injury, therapy, and ancillary treatments associated with outcomes at discharge and 9 months postdischarge. Arch Phys Med Rehabil. 2015;96(8, suppl):S304–S329.
21. Effgen SK, McCoy S, Jeffries L, et al. Reliability of the school-physical therapy interventions for pediatrics data system. Pediatr Phys Ther. 2014;26(1):118–119.
22. McCoy SW, Effgen SK, Chiarello LA, Jeffries LM, Villasante Tezanos AG. School-based physical therapy services and student functional performance at school. Dev Med Child Neurol. 2018;60(11):1140–1148.
23. McCoy SW, Effgen S, Chiarello LA, et al. Training Manual School-Physical Therapy Interventions for Pediatrics (S-PTIP) Data Form (Version 4, 2014). https://www.uky.edu/chs/sites/chs.uky.edu/files/Docs/Revised-S-PTIP-Training-ManualPT-COUNTS-7_20_14-1.pdf
24. Randolph G, Esporas M, Provost L, Massie S, Bundy DG. Model for improvement-part 2: measurement and feedback for quality improvement efforts. Pediatr Clin North Am. 2009;56(4):779–798.
25. Provost LP, Murray S. The Health Care Data Guide: Learning From Data for Improvement. New York, NY: John Wiley & Sons; 2011.
26. Gannotti ME, Christy JB, Heathcock JC, Kolobe TH. A path model for evaluating dosing parameters for children with cerebral palsy
. Phys Ther. 2014;94(3):411–421.
27. Horn SD, Gassaway J. Practice-based evidence
study design for comparative effectiveness research. Med Care. 2007;45(10, suppl 2):S50–S57.
28. NINDS. PT Individual Session Form 2017. https://www.commondataelements.ninds.nih.gov/CP.aspx#tab=Data_Standards
. Accessed Dece-mber 6, 2018.