Trained practices identified significantly more developmental or behavioral concerns, with 18% of reviewed charts noting a concern, compared to 9% in those same practices before training (odds ratio: 2.34; 95% confidence interval: 1.42, 3.85; p = .001) (Table 3). The increase in children identified occurred across most practices trained.
In those charts containing some evidence of developmental surveillance, there were no differences in the child's age, sex, insurance status, or type of delay in trained vs. untrained practices. Of those children where a developmental concern had been identified, there were also no differences in these factors by practice training status.
Provider calls to Child Development Infoline (CDI).
There were 1217 calls from community-based practices to CDI over the study period. Of these, 33% were from practices trained by the date of the call, while 67% of calls were from practices that were not or had not yet been trained. Calls to CDI from intervention practices increased over the training period. By the end of the training period, 44% of practices had been trained, but 57% of calls came from trained practices.
The average age of children referred to CDI from practices that had received the intervention was 23.0 months, compared to an average age referred from non-trained practices of 20.9 months (p = .006). Fourteen percent of calls from trained practices were for children over 3, compared to 6.4% for untrained practices (p < .0001). Among the 1106 calls for children ages 0 to 36 months, the average age of children referred from trained practices was 18.1 months, compared to 19.0 months (p = .11) from untrained practices.
There were significant differences in the conditions for which young children were referred in trained vs. untrained practices. Behavioral conditions were involved in 4.2% of referrals from trained practices, compared to 1.4% for untrained practices (p = .005). Four percent of referrals from trained practices were for parent support, compared to 0.8% from untrained practices (p = .0002). Trained practices referred relatively less frequently for health and communication issues. There were no differences by practice training status in the sex, language spoken at home, and Medicaid status of referred children.
Questionnaires were collected from 517 staff members from 105 practices, representing 85% of the recorded attendance at those trainings. Seventy percent of providers reported that they expected a very significant or significant change in their practice of developmental surveillance attributable to the training. Providers reported various barriers to conducting developmental surveillance, including lack of time, with 26.9% describing lack of time as a significant or very significant barrier. Other reported barriers to conducting developmental surveillance were lack of medical office staff support (15.5%), inadequate reimbursement (11.3%), lack of confidence in ability to conduct developmental surveillance (5.5%), and unspecified barriers (23.6%). Ninety-three percent of providers and staff reported that they definitely or probably intended to use the Help Me Grow program in their practices. Fifteen percent of providers and staff reported that the overall effect of all barriers to using Help Me Grow would be significant or very significant.
This brief, in-office training program for community-based pediatric and family medical practices was successful in influencing provider and practice behavior. Among practices receiving the intervention, the identification of children with developmental or behavioral concerns doubled. Practices that received the intervention referred children to CDI in greater numbers than control practices. The training increased referrals to CDI for older children, and for children with behavioral issues or a family need for parent support, although the chart review study did not find any differences in age or types of need in children identified with concerns. Trained practices may be differentially referring to CDI, using the CDI more often for certain groups of children, perhaps where there are fewer available alternatives.
The model used in developing the evaluation design assumed that the training program would lead to more consistent use by the medical practices of certain techniques of developmental surveillance (i.e., chart instruments, solicitation of parental concerns, and consistent monitoring of the child's development) and that this would be expressed in a greater frequency of chart notations and completed instruments. The model further assumed that as a result of this consistent use, more children with developmental and behavioral concerns would be identified. Although more children with developmental and behavioral delays were identified in the trained practices, it is unclear what changes in provider and staff practices, knowledge, or attitudes resulted in this change. There were no changes in charted evidence of developmental surveillance after training.
The rates of developmental surveillance noted in the charts were very high, over 90% in both intervention and control practices. This is much higher than recent surveys conducted by American Academy of Pediatrics4 and is contrary to current understanding of the state of developmental surveillance in pediatric and family practice. The high rates of surveillance may have been driven by a liberal definition of a "completed instrument" as one type of evidence of surveillance. Other types of evidence of surveillance were present in the charts with less frequency, including a note on development (45% of charts prior to intervention) and a note on parental concerns (30% of charts).
Further research on training pediatric and family providers in the use of developmental surveillance and referral may wish to investigate:
- Medium- and long-term effects of this program-How long does the increased rate of identification of children last? What is the longer-term effect of the training on calls to CDI?
- Mechanism-What changes in provider behavior or attitudes have been effected by the intervention? What is the mechanism leading to greater identification of children with developmental delays? How can these techniques or changes in attitude be more effectively transmitted to providers?
- Attendance-In practices that were trained, only half of the providers attended the training program. Since the primary effect of the program is on provider behavior, how can this attendance rate be improved in future training, and what effect will it have?
All of these questions are potential future issues for study that can build on the strong base of the program Strengthening the Developmental Surveillance and Referral Practices of Child Health Providers, which successfully influenced provider surveillance and referral behaviors for children in primary care practices across Connecticut.
We are grateful to The Commonwealth Fund for the generous support of the evaluation of the training program and to Edward Schor, MD for his guidance and encouragement.
1. Glascoe FP. Parents' evaluation of developmental status: how well do parents' concerns identify children with behavioral and emotional problems? Clin Pediatr
2. Frankenburg WK, Dodds J, Archer, et al. The Denver II: a major revision and restandardization of the denver developmental screening test. Pediatrics
3. Squires J, Lawanda P, Bricker D. The ASQ User's Guide
. 2nd ed. Baltimore, MD: Paul H. Brookes; 1999.
4. American Academy of Pediatrics, Division of Health Policy Research. Identification of children <36 months at risk for developmental problems and referral to early identification programs. Executive Summary. April 2003. Available on online at http://www.aap.org/research/periodicsurvey/ps53exs.htm
APPENDIX: IMPACT EVALUATION DESIGN
The chart review study was conducted among a group of practices enrolled in the ProHealth physicians' network. Ten intervention practices of varying sizes and patient populations provided access to their charts for review of well-child visits both before and after the intervention. Ten practices that did not receive training were matched on practice characteristics with the first ten, and served as a comparison group. For each outcome of interest, results from each practice that received training were compared to results in two comparison groups-the same practice prior to training, and a matched practice not receiving training.
REFER CENTER DATA
This study examined records of calls from primary care providers to Child Development Infoline (CDI) during the one-year training period. In the CDI data, the practice of the referring primary care provider was matched to the program database to determine whether the call came from a practice receiving or not receiving training, and whether the call occurred prior to or following the training session. Monthly trends in calls during the training period were examined to determine if there were increases in the number of calls to Help Me Grow and whether trained practices were disproportionately represented among practices calling or referring to Help Me Grow.© 2006 Lippincott Williams & Wilkins, Inc.