Many intervention programs to increase physical activity (PA) among children, even those developed by gifted investigators with substantial funding, have had no effects, small effects, or small effects in unplanned subsets of the participating children. One possible explanation for such low efficacy or effectiveness is a lack of knowledge of how change programs induce behavior change. A better understanding of such mechanisms holds the promise of enhancing the effectiveness of change programs.
The mediating variable model has been proposed as a framework for understanding program-induced change (1). The simplest representation of this model appears in Figure 1. This model suggests that change in outcomes (behavior) is the result of change in mediating variables (e.g., social, ecologic, or psychosocial variables), and changes in the mediating variables are induced by implementing programs designed to change them. Moderating variables are those for which outcomes of the intervention are different. (This is usually detected by a significant treatment group by moderating variable interaction term.) For example, if a program worked for girls but not boys, gender would be a moderating variable. And it is possible that implementation could have a direct effect on behavior (e.g., through the use of rewards), but the longer-term maintenance of those effects is not promising once the implementation (and rewards) were completed.
A detailed framework for program implementation has been proposed as a basis for program process evaluation (2). This implementation model goes beyond simple implementation to include participant use of the materials or procedures. The implementation and mediation models can be merged into a composite model of mechanisms of behavior change from intervention to outcome as seen in Figure 2.
Figure 2 deserves some explanation. An intervention must be designed and developed. Incorporated into design and development are considerations of theory that involve mostly selecting and targeting mediating variables, an understanding of the target population from the perspective of the target behavior and theory (often obtained from qualitative research), selection or creation of procedures necessary to manipulate mediating variables in desired directions (1), anticipation of the barriers to be encountered in delivering the program, and anticipation of factors influencing all subsequent steps in a probabilistic chain, leading to behavior change. Selection of a theory not strongly related to behavior will seriously impair program effectiveness because change in its variables will not result in substantial change in behavior (lack of torque). Similarly, theory limited in its application by gender, age, or ethnic group factors, or use of procedures that do not effectively change theory-based mediators (1) will seriously impair program effectiveness.
With the intervention in hand, appropriate individuals need to be selected and trained to deliver the intervention. Issues of concern here include selection criteria for trainees, the ability of trainees to deliver an intervention, the extent of the intervention in which the trainees were trained, and the fidelity of that training to the original intervention design. The total amount of training delivered can be thought of as the product of extent by fidelity. Trainees may or may not appreciate (prefer) the training and material presented. A shortcoming in any one of these areas could impair the resulting efficacy or effectiveness of the intervention (8).
With training complete, the intervention must be implemented. One can think of how much of the intervention was delivered (i.e., extent), and whether it was delivered in such a way as to clearly reflect the design of the intervention (i.e., fidelity). The total amount of program delivered can be thought of as the product of extent and fidelity. Factors that influence implementation/delivery include the types of intervention components (content) and the levels of complexity in their delivery, and the types and levels of resources necessary to effectively deliver the intervention as designed. Too extensive content, too difficult levels of content, or inadequate resources (e.g., even a sublime physical education curriculum requires balls, jump ropes, and other activity-related equipment resources) can impair implementation and mitigate change down the paths of effects to outcomes.
Although a program can be delivered in its entirety (extent × fidelity), it may not reach its intended audience (e.g., it may have been offered on evenings when the intended audiences could not attend). Reach can vary by the percent of people reached (spread) or by the amount of programming delivered past the barriers (depth). Total program reach can be thought of as the product of spread and depth. Issues in regard to reach include whether people being reached are representative of the target population, and those in greatest need.
Among people reached (each represented by the supine quadrangles in Figure 2), some will have more exposure to the program and some less. Exposure will vary by the extent of the program to which the person was exposed and a person's preference for that type and amount of program. Whereas the concepts of reach and exposure are similar, reach deals more with program delivery and exposure more with program reception. For example, it is possible that a program has full reach, but the person sitting there does not receive the program (e.g., because of personal or environmental distractions). Substantial program exposure should motivate participant use of self-control procedures (e.g., goal setting, self-monitoring, and problem solving).
To maximize an effect on behavior, the person must use the materials or procedures delivered in the program exposure (e.g., use the pedometers distributed, implement the exercise routines) (8). This material or procedure use can vary by type and amount of use. Full use can be thought of as the product of type and amount. In regard to a tailored intervention, use would refer to the materials and information tailored to that individual. Although it is important that participants initially use some of the program materials or procedures, it is usually more important that they continue to use it (usually over months or years). Initial and continued use should change the theory-specified mediating variables that should, in turn, change the targeted behaviors.
A pervasive aspect of this model is the presence of barriers. All barriers vary in type and difficulty. For example, training barriers can include not having sufficient time in the classroom teacher's day to implement training, or not anticipating the way in which a group of trainees prefer (or benefit the most from) training. Implementation barriers may include trainees who believe they already made sufficient changes, but who are really below program standards, or classroom teachers who have too many other time commitments to make the new program a priority. Delivery barriers could be messages that are too text-dense to meet the needs of participants, or inadequate time and venues (e.g., no school assembly) to initiate a program. Use barriers include not having the resources, for instance, not having the time to exercise. Any of these barriers could be reasons why programs underachieve success. Experienced program developers and implementers have an advantage over novices to the extent that their experience enables them to correctly anticipate the many barriers in program delivery, and they have learned ways to minimize or overcome these barriers (effective program problem-solving).
Whereas this mechanistic model makes intuitive sense, we are not aware of anyone who has used such a model to design or evaluate their program. As a result, there are no published studies that can totally validate the model. Alternatively, there is a body of literature on mediation of physical activity interventions (4,7) and on process evaluation of intervention programs (3, 9–13). We review mediation and process evaluation literature to assess the extent to which we can find support for this model. The article ends with a consideration of implications of the model for program design and evaluation.
Literature on Process Evaluation
The Active Winners Trial (11) evaluation showed how process variables can help explain the effectiveness of a physical activity intervention, especially how problems at one point in program delivery can impair delivery at subsequent points. Active Winners was a school- and community-based intervention that included four components: an after-school and summer activity program, a home component that included newsletters and homework assignments, a school component that was designed to make physical activity attractive within the school, and a community component designed to promote activity within the surrounding community. Although intriguingly conceived, the study did not result in a significant increase in self-reported physical activity (11). Process evaluations in the form of meeting records, interviews, attendance, and surveys of participants, staff, and key informants, provided information about resources, reach, dose, and content of the intervention. Review of study logs and time indicated that the majority of staff time was devoted to implementing the summer and after-school sessions, but this concentration of effort prevented the full implementation of other study components (exhaustion of resources leading to partial program implementation). Attendance records highlighted that whereas 82% of the participants were reached by one or more sessions of the program, only 5% attended at least half of the total sessions provided (limited depth of reach). Although the after-school and summer school sessions were implemented as planned, there was insufficient production (underimplementation) and distribution (underreach) of the newsletters. The school committees that were necessary for the active school component were not implemented, and there was only limited community awareness and promotion of the campaign suggesting the community element was also only partially implemented. Discipline issues limited the activity component (exposure) of some, particularly the early, activity sessions. Thus, these factors singly or in combination likely accounted for the lack of a change in the physical activity of the participants, but we cannot be sure, because these analyses were not conducted. If these variables had not been assessed, the likely reasons for lack of effect would not be known, and it would not be clear what could be done to improve program delivery the next time.
Pathways (12,14), a large, multisite elementary school-based intervention designed to lower percent body fat among Native American children, was another study in which process evaluation was used to explain the effectiveness of an intervention. The Pathways intervention included diet and physical activity components. The physical activity aspects included a modified physical education curriculum and a family component designed to create positive environments for diet and physical activity at home. There were statistically significant differences in physical activity between the intervention (14) and control groups at the end of the study using the self-report methods, but not using an accelerometer (6), and no differences in adiposity. Process evaluation (12) indicated that teacher attendance at training sessions (training extent) ranged from 92 to 98%. Teachers completed evaluations of training and materials at the end of each semester (training content). The teachers believed that the curriculum was designed well and was interesting for the students (training preference). All schools reported that each physical education lesson was at least 30 min in duration and was taught on average 81% of the available days that it could have been taught (extent of implementation). Across the 3 yrs of the study, 47% of the students had a parent or guardian who attended a family event (spread of reach) designed to promote a healthy diet and physical activity home environment. The authors did not relate process evaluation to mediators or outcomes. Thus, we cannot be sure of the reasons for lack of effect on adiposity or objectively assessed physical activity. Limited reach of the family component may have been responsible for part of this lack of effect. The high levels of completed training and school based program delivery suggest that either the mediators were not appropriate to this sample or that the procedures selected to manipulate these mediators were not effective. Future versions of programs for Native American children need to more carefully address the selection of mediators and the processes to manipulate mediators.
Process measures were used extensively in the physical activity components of the Child and Adolescent Trial for Cardiovascular Health (CATCH) (9,10). CATCH was a 3-yr multicenter field trial implemented in 96 schools in grades 3, 4, and 5. The physical education intervention focused on increasing time spent in moderate to vigorous intensity physical activity. Existing classroom teachers delivered the CATCH physical education curriculum (9). Some of these teachers were nonphysical education regular classroom teachers (perhaps limited trainee ability) and others were physical education specialists (9). Minutes of activity per lesson were assessed by observation using the SOFIT system; 94.3% of the physical education specialists attended training, as opposed to 60.5% of classroom teachers (low extent of training for the latter). Expert study staff modeled teaching techniques five times more often for classroom teachers than specialists (49 vs 10% of visits) (likely a form of remedying the low extent of training through enhanced total implementation) (9). The physical education specialists also reported more favorable responses to the curriculum materials (content) than classroom teachers (6.8 vs 6.4) (a form of training preference) (9). Observations indicated that group sizes were appropriate in 94% of the classes observed and 76% of the classes started within 5 min of the scheduled start time (implementation fidelity) (10). Warm-ups were included in 87% of the classes, with 55% including a cool-down (modest implementation extent in the latter) (10). The intervention schools provided more moderate to vigorous physical activity than the control schools (more depth and spread of reach) (9). There were larger increases in the minutes of activity among the classroom teachers than among the specialists (4.5 vs 1.0 min) (9). However, there was increased activity in the classes taught by the specialists with the specialists including more time for drills (4.5 vs 2.7 min) and for knowledge components (4.4 vs 3.2 min) (9). Thus, the process measures indicated that the intervention was largely implemented as planned, reaching large numbers of children, and thus was responsible for the increased activity. However, the more detailed comparisons between the classroom and specialist teachers indicated that increased change was possible among the less experienced classroom teachers, but increased activity was obtained among those teachers with more previous experience.
These three programs adopted different process measurement approaches. The approaches supported various elements of the model as they showed how process measures can be used to explain why an intervention did or did not achieve its desired behavioral objectives. None of these studies statistically related the processes to mediators or outcomes, which would have confirmed what appear to be likely (but at this point unconfirmed) relationships. Some of the process measures provided information that can be used to revise the design of future studies, thereby moving the field forward.
Literature on Mediating Variables
Few studies have assessed the effects of programs on mediators or of mediators on outcomes in childhood physical activity interventions (11). Criteria for establishing mediation have been proposed (1). Mediation was tested in the Lifestyle Education for Activity Promotion (LEAP) intervention study (4). LEAP was based on social cognitive theory and designed to increase physical activity among high school girls. The study was designed to enhance physical activity self-efficacy through successful experiences with physical activity in and out of school, as well as developing the behavioral skills to maintain a physically active lifestyle into adulthood (4). Physical activity was assessed using the 3-d physical activity recall. Mediators included self-efficacy, outcome expectancy, and goal setting, assessed by self-report. The intervention resulted in small, but statistically significant, increases in self-reported physical activity and physical activity self-efficacy, and self-efficacy partially mediated the effect of the intervention (4). Thus, physical activity self-efficacy accounted for a portion of the effect of the intervention on children's physical activity.
This study highlights the potential of mediating variables to describe how interventions induce change in outcomes of physical activity intervention trials. Relating process variables to mediators would further delineate pathways of effects.
Implications for Program Design and Theory
Identifying the pathways of effects from intervention to outcomes can inform program design in several ways: 1) strong positive pathways, that is, expected changes in processes along the pathways, confirm the procedures for promoting change, thereby identifying effective components of the interventions which should be maintained in future versions; 2) weak positive pathways suggest pathways for which enhanced procedures should be sought to enhance impact on mediators, or new more strongly related mediators need to be sought; 3) pathways of no changes identify the pathways that need revision to enhance future program outcomes; and 4) moderators may identify stratification variables for which different types of interventions should be offered to different strata. In addition, there are implications for theory. Positive pathway relationships with mediating variable analyses confirm the importance of these theoretical constructs in the design of the program; pathways not supported by mediating relationships suggest new theoretical variables are needed to effect change.
Process evaluation has often been used as a method of quality control to enhance the function of that specific intervention (13). If the data are processed rapidly enough (days or weeks), feedback from the process evaluation can be used to revise program practices and, hopefully, thereby increase desired outcomes. This is an important function of process evaluation. The role of the mechanistic model in such cases would be to guide and specify the development of the intervention to anticipate all pathways and key steps in each pathway. Figure 3 structures the sequence of activities in program design, delivery, and evaluation. It would be up to the process evaluator to specify appropriate measures at key steps in each pathway to feedback to program staff.
The graphic representations presented here (Figs. 1 and 2) are relatively simple with only one or two pathways from intervention to outcome. In the real world, there are many such linear relationship pathways with the possibility of nonlinear relationships between variables (statistical interaction terms) in distinct pathways. Albeit complex, the underlying conceptual framework is probabilistic, that is, probabilities can be associated at each step in the chain of events from intervention to outcome. Knowing these probabilities should give guidance to intervention design. A variety of more complex, often nonlinear, decision models have been proposed in other social science disciplines (15). Future research will need to explore the extent to which these approaches hold more promise for understanding intervention effects.
Time is an unexplored aspect of the model in Figure 2, except for sequentiality of effects (i.e., time irreversibility). The optimal duration of an intervention is not clearly known. Whether outcomes always decline as time from the end of the intervention increases is also not known. The authors believe, however, that no programs will be effective for a substantial time after the end of a program because no one knows what variables internal to the person will forever maintain an effect and because the environment is usually constantly working against the changes intended. Thus, programs of one type or another likely will be necessary to initiate and to maintain behavior change.
In the current article, we emphasize the potential for use of process evaluation in better understanding how interventions function and identifying effective components of interventions. From the scientific perspective, randomized clinical trials are the optimal method for determining efficacy or effectiveness of a specific intervention component. Given the very high costs of such field trials (often millions of dollars), we need to find ways to use the data available from large trials to determine component effectiveness. In addition, the sequential pathways to outcomes need to be determined to better understand how interventions function.
Most published process evaluations have reported percentage completion or attainment at several points in program delivery and have not related process to outcome. This is because high levels of delivery are generally desired, and if high levels of delivery can be accomplished, there is little or no variability with which to assess sequential dependencies. Sometimes the units on which process data are collected (e.g., classes) may not be the same as the units of outcome (e.g., students). Thus, more attention needs to be devoted to analysis of such data.
In many process evaluations, moderate levels (e.g., 40–60%) of delivery are obtained on average. The variability around this average delivery can be related to variability around delivery of the next step, to mediators and outcomes, with or without moderators. A caution must be applied in that moderate delivery and variation around that delivery could be related to third variables, which also affect variations around delivery in the next step in the pathway and not reflect causal relationships.
If low levels of delivery are obtained (e.g., <40%), the process and method of delivery need to be rethought and revised procedures and anticipatory problem-solving applied to the barriers.
If high levels of delivery are obtained (e.g., >80%), this is very good from the point of view of the program and likely success, but little or no variability is left for analysis of sequential dependencies. What could be learned, alternatively, is the maximum level of change that could be obtained in the next step from the high level of implementation at this step.
There has been no work on assessing total implementation, that is, combining extent and fidelity. For each task to be implemented, one could be concerned with whether it was implemented (extent of implementation) and how well each was implemented (fidelity) (Fig. 4). Thus, total implementation would be the sum of fidelity scores for each task to be implemented. Total implementation should be more closely related to change along the pathway than extent of implementation alone. More work must elucidate these processes.
This mechanistic model differs from the RE-AIM model (5) in that the purpose of RE-AIM is to understand broader public health implications, whereas the purpose of the mechanistic model is to understand how intervention programs induce change in participants' behavior.
A better understanding of how intervention programs work holds the promise of refining program design and implementation across sequential projects. A probabilistic mechanistic model of program delivery proposes multiple pathways for program delivery, strategic steps along each pathway, and relation of process to mediators and outcomes. Identifying barriers and sequential dependencies across steps should enable ensuing programs to devise solutions to the barriers and procedures to enhance attainment of change at each sequential step in a pathway. This model is a preliminary specification, which likely will be refined as experience is gained from its use.
This study was funded by a grant from the American Cancer Society (ACS TURSG-01). This work is also a publication of the USDA/ARS Children's Nutrition Research Center, Department of Pediatrics, Baylor College of Medicine and Texas Children's Hospital, Houston, Texas. This project has been funded in part by federal funds from the USDA/ARS under co-operative agreement 58–6250–6001. The contents of this publication do not necessarily reflect the views or polices of the USDA, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.
1. Baranowski, T., C. Anderson, and C. Carmack. Mediating variable framework in physical activity interventions. How are we doing? How might we do better? Am. J. Prev. Med.
2. Baranowski, T., and G. Stables. Process evaluations of the 5-a-day projects. Health. Educ. Behav.
3. Davis, M., T. Baranowski, M. Hughes, C.L. Warneke, C. De Moor, and R. Mullis. Using children as change agents to increase fruit and vegetable consumption among lower-income African-American parents: Process evaluation results from the Bringing it Home Program. In: Process evaluation for public health interventions and research,
edited by A. Steckler and L. Linnan. San Francisco: Jossey-Bass, 2002, pp. 249–267.
4. Dishman, R.K., R.W. Motl, R. Saunders, G. Felton, D.S. Ward, M. Dowda, and R.R. Pate. Self-efficacy partially mediates the effect of a school-based physical-activity intervention among adolescent girls. Prev. Med.
5. Glasgow, R.E., T.M. Vogt, and S.M. Boles. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am. J. Public Health
6. Going, S., J. Thompson, S. Cano, D. Stewart, E. Stone, L. Harnack, C. Hastings, J. Norman, and C.B. Corbin. The effects of the Pathways Obesity Prevention Program on physical activity in American Indian children. Prev. Med.
7. Lewis, B.A, B.H. Marcus, R.R. Pate, and A.L. Dunn. Psychosocial mediators of physical activity behavior among adults and children. Am. J. Prev. Med.
8. Lichstein, K., B.W. Riedel, and R. Grieve. Fairtests of clinical trials: A treatment implementation model. Advances in Behavior Research and Therapy
16: 1–29, 1994.
9. McKenzie, T.L., E.J. Stone, H.A. Feldman, J.N. Epping, M. Yang, P.K. Strikmiller, L.A Lytle, and G.S. Parcel. Effects of the CATCH physical education intervention teacher types and lesson location. Am. J. Prev. Med.
10. McKenzie, T.L., P.K. Strikmiller, E.J. Stone, S.E. Woods, S. Ehlinger, K.A. Romero, and S.T. Budman. CATCH: Physical activity process evaluation in a multicenter trial. Health Educ. Q.
Suppl 2:S73–S89, 1994.
11. Pate, R.P., R.P. Saunders, D.S. Ward, G. Felton, S.G. Trost, and M. Dowda. Evaluation of a community-based intervention to promote physical activity in youth: lessons from active winners. Am. J. Health Promot.
12. Steckler, A., B. Ethelbah, C.J. Martin, D. Stewart, M. Pardilla, J. Gittelsohn, E. Stone, D. Fenn, M. Smyth, and M. Vu. Pathways process evaluation results: a school-based prevention trial to promote healthful diet and physical activity in American Indian third, fourth and fifth grade students. Prev. Med.
13. Steckler, A. and L. Linnan (Eds.). Process Evaluation for Public Health Interventions and Research.
San Francisco: Jossey-Bass, 2002.
14. Stevens, J., M. Story, K. Ring, D.M. Murray, C.E. Cornell, Juhaeri, and J. Gittelsohn. The impact of the Pathways intervention on psychosocial variables related to diet and physical activity in American Indian schoolchildren. Prev. Med.
15. Wan, H., A. Hunter, and P. Dunne. Autonomous Agent models of stock markets. Artificial Intelligence Review.