ONE OF THE well-documented disruptive effects of traumatic brain injury (TBI) is severe and persistent familial stress.1–3 Compounding factors, such as the financial repercussions of a lengthy hospitalization, the ongoing medical needs of the survivor,4,5 protracted legal negotiations or litigation, and changes in household responsibilities, parenting, employment, and social ties,6,7 can all leave family members without the social support they need to cope.8
There is considerable evidence that when family members and service providers work collaboratively, the complicated problems resulting from brain injury are solved more effectively.9,10 Families need strategies to address the challenges associated with brain injury, particularly in a healthcare climate that significantly limits services provided to individuals with brain injuries and their families.11 In the last decade, the average length of inpatient rehabilitation declined significantly, and the majority of patients are now discharged directly to the home environment.12 Therefore, families are quickly confronted with providing primary support for the injured individual as well as the responsibility for navigating the rehabilitation and social service arenas, tasks for which they feel largely unprepared.13,14 Furthermore, the need for services does not abate across the life span, as most brain injuries occur prior to age 35 and survivors are expected to live for decades after injury.15 The long-term onus for improving quality of life and functional ability for survivors rests with families.16
FAMILIES AS ADVOCATES
Effective family advocacy leads to better outcomes for both survivors and their families, diminishes caregiver burden, and reduces the overall need for services.17,18 This is, in part, because families can prioritize concerns and services better than medical or social service personnel working alone.19,20 Also, because the social/behavioral effects of brain injury escalate over time in the absence of appropriate services, effective family advocacy reduces the likelihood that survivors will require expensive, last resort options such as institutionalization.21 For families to be effective advocates, they need to know how to identify and engage professionals and obtain resources.11,22,23 The research on disability and family/professional interactions has identified 3 critical areas to help family members become effective advocates. First, family members need to be able to (a) clearly identify and summarize problems, (b) prioritize, set goals, and evaluate progress, and (c) use effective communication skills.24,25 Second, families must learn to access information and resources to minimize disruption of services.26 Third, family members need access to social support as a buffer against the challenging demands of caregiving.27
Web-based technologies can effectively and efficiently deliver information and services to families affected by brain injury. Wade and colleagues28,29 examined the efficacy of Web-based interventions that allow families of children with brain injury to communicate online with therapists. They reported both a therapeutic benefit (ie, reduction in problem behaviors and increased knowledge) and participant satisfaction with the technology and therapeutic relationship. Our recent randomized trial successfully tested a computerized program targeting parent-teacher communication—the Brain Injury Partners: Advocacy Skills for Parents program.30
This randomized controlled trial examined the efficacy of a Web-based intervention designed to improve family advocacy skills. The Brain Injury Partners (BIP) program (http://adult.braininjurypartners.com/) is an interactive Web site for families of adults with brain injury. The present study's primary research questions were as follows:
- Does use of the BIP program by families of individuals with brain injury increase knowledge of effective advocacy skills when compared with a control condition?
- Does use of the BIP program increase family members' ability to apply effective advocacy skills in a video situations test (VST) when compared with a control condition?
- Does use of the BIP program increase family members' intention to use effective advocacy skills when compared with a control condition?
- Does use of the BIP program result in improved perception of overall life satisfaction when compared with a control condition?
METHODS
Participants
Participants were recruited through advertising by the Brain Injury Association of America (BIAUSA). Potential participants accessed an online screening tool on the BIAUSA Web site (BIAUSA.org). Criteria for participation included (a) family member of an adult with a brain injury; (b) providing at least limited support (ie, checking in occasionally, helping with some activities); (c) English speaking; and (d) access to a high-speed Internet connection.
An overview of participant characteristics is given in Table 1. A total of 201 family members of an adult with brain injury from 42 states participated in the study. Table 2 presents characteristics of the survivors of brain injury.
;)
TABLE 1: Participant characteristics
TABLE 2: Demographics of care recipients
Procedures
Brain Injury Partners Web site
The BIP Web site was developed to (a) train family members in advocacy skills, particularly the core skills needed for effective communication including active listening and problem solving (eg, appropriate body language, acknowledging different perspectives); (b) help users find a broad range of services and supports (ie, through external links and a library of articles on the Web site; (c) provide strategies for reducing stress (eg, coping with grief, guilt, and burnout through healthy living and stress management and requesting help); and (d) help determine necessary supports (eg, independent living needs, transition planning). The training uses text, interactive video examples, and video-based skills exercises. Content for the Web site was developed from sources including (a) the text-based FAST (Family Advocacy Skills Training) program,32 (b) evidence-based articles written for caregivers, and (c) interviews with family members and professionals with experience in brain injury. Use of the Web site is self-directed and requires no special training.
Data collection
Evaluation data were collected through Survey Console, a secure research hosting Web site. Following determination of eligibility, participants completed an online informed consent form and the pretest (T1). Participants were then randomly assigned to treatment or control groups and either (a) granted access to the program (treatment group) or (b) directed to the BIAUSA Web site (control group). Like the BIP program, the BIAUSA site contains information for caregivers about managing stress, requesting help from friends and family, and obtaining services; however, its main advocacy focus is legislative. Control participants were asked to use the Web site for a minimum of 30 minutes to match the minimum time treatment participants would view the BIP program. Treatment participants received e-mailed log-in information and a link to the program. Both groups could contact a research assistant by phone or e-mail if they had problems or questions. After 10 days, all participants received a link to complete the posttest (T2). Following T2 completion, treatment participants were encouraged to continue using the program. Both treatment and control participants completed the follow-up assessment (T3) 3 months after completing T2, and control participants were then given access to the BIP site.
Measures
Computer-based pretest, posttest, and follow-up measures were used to assess knowledge, skill application, behavioral intention, and overall life satisfaction. In addition, program use data were collected to evaluate treatment participants' exposure to the BIP materials.
Demographics
In addition to providing the demographic information in Tables 1 and 2, participants answered questions about the types of services and supports they had already attempted to secure for their family member and how successful they felt. The demographics questionnaire contained 11 questions and was completed at T1.
Knowledge
Knowledge measures included 18 questions about the key learning objectives related to communication skills. For example, participants viewed a list of behaviors and chose those that were important for demonstrating active listening (eye contact—correct; explaining how you feel—incorrect). Participants also used a 5-point Likert scale to rate the importance of specific behaviors/strategies when interacting with care providers (eg, how important is it to explain how you feel about the situation where 1 = not at all important and 5 = very important). Scores on the knowledge measures were the percentage of correct items.
Skill application
Treatment enactment33 was assessed through a series of 17 VSTs. The VST assesses subject responses to realistic situations and behaviors.34 After watching each situation, participants rated the family member's effectiveness on a 5-point Likert scale (1 = ineffective; 5 = effective). Each response was assigned a score from 1 to 5, with a higher score representing a more accurate appraisal of effectiveness. In some cases, a 1 on the Likert scale corresponded with an accurate appraisal (ie, the coding was reversed). The skill application scale showed marginal internal consistency (α = .78) and good test-retest reliability in the control condition (r = 0.78).
Behavioral intention
Following the Theory of Reasoned Action, which posits that attitudes and behavioral intentions predict behavior,35 we also assessed behavioral intention using the VSTs. Participants watched scenarios of a family member communicating with a service provider and indicated on a 5-point Likert scale how likely they were to respond that way in a similar situation (1 = not likely; 5 = likely). Again, we assigned each response a score from 1 to 5, representing intention-to-use appropriate or intention-to-not-use inappropriate responses. The behavioral intention scale showed marginal internal consistency (α = .78) and good test-retest reliability in the control condition (r = 0.76).
Satisfaction with Life Scale36
The Satisfaction with Life Scale (SLS) assesses agreement with 5 statements addressing overall satisfaction with life (eg, in most ways, my life is close to my ideal), with a 7-point Likert scale score ranging from “strongly disagree” to “strongly agree.” Participants completed the SLS at T1 and T3. The SLS showed excellent internal consistency (α = .88) and good test-retest reliability in the control condition (r = 0.68).
User satisfaction and program usability
At posttest, treatment participants completed a 12-item questionnaire on program usability (eg, appearance, navigation, level of engagement during use) and satisfaction with the program (helpfulness, level of enjoyment, likeliness to recommend the program, and overall satisfaction). Participants rated 8 program usability statements on a 5-point Likert scale (where 1 = strongly disagree and 5 = strongly agree). The 4 satisfaction questions were rated on a 7-point scale (where 1 = not at all and 7 = extremely).
With the exception of the SLS, the measures were created by the authors to evaluate changes specific to BIP program use. The participants completed the same knowledge, skill application, and behavioral intention measures at each of the 3 assessments.
Program use
Using Google Analytics, we collected information regarding each treatment participant's program use—specifically, the date and time specific pages were accessed between initial program access and T2—to calculate the number of times the site was accessed and the total time spent using the program.
RESULTS
Baseline equivalency, attrition, and missing data analyses
Study conditions were compared using the caregiver demographic characteristics shown in Table 1, the care receiver characteristics in Table 2, and all of the pretest outcome measures. No significant (at P < .05) differences were found with the exception of sex of the care receiver (χ21,200 = 4.49, P = .034). The treatment condition reported 24% female care receivers compared with 38% for the control condition. Sex of the care receiver was used as a covariate in all subsequent statistical models.
The number of assessments completed did not differ significantly by study condition (χ22,201 = 5.36, P = .068). Two percent of treatment participants completed only 1 assessment, 12% only 2 assessments, and 86% completed all 3 assessments compared with 3%, 3%, and 94%, respectively, for control participants. Participants who completed all 3 assessments were compared with those who did not complete them on study demographic characteristics and pretest outcome measures. No significant differences (at P < .05) were found.
Group differences
Statistical approach
One-way multivariate analysis of covariance (MANCOVA), adjusting for pretest measures, was conducted to determine the overall multivariate between-subject effect at the posttest and follow-up assessment. Univariate analysis of covariance (ANCOVA), adjusting for pretest measures, was used to evaluate between-subject effects in each of the outcome measures at posttest and follow-up. We employed an intent-to-treat analysis by using maximum likelihood estimates to impute missing data with the NORM software program,37 as it produces more accurate and efficient parameter estimates than listwise deletion or last-observation-carried-forward.38
The one-way between-subjects MANCOVA at posttest, adjusting for pretest measures, was significant (F3,193 = 20.28, P < .001, d = 1.12) and corresponds to a large multivariate effect size. Subsequent univariate ANCOVA models (see Table 3) showed that all 3 posttest measures, controlling for pretest measures, yielded statistically significant group differences. Skill application had the largest effect size (d = 1.01 large effect), followed by intentions (d = 0.70, medium to large effect) and knowledge (d = 0.67, medium to large effect).
TABLE 3: Descriptive statistics for baseline and posttest outcome measures and pretest to posttest analysis of covariance resultsa
The same statistical methods were applied at follow-up. The one-way between-groups MANCOVA, adjusting for pretest measures, was significant (F5,189 = 7.91, P < .001, d = 0.92) and corresponds to a large multivariate effect size. Follow-up univariate ANCOVA models (see Table 4) showed that 3 of the 4 follow-up measures, controlling for pretest measures, produced statistically significant group differences, with skill application having the largest effect size (d = 0.84, large effect), followed by intentions (d = 0.80, large effect) and knowledge (d = 0.32, small to medium effect). Satisfaction with life, assessed at follow-up but not at posttest, was not significant (P = .054 with d = 0.29, small to medium effect).
TABLE 4: Descriptive statistics for baseline and follow-up outcome measures and pretest to follow-up analysis of covariance resultsa
Moderation effects
Advocacy status and knowledge
The univariate ANCOVA models described earlier were rerun to include a study condition by advocate status (based on pretest self-report on the demographic questionnaire as either “ineffective” or “effective” advocate) interaction term to test the hypothesis that the program would have greater effects for participants who indicated they were ineffective advocates compared with effective advocates. Examination of the moderator term showed a significant condition by advocate status interaction effect for the knowledge items at the posttest (F1,195 = 9.43, P = .002) and follow-up assessments (F1,195 = 6.21, P = .014). No significant condition by advocate status interactions were found for the other study outcomes at posttest or follow-up. To elucidate the significant interaction terms, separate ANCOVA models were run for effective and ineffective advocates. Results at posttest showed a significant between-subjects condition effect on knowledge for both effective (F1,145 = 6.98, P = .009, d = 0.44) and ineffective (F1,54 = 27.11, P < .001, d = 1.48) advocates. Results at follow-up showed a nonsignificant between-subjects condition effect for effective advocates (F1,145 = 0.71, P = .402, d = 0.14) and a significant effect for ineffective advocates (F1,54 = 12.60, P = .001, d = 0.99). The between-subjects condition effect on pretest-adjusted knowledge for ineffective advocates was approximately 3 times greater than the condition effect on pretest-adjusted knowledge for effective advocates at posttest and approximately 7 times greater at follow-up.
Education level and behavioral intentions
Univariate ANCOVA models tested whether level of education (0 = no college degree, 1 = college degree) moderated the condition effects reported in Tables 3 and 4. Examination of the moderator term showed a significant interaction effect on pretest-adjusted intentions at the posttest (F1,200 = 5.25, P = .023) and follow-up (F1,200 = 4.35, P = .038) assessments. No significant condition by education level interactions were found for the other study outcomes at posttest or follow-up. Separate ANCOVA models for participants with and without a college degree at posttest revealed a significant between-subjects condition effect on intentions for participants without a college degree (F1,111 = 26.81, P < .001, d = 1.00), but no significant condition effect on intentions for participants with a college degree (F1,88 = 1.91, P = .171, d = 0.30). Results at follow-up showed a significant between-subjects condition effect on intentions for participants without a college degree (F1,111 = 24.79, P < .001, d = 0.99) and with a college degree (F1,88 = 5.25, P = .024, d = 0.50). The differential effect on intentions for participants without a college degree was approximately 3.5 times greater than for participants with a college degree at posttest and approximately 2 times greater at follow-up.
Program usage and dose-response analysis
Most treatment participants (51%) made 1 visit to the program Web site, 24% made 2 visits, 18% made 3 or more visits, and 7% did not visit the Web site at all. The average amount of time spent on the Web site across all visits was 57 minutes (SD = 47 minutes), with a median of 45 minutes. Program use data were not available for the BIAUSA Web site.
To assess whether more time on the site resulted in greater improvement in the study outcome measures, we computed change scores for treatment participants (posttest minus pretest and follow-up minus pretest) and correlated both with time spent on the program Web site. A significant correlation (r = 0.24, P = .016) was found with the knowledge item posttest change score. The observed correlation corresponds to a small effect size. No other significant correlations were detected.
User satisfaction and program usability
User satisfaction and program usability items indicated a favorable perception of the program. An overall satisfaction score computed across the 4 items yielded a mean score of 5.14 (SD = 1.46), indicating that treatment participants were quite satisfied with the program. Users were also asked to agree or disagree with program usability statements on a 5-point scale. An overall program usability score computed across the 8 items produced a mean score of 4.21 (SD = 0.53), indicating that the treatment participants found the program quite usable.
DISCUSSION
This study evaluated the efficacy of the BIP program in training family members in the key components associated with effective advocacy. The results demonstrated significant improvement at both posttest and follow-up in participants' knowledge of the key program components (ie, active listening and problem-solving skills), their application in video-based situations, and their intention to use those skills compared with the control group. Likewise, participants who used the program demonstrated greater increases in intention-to-use advocacy skills. These results, related to our first 3 research questions, were evident at posttest and follow-up. Our fourth research question, related to overall life satisfaction, was measured only at pretest and at follow-up. This comparison did not reach statistical significance. It is unsurprising that such a brief intervention would not affect this broad measure. The treatment group indicated satisfaction with the program and its content.
We also examined the effects of several participant variables on outcomes to determine who benefited most from the program. First, treatment participants who rated themselves as ineffective advocates at pretest demonstrated greater gains in knowledge than participants who rated themselves as effective advocates. This finding was only present for the knowledge measures. However, it suggests that it may be beneficial to include more advanced materials to address the needs of individuals with more advanced skill bases. Second, the program had greater intention effects among users with no college degree than among users with a college degree This finding was true only for intention. It is possible that individuals with less education were somewhat more motivated to learn and apply the skills because they had less exposure to this type of information/training than those with college experience. Third, time spent viewing the program correlated somewhat with improvement on knowledge items at posttest, but this dose-response effect was no longer present at 3-month follow-up. This finding suggests that booster sessions could be a beneficial approach to enhancing longer-term effects.
The primary limitations of this study are the participant sample and aspects of the measurement process. Although the sample was taken from a broad geographic region, represented rural and urban communities, and was reasonably distributed across socioeconomic levels, study participants were not ethnically or racially diverse. Future recruitment could expand specifically to include individuals from underrepresented groups. In addition, potential users without a high-speed Internet connection were excluded. Excessive cost or lack of availability in a geographic area are cited as primary reasons that home dial-up users do not have high-speed Internet connections,39 so our recruitment efforts may have limited socioeconomic and geographic representation.
Perhaps, the most significant limitation is inherent in computer-based assessment. Although measures of knowledge and skill application assessed participants' ability to discriminate between effective and ineffective family advocacy, the research design did not include a measure of participants' actual application of skills in their daily lives. Future evaluation efforts could include an observational measure to provide objective evaluation of skill use and related outcomes. Furthermore, the treatment group had access to the BIP program throughout the study period, with the potential to access the Web site while completing the measures. Although the skill application measures used VSTs with novel scenarios, access to the Web site could limit the validity of results on the knowledge measures. Also, the measures were curriculum based–aligned with content from the program not found in the control Web site and expected to produce proximal outcomes.40 The control condition should be considered a nontreatment control, as it did not provide the same information or training in key advocacy skills as did the BIP program, nor did it offer guidance through the site. The results would be strengthened through comparison of the BIP program to an analogous training program had one been available. Another study limitation involves our inability to examine dose response as a factor in analyses. Future studies should use a control condition that allows comparison of relative treatment dosage to determine whether that is a mitigating factor in outcomes. Finally, the measure of the more global construct of life satisfaction may have been too broad to be significantly altered by a short-term skills-based training. In a future study, it would be beneficial to include measures of psychosocial impact that can be expected to change to a greater degree following a short-term intervention.
This study demonstrated the effectiveness of a Web-based intervention in teaching effective skills to caregivers advocating for a family member with brain injury. The results replicate findings from our previous study.30 In both studies, caregivers demonstrated significant increases in knowledge, skill application, and intention to use the skills learned in their advocacy training program. In both cases, participants used a self-directed intervention that did not require special training, and program benefits were gained without professional assistance, demonstrating the utility of Web-based interventions. The acquired skills could result in improved family functioning and better outcomes for survivors of brain injury through greater access to services and reduced caregiver burden. The BIP program is a first step in educating and developing skills among family members who need to be effective advocates.
REFERENCES
1. Gervasio A, Kreutzer J. Kinship and family members' psychological distress after traumatic brain injury: a large sample study. J Head Trauma Rehabil. 1997;12:14–26.
2. Gillen R, Tennen H, Affleck G, Steinpreis R. Distress, depressive symptoms, and depressive disorder among caregivers of patients with brain injury. J Head Trauma Rehabil. 1998;13:31–43.
3. Verhaeghe S, Defloor T, Grypdonck M. Stress and coping among families of patients with traumatic brain injury: a review of the literature. J Clin Nurs. 2005;13:1004–1012.
4. Kosciulek J, Lustig D. Predicting family adaptation from brain injury-related family stress. J Appl Rehabil Couns. 1998;29:8–12.
5. Relyea-Chew A, Hollingworth W, Chan L, Comstock BA, Overstreet KA, Jarvik JG. Personal bankruptcy after traumatic brain or spinal cord injury: the role of medical debt. Arch Phys Med Rehabil. 2009;90:413–419.
6. Kozloff R. Networks of social support and the outcome from severe head injury. J Head Trauma Rehabil. 1987;2:14–23.
7. Sander AM, Fuchs KL, High WM Jr, Hall KM, Kreutzer JS, Rosenthal M. The community integration questionnaire revisited: an assessment of factor structure and validity. Arch Phys Med Rehabil. 1999;80:1303–1308.
8. Maitz EA, Sachs PR. Treating families of individuals with traumatic brain injury from a family systems perspective. J Head Trauma Rehabil. 1995;10:1–11.
9. Man D. Community-based empowerment programme for families with a brain injured survivor: an outcome study. Brain Inj. 1999;13:433–445.
10. Sohlberg MM, McLaughlin KA, Todis B, Larsen J, Glang A. What does it take to collaborate with families affected by brain injury? A preliminary model. J Head Trauma Rehabil. 2001;16:498–511.
11. Boschen K, Gargaro J, Gan C, Gerber G, Brandys C. Family interventions after acquired brain injury and other chronic conditions: a critical appraisal of the quality of the evidence. NeuroRehabilitation. 2007;22:19–41.
12. Granger CV, Markello SJ, Graham JE, Deutsch A, Reistetter TA, Ottenbacher KJ. The uniform data system for medical rehabilitation: report of patients with traumatic brain injury discharged from rehabilitation programs in 2000-2007. Am J Phys Med Rehabil. 2010;89:265–278.
13. Gan C, Campbell KA, Gemeinhardt M, McFadden GT. Predictors of family system functioning after brain injury. Brain Inj. 2006;20:587–600.
14. Turner B, Fleming J, Cornwell P, et al. A qualitative study of the transition from hospital to home for individuals with acquired brain injury and their family caregivers. Brain Inj. 2007;21:1119–1130.
15. Harrison-Felix CL, Whiteneck GG, Jha A, DeVivo MJ, Hammond FM, Hart DM. Mortality over four decades after traumatic brain injury rehabilitation: a retrospective cohort study. Arch Phys Med Rehabil. 2009;90:1506–1513.
16. Williams JM. The relative nature of brain injury. In: Singer GHS, Powers LE, Olson AL, eds. Redefining Family Support: Innovations in Public-Private Partnerships. Baltimore, MD: Paul H Brookes; 1996:225–238.
17. Horwitz RC, Horwitz SH, Orsini J, Antione RL, Hill DM. Including families in collaborative care: impact on recovery. Fam Syst Health. 1998;16:71–83.
18. McLaughlin KA, Ross KB. Collaborative interventions in clinical settings. J Med Speech Lang Pathol. 2006;14:xi–xiv.
19. Albin RW, Lucyshyn JM, Horner RH, Flannery KB. Contextual fit for behavior support plans: a model for “goodness-of-fit”. In: Koegel LK, Koegel RL, Dunlap G, eds. Positive Behavioral Support: Including People With Difficult Behavior in the Community. Baltimore, MD: Paul H Brookes; 1996:81–98.
20. Hood DR, Glang A, Morvant M. Family advocacy skills training: skills training for family advocates. Paper presented at: Conference on Traumatic Brain Injury in the 21st Century; 1999; Bethesda, MD.
21. Ylvisaker M, Feeney TJ. Collaborative Brain Injury Intervention: Positive Everyday Routines. San Diego, CA: Singular Publishing Group; 1998.
22. Holland D, Shigaki C. Educating families and caretakers of traumatically brain injured patients in the new health care environment: a three phase model and bibliography. Brain Inj. 1998;12:993–1009.
23. Sohlberg M, Mateer C. Cognitive Rehabilitation: An Integrative Neuropsychological Approach. New York, NY: Guilford Press; 2001.
24. Rempel RG. Advocacy and support for victims of brain injury and their families. In: Finlayson MAJ, Garner SH, eds. Brain Injury Rehabilitation: Clinical Considerations. Baltimore, MD: Williams & Wilkins; 1994:376–392.
25. Malik-Hoegh S. Working together: reflections on fostering an integrated service network for individuals with brain injury. In: Finlayson MAJ, Garner SH, eds. Brain Injury Rehabilitation: Clinical Considerations. Baltimore, MD: Williams & Wilkins; 1994:393–422.
26. Smith JE, Smith DL. No map, no guide. Family caregivers' perspectives on their journeys through the system. Care Manag J. 2000;2:27–33.
27. Ergh T, Rapport L, Coleman R, Hanks R. Predictors of caregiver and family functioning following traumatic brain injury: social support moderates caregiver distress. J Head Trauma Rehabil. 2002;17:155.
28. Wade SL, Carey J, Wolfe CR. The efficacy of an online cognitive-behavioral family intervention in improving child behavior and social competence following pediatric brain injury. Rehabil Psychol. 2006;51:179–189.
29. Wade SL, Walz NC, Carey JC, Williams KM. Brief report: description of feasibility and satisfaction findings from an innovative online family problem-solving intervention for adolescents following traumatic brain injury. J Pediatr Psychol. 2009;34:517–522.
30. Glang A, McLaughlin K, Schroeder S. Using interactive multimedia to teach parent advocacy skills: an exploratory study. J Head Trauma Rehabil. 2007;22:198–205.
31. DeNavas-Walt C, Proctor BD, Smith JC. U.S. Census Bureau, Current Population Reports, P60-238, Income, Poverty, and Health Insurance Coverage in the United States: 2009. Washington, DC: US Government Printing Office; 2010.
32. Hood D, Glang A, Lash M, Todis B. Family Advocacy Skills Training: A Handbook for Family Advocates. Eugene, OR: Teaching Research; 2001.
33. Lichstein K, Riedel B, Grieve R. Fair tests of clinical trials: a treatment implementation model. Adv Behav Res Ther. 1994;16:1–29.
34. Irvine A, Ary D, Bourgeois M. An interactive multimedia program to train professional caregivers. J Appl Gerontol. 2003;22:269–288.
35. Ajzen I, Fishbein M. Attitude-behavior relations: a theoretical analysis and review of empirical research. Psychol Bull. 1979;84:888–918.
36. Diener E, Emmons RA, Larsen RJ, Griffin S. The satisfaction With Life Scale. J Pers Assess. 1985;49:71–75.
37. NORM:
Multiple Imputation of Incomplete Data Under Normal Model [computer program]. Version 2.03. University Park, PA: Pennsylvania State University; 1999.
38. Schafer J, Graham J. Missing data: our view of the state of the art. Psychol Methods. 2002;7:147–177.
39. Pew Research Center's Internet and American Life Project. Home broadband adoption 2009.
http://pewinternet.org/Reports/2009/10-Home-Broadband-Adoption-2009.aspx. Accessed September 1, 2011.
40. Kaminski JW, Balle LA, Filene JH, Boyle CL. A meta-analytic review of components associated with parent training program effectiveness. J Abnorm Child Psychol. 2008;36:5667–5689.
