Lightfoot, Marguerita PhD*; Rotheram-Borus, Mary Jane PhD†; Comulada, W Scott DrPH†; Reddy, Vanessa S MS‡; Duan, Naihua PhD§
Prevention of HIV transmission from patients living with HIV (PLH) is a high national priority.1 From 33% to 50% of PLH continue to have unprotected sex after learning their HIV serostatus.2 To reduce PLH transmission behaviors, a few efficacious interventions for reducing sexual and substance use risk behaviors have been designed and evaluated.3-5 Most existing evidence-based programs are delivered to PLH individually or in a small group delivery modality at community-based agencies or in case management settings.3-9 These delivery modalities face significant barriers to broad dissemination; each is relatively expensive ($250-$1500 per patient),10-12 requires coordination of schedules for multiple individuals, incentives for participation, transportation and child care, and is not designed to anticipate relapse over time. Therefore, strategies that are easier to implement and sustain to eliminate sexual transmission acts among PLH are needed.
We developed an intervention that focused primarily on enhancing motivations without providing the intensity of the existing evidence-based programs for PLH. Motivational interviewing (MI)13,14 is a brief psychotherapeutic intervention strategy intended to increase the probability that a person will enter into, continue, and adhere to a specific change strategy aimed at reducing harmful behaviors. MI has successfully reduced alcohol abuse among outpatients15 and pregnant women16 and reduced sexual risk behaviors.17 Given MI's previous success in reducing sexual risk, we aimed to examine the efficacy of MI to reduce transmission acts among PLH.
To deliver MI, we examined the efficacy of two potential delivery strategies: 1) a computerized program delivered in waiting rooms before a medical appointment; and 2) using the positive relationship with a provider to enhance motivation to reduce transmission. Computer-assisted instruction has been used therapeutically with phobic patients,18 depressed patients,19,20 overweight patients,21 patients with eating disorders,22 diabetics,23,24 and patients with obsessive-compulsive disorder.25 As a result of these successes, computer-based interventions have been widely advocated in the fields of health education and prevention.26,27 However, it is unknown whether PLH will use computerized programs and whether such programs could be used for reducing sexual risk transmission.
A more traditional strategy is to use providers to deliver MI. Physicians have been effective in delivering brief interventions for depression,28 smoking,29-32 alcohol abuse,33,34 weight and diet,35 and physical activity.36 Counter to stereotypes, physicians consistently implement interventions when trained.37 Recently, using providers for delivering HIV preventive interventions has shown promise.38,39
To examine the relative efficacy of these two strategies, our intervention was delivered by providers or computers in medical care settings. A quasiexperimental design was implemented in which all PLH in one medical clinic were assigned to one condition (computerized MI, provider MI, or standard care). As a result of contamination concerns, PLH were not randomized within the clinic. Thus, we compare the efficacy of computer-delivered and provider-delivered interventions in reducing sexual transmission acts of PLH over time. Both MI strategies were expected to be efficacious, but the comparative advantages were unknown.
Design and Objectives
Six medical clinics were assigned to one of three conditions: 1) computer-delivered (N = 2 clinics); 2) physician-delivered (N = 2 clinics); or 3) standard care (N = 2 clinics).
Between December 2001 and May 2004, 872 PLH were recruited from six medical clinics that served at least 100 PLH in Los Angeles County, CA. Clinics included four community health centers and two health maintenance organizations (HMOs). To be eligible, patients had to be HIV-seropositive, receiving care from one of the participating health clinics, older than 18 years old, and able to read and comprehend English. Exclusion criteria included severe neuropsychologic impairment, involvement in another HIV behavioral intervention trial, or self-identification as transgender. Study PLH were recruited by clinic or research staff members. There were days on which research staff were not in the clinic and it was not possible to document which PLH were approached by clinic staff on those days. Consequently, although there was a comprehensive approach to recruitment, with both clinic staff and research staff, we were unable to obtain consistent and reliable refusal rates. The refusal rate based on the research staff's efforts only was 28%. The Institutional Review Boards at each of the participating institutions and clinics approved all study procedures.
Clinic or research staff approached and recruited PLH, referring patients to research staff for completion of consent forms. PLH in all intervention conditions were asked to complete an assessment of sexual risk acts before completing their appointment with their treating physician. When the PLH presented to the receptionist at the medical clinic during regularly scheduled routine visits, the PLH who had agreed to participate in the study received the assessment. Consequently, assessment and intervention delivery did not occur on a prescribed schedule but rather when PLH were scheduled for routine care. The assessments were conducted on laptop computers using audio computer-assisted self-interviewing. Audio computer-assisted self-interviewing approach has been proposed as an effective method of decreasing social desirability and, thereby, enhancing veracity of self-report of sensitive behaviors, including sexual risk acts.39,40 The assessment took approximately 15 to 20 minutes to complete. From this assessment, two measures were calculated: 1) the number of HIV-/unknown partners or sexual partners of unknown serostatus; and 2) the number of unprotected vaginal and anal sex acts with HIV-/unknown partners or partners of unknown status. These data were obtained from an extensive sexual history in which the PLH reported the number of sexual partners over the last 3 months, the types and frequency of sexual encounters, and condom protection during acts.41
Study participation was confidential and all data were collected anonymously. A unique identifier was constructed from PLH responses to a series of five questions (ie, what are the first two letters of your mother's first name), allowing for matching of each patient's assessments over time. After completion of the assessment, PLH in the computerized- and the physician-delivered conditions received a summary sheet that included a personal risk profile (ie, number of sexual partners, percent of unprotected anal/vaginal sex). PLH in the standard care clinics completed the assessment but did not receive a printed summary of their sexual risk acts. The staffing at each clinic was equal and the assessment took approximately 15 minutes to complete.
The computerized program was designed and field-tested with PLH. The design process was implemented to ensure the program was clear, interesting, and acceptable. The computerized intervention included: 1) assessing personal values; 2) encouraging personal responsibility and responsibility towards other people and comparing their sexual risk acts with their values; 3) providing feedback regarding the congruence between personal values and behavior (see Fig. 1); and 4) encouraging self-efficacy to implement new actions.12 The computerized intervention took approximately 10 minutes to complete.
Once the patient completed the risk assessment, the computer would provide the feedback and ask the patient if any of the feedback surprised or concerned them. The computer would then provide feedback regarding congruence of values and behavior. Once the feedback was delivered for at least three values, the patient received suggestions for possible behavioral changes and encouraged the patient in their decisions.
A manual was written for training of providers in medical settings that covered each of the components. Providers were trained to deliver the intervention in a supportive, nonjudgmental manner, including: 1) discussing the patient's personal values; 2) encouraging personal responsibility and responsibility toward other people and comparing their sexual risk acts with their values; 3) providing feedback regarding the congruence of personal values and behaviors; and 4) reinforcing the patient's sense of self-efficacy regarding their ability to make or maintain behavioral or lifestyle changes. To assist providers to address each of these components, a protocol was provided that followed a FRAMES framework12: 1) Feedback: provide feedback to the patient exploring inconsistencies between stated values and behavior; 2) Responsibility: reinforce the patient's responsibility for their own behavior and changes; 3) Advice: provide acceptable advice and feedback in a concise, nonjudgmental manner; 4) Menu of options: reinforce that the patient has many options for improving behavior and health; 5) Empathy: express empathy for the difficulties in making potential behavioral changes, and 6) Self-efficacy: build self-efficacy in the patient. Healthcare providers at the assigned clinics were provided continuing education credits and attended an 8-hour training on the intervention protocol. The intervention was designed to be delivered at the end of the patient's scheduled medical appointment within a 5- to 15-minute interaction with PLH. Depending on clinic staffing, providers included physicians, physician assistants, nurses, health educators, and case managers. To ensure adherence to the intervention protocol, providers were supplied a pocket-sized version of the protocol to be used during sessions. In addition, both PLH and providers were asked to complete separate short quality assurance forms after each clinic visit. If the quality assurance forms indicated the provider was drifting from the intervention protocol, they were provided a brief booster training. Providers spent 5 to 15 minutes delivering the intervention.
Patients received no additional intervention.
Sample Size and Statistical Methods
Power calculations were carried out in two steps. First, we calculate the needed sample size per intervention arm (N* = 126) in RMASS2 software.42 We assume 80% power, a Type I error of 0.05 for a two-sided test, 11 repeated measurements, from baseline to 30 months at 3-month intervals, an attrition rate of 2% between follow ups, resulting in 82% of the sample remaining at 30 months, and an autoregressive correlation structure on repeated measurements with an autocorrelation (ie, the correlation between adjacent repeated observations) of 0.50. In the second step, we adjusted for the clustering of patients within clinics by estimating the intraclass correlation based on previous research.43 For clinic size m, the minimum needed sample size per intervention arm was N = [1 + (m-1)ICC]N* = 134.
Baseline differences on continuous and categorical sociodemographics and other background characteristics, between intervention conditions, were examined using analysis of variance and chi square tests, respectively. Fisher exact test was used for categorical measures when contingency table cell counts were small.
Sociodemographic characteristics were not homogeneous across clinic. This was expected as a result of the quasiexperimental study design; intervention conditions were assigned at the clinic level and clinics differed in the clientele they served. As a result of the small number of clinics in the study, the two clinics in the standard care condition were also non-HMO clinics.
Similar to Sanguanwongse,44 propensity scores45,46 were used to produce a more accurate estimate of the true association between the intervention condition assignment and sexual behavior outcomes if the baseline characteristics that differed across intervention conditions also influenced sexual behavior. Propensity scores reduce potential biases that often occur when intervention effects are estimated in quasiexperimental trials.47,48 Propensity scores are used to create strata. Within each stratum, intervention conditions are homogeneous. Analyses are conducted within each stratum and the results are combined across strata.
First, propensity scores (the probabilities of being in the computerized or provider condition versus the standard care condition) were derived from a logistic regression on covariates for gender and baseline measures that differed across intervention conditions (Table 1). There were no HMO clinics in the standard care condition, so this variable was not used to determine propensity scores. PLH were then ordered by their propensity scores and divided into five equally sized strata. There were sufficient numbers of participants in each condition across strata.
We ran two-way analysis of variance, logistic regressions, and multinomial regressions to compare continuous, binary, and multinomial baseline characteristics across intervention conditions postpropensity scoring to examine how well the propensity score removed imbalances between intervention conditions. We ran two sets of models. The first set of models contained the intervention condition and stratum as covariates. Test statistics for main effects by intervention condition, postpropensity scoring, are also provided in Table 1. This analysis confirms that the propensity scores removed overall baseline differences in the covariates. The second set of models added intervention condition by stratum interactions as covariates to make sure differences across intervention conditions did not occur within the strata; no significant differences were found.
Random-intercept regression models were run to test randomized assignment to an intervention condition on sexual behavior outcomes over time.49-51 adjusting for the strata each PLH were categorized into. Logistic and Poisson regressions were fit for binomial and count sexual behavior outcomes, respectively. A random intercept was included for each person to account for the correlation between repeated assessments. To model differing time intervals between the baseline assessment and start of the intervention across PLH, we fit piecewise regressions that included covariates for the time from the baseline assessment (time 1) and time from the start of the intervention (time 2). Covariates were also included for each stratum, each two-way stratum by time 1 interaction, each two-way stratum by intervention condition interaction, and each three-way stratum by time 2 by intervention condition interaction. Our model parameterization allowed mean sexual behavior trajectories to have differing intercepts and slopes across each intervention condition within each stratum. A change in slope was also modeled after the start of the computerized or provider interventions within each stratum.
Overall intervention effects are estimated as the average effect of the time 2 by intervention condition interactions across the five strata.
We focus on the adjusted analyses to make proper inferences but examined the unadjusted analyses to get a picture of how much bias was removed in the adjusted analyses. Assuming all covariates potentially leading to systematic differences across intervention conditions at baseline were included, adjusted and unadjusted analysis results leading to similar conclusions suggest PLH across intervention conditions are comparable. Therefore, we fit random-intercept regression models, unadjusted for strata, by excluding covariates for the strata and interactions with the strata.
Analyses were conducted in SAS Version 9.1 software (SAS Institute Inc., Cary, NC); regression models were fit using the GLIMMIX macro.
Flow of Participants, Follow Up, and Sample Characteristics
Of the 843 PLH that completed the informed consent, 763 (91%) completed the first assessment (Fig. 2). PLH had access to the assessments and intervention every 3 months for up to 30 months (n = 11 potential assessments) during their regularly scheduled medical appointment. PLH who did not attend the clinic on a quarterly basis completed the program when the PLH came to the clinic. If a PLH was seen in the clinic more often, the program was given only on the visit that corresponded to the 3-month timeframe so as to correspond with routine care visits (DHHS, 2005) and not include acute care visits. If more than one assessment was completed within a single 3-month period, only the second assessment was used in the analysis (52 “duplicate” observations were deleted).
To examine the efficacy of the interventions, all PLH that completed only the first assessment were removed, resulting in a reduction of the sample size to 566 PLH. PLH completed only one assessment because they no longer received care or attended multiple clinics returning infrequently to any one clinic. These patients did not significantly differ from those that completed more than one assessment. Given the differences in care provision across clinics, the rate of PLH return visits varied across clinic with an overall mean of 26% of PLH not returning to the clinic during the course of the study (range, 12%-34% per clinic). An additional 37 PLH were removed from the analysis because of incomplete baseline data on covariates used to construct propensity scores in the analysis, resulting in a final sample size of 529 PLH. The median time from first assessment to receiving the intervention was 4.1 months (maximum, 29.7 months).
Given that PLH completed the risk assessment quarterly, overall, PLH completed the risk assessment 4.9 times (standard deviation, 2.2; range, 2-11 assessments). The number of risk assessments completed did not vary significantly between experimental conditions. In addition, the PLH in the provider condition received the intervention a mean number of 2.9 times (standard deviation, 1.9; range, 0-8) and PLH in the computerized condition received the intervention a mean number of 2.2 times (standard deviation, 1.9; range, 0-8).
Baseline sociodemographic and sexual behavior characteristics by intervention condition are shown in Table 1. PLH differed across intervention conditions on multiple characteristics: age (F = 3.61, df = 2, 526, P = 0.03), ethnicity (χ2 = 15.80, df = 4, P < 0.01), high school graduation (χ2 = 15.36, df = 2, P < 0.01), self-reported financial status (χ2 = 22.77, df = 2, P < 0.01), home ownership (χ2 = 9.60, df = 2, P < 0.01), receiving medical services from an HMO or non-HMO provider (Fisher exact test, P < 0.01), undetectable viral load (Fisher exact test, P = 0.01), and the number of sexual partners (F = 4.18, df = 2, 526, P = 0.02). A lack of HMO clinics in the control condition could not explain all the differences; after removal of the HMO clinics, significant differences remained for ethnicity, owning a home, and having an undetectable viral load.
Multiple comparison tests showed the significant differences were between the standard care and both intervention conditions (all P < 0.05), except for detectable viral load in which the significant difference was between the computerized and provider conditions (χ2 = 6.94, df = 2, P = 0.03). Significant differences were also present between the computerized and provider conditions for whether PLH received services from an HMO (χ2 = 5.89, df = 1, P = 0.02). PLH did not differ significantly across intervention conditions by gender, recent employment, CD4 cell counts, sexual abstinence, the fraction of protected sex acts with HIV-/unknown partners, or the number of HIV-/unknown sexual partners.
Sexual behavior outcome trajectories estimated by the random-intercept regression models are shown in Figure 3 for each intervention condition. The “adjusted” trajectories are the average trajectories across the five strata derived by the propensity scores. The “adjusted” trajectories are derived for each intervention condition in two stages. First, we derive the trajectories within each stratum constructed previously using propensity scores. Next, we average the trajectories across the five strata. Plots of trajectories “adjusted” and “unadjusted” for propensity scores provide a visual comparison of how much the background characteristics that differ across intervention conditions bias the results, ie, influence the impact of the intervention on the outcomes over time. Adjusted and unadjusted trajectories are similar, suggesting little bias was present.
PLH decreased the number of HIV-/unknown sexual partners in the computerized condition compared with the standard care condition (t = 2.79, df = 1952, P < 0.01) and compared with the provider condition (t = 2.34, df = 1952, P = 0.02) over time. PLH significantly decreased the number of unprotected acts with HIV-/unknown sexual partners in the computerized condition compared with the standard care condition (t = 3.23, df = 2029, P < 0.01) over time.
As a sensitivity analysis, we also conducted unstratified analyses (Fig. 3) on the subset of PLH who did not receive services from an HMO. We hypothesized that the non-HMO subset would have fewer imbalances than the whole sample and lead to results that are similar to the stratified analyses. The main difference between the non-HMO subset analyses and stratified and unstratified analyses is that although the direction of the finding is the same and smaller, there is the loss of a significant intervention effect on the number of HIV-/unknown partners in the computerized condition compared with the standard care condition. The significant finding for number of unprotected acts with HIV-/unknown partners remained.
This intervention trial found that brief, motivational interventions implemented in medical care settings can be efficacious in reducing sexual transmission risk acts, particularly when delivered by a computer. PLH receiving the computer-delivered intervention reported they significantly reduced their unprotected sex acts with HIV- or unknown partners as well as reducing the numbers of unprotected sex acts overall when compared with the provider-delivered intervention and standard care condition. These findings suggest that incorporating HIV prevention in medical care settings can play a key role in reducing the transmission of HIV.
The computer-delivered intervention was significantly better at reducing sexual transmission behavior than the provider-delivered intervention. The computer offers several unique advantages to interpersonal interventions that may account for this finding. First, computers ensure private and confidential reporting of risk acts, creating a context that elicits greater honesty and reporting of risk by PLH.37,52,53 Second, the computer eliminates the interpersonal barriers of avoidance, denial, discomfort, and confidentiality issues. The computer is able to control output and feedback in such a way that the branching and decision-making process is dependent on the user's choices or responses, creating a private and personalized experience.54 Third, feedback received from the computer may be considered to be objective and have no value judgments and consequently more accepted by participants. Fourth, other types of self-behavior management programs (eg, smoking elimination) have demonstrated that repeatedly monitoring one's own behavior can be effective, for example, monitoring one's food reduces intake of food55 and monitoring one's blood sugar reduces insulin rates over time.56 Previous research also indicates that self-monitoring of sexual behavior results in reductions of sexual risk behaviors.57
These findings must be interpreted in the context of the limitations of the study. Some argue that sexual risk reduction interventions should be assessed with biologic end points such as seroconversion or acquisition of sexually transmitted infections. However, it was not feasible in this trial to assess seroconversion among partners of HIV-infected individuals given the practical challenges of recruiting and maintaining a cohort of such individuals. A sexually transmitted infection end point was also problematic because it is possible to contract such infections from other HIV-infected individuals. Although our self-report outcome of sexual transmission risk acts is subject to demand of social desirability, we countered this threat by assurances of confidentiality, careful construction of assessment items, and use of audio computer-assisted self-interviewing as recommended by the National Institutes of Health.58 Because program implementation required recruitment by clinic providers, we were unable to document refusal rates or assess differences between those PLH recruited and those that were not. Consequently, it is possible that providers were biased in who was referred to the study. We used well-established methodology to account for socioeconomic differences that occurred across intervention conditions. Despite adjusting for measures that would most likely be associated with sexual behavior, it is possible there were unobserved measures that could have been adjusted for. This is a drawback of all quasiexperimental studies and does not diminish the importance or strength of our findings.
A number of interventions have successfully reduced the HIV transmission risk behaviors of PLH, but these interventions are limited in their ability to reach a large number of PLH. To increase the reach of preventive interventions, implementation in medical care settings must be considered as currently promoted by the US Centers for Disease Control and Prevention.59 However, to meet the needs of care settings, these interventions must be brief and easily integrated into the clinical settings. A recent meta-analysis of HIV preventive interventions found that most efficacious interventions do not address these important needs.60 The current study suggests interventions provided in medical care settings can be efficacious in reducing sexual risk behaviors. The intervention used the already existing healthcare delivery system and did not require significant change in that system or staffing to be implemented. Although a formal cost-effectiveness analysis was not conducted, the intervention's limited use of resources, both of time and personnel, suggests the cost-efficiency of this strategy. The costs of the computerized program are in the iterative process of program design that must be invested before the implementation of any program. Once the program has been designed, the implementation costs are relatively small. Other costs for a computerized intervention may emerge if there is a need to refine and adapt the program after implementation. However, computerized programs allow tailoring to subgroups of PLH based on a variety of characteristics: ethnicity, sexual preferences, gender, age, or a variety of other issues. Although the computer-delivered intervention lacked the power of an interpersonal relationship with a healthcare provider, computer-based interventions offer important, effective, and ongoing opportunities to deliver interventions in highly accessible settings. Computer-based interventions are particularly promising because many provider-based interventions are difficult to sustain behaviors and may be an efficient strategy to reduce HIV transmission among PLH.
M. Lightfoot originated the study and supervised all aspects of its implementation. M. J. Rotheram-Borus assisted with writing the manuscript and study development. W. S. Comulada supervised the statistical analysis. V. Reddy assisted with study development and writing. N. Duan assisted with the study interpretation and analyses. All authors helped to conceptualize ideas, interpret findings, and review drafts of the manuscript.
We acknowledge the collaboration and commitment of the participating clinics: 5P21 HIV/AIDS Clinic at the LAC + USC Medical Center, Los Angeles Gay and Lesbian Community Services Center, Northeast Valley Health Corporation, Kaiser Permanente Harbor City, Kaiser Permanente Los Angeles, and St. Mary's Medical Center. We also thank the participants for their generous participation in this research. Study protocol, survey instruments, and informed consent forms and procedure were reviewed and approved by the Institutional Review Board (IRB) of the University of California, Los Angeles and the respective clinics.
1. Centers for Disease Control. Incorporating HIV prevention into the medical care of persons living with HIV. MMWR Morb Mortal Wkly Rep
2. Crepaz N, Marks G. Towards an understanding of sexual risk behavior in people living with HIV: a review of social, psychological, and medical findings. AIDS
3. Kalichman SC, Rompa D, Cage M, et al. Effectiveness of an intervention to reduce HIV transmission risks in HIV-positive people. Am J Prev Med
4. The Healthy Living Project Team. Effects of a behavioral intervention to reduce risk of transmission among people living with HIV: the Healthy Living Project randomized controlled study. J AIDS
5. Rotheram-Borus MJ, Lee MB, Murphy DA, et al. Teens Linked to Care Consortium. Efficacy of a preventive intervention for youths living with HIV. Am J Public Health
6. Crepaz N, Lyles CM, Wolitski RJ, et al. Do prevention interventions reduce HIV risk behaviours among people living with HIV? A meta-analytic review of controlled trials. AIDS
7. Rotheram-Borus MJ, Flannery D, Lester P, et al. Prevention for HIV-positive families. J Acquir Immune Defic Syndr
. 2004;37(Suppl 2):S133-S134.
8. Gore-Felton C, Rotheram-Borus MJ, Weinhardt LS, et al. The Healthy Living Project: an individually tailored, multidimensional intervention for HIV-infected persons. AIDS Educ Prev
. 2005;17(Suppl A):21-39.
9. Richardson JL, Milam J, McCutchan A, et al. Effect of brief safer-sex counseling by medical providers to HIV-1 seropositive patients: a multi-clinic assessment. AIDS
10. Holtgrave DR, Kelly JA. Preventing HIV/AIDS among high-risk urban women: the cost-effectiveness of a behavioral group intervention. Am J Public Health
11. Pinkerton SD, Johnson-Masotti AP, Otto-Salaj LL, et al. Cost-effectiveness of an HIV prevention intervention for mentally ill adults. Ment Health Serv Res
12. Pinkerton SD, Kelly JA, Johnson-Masotti AP, et al. Cost-effectiveness of an HIV risk reduction intervention for adults with severe mental illness. AIDS Care
13. Resnicow K, Baskin ML, Rahotep SS, et al. Motivational interviewing in health promotion and behavioral medicine. In: Cox WM, Klinger E, eds. Handbook of Motivational Counseling
. New York: Wiley; 2004.
14. Miller WR, Rollnick S. Motivational Interviewing Preparing People to Change Addictive Behavior
. New York: Guilford; 1991.
15. Bien TH, Miller WR, Boroughs JM. Motivational interviewing with alcohol outpatients. Behav Psychother
16. Handmaker NS. Motivating Pregnant Drinkers to Abstain: Prevention in Prenatal Care Clinics
. Albuquerque, NM: University of New Mexico; 1993.
17. Kalichman SC, Cain D, Weinhardt L, et al. Experimental components analysis of brief theory-based HIV/AIDS risk-reduction counseling for sexually transmitted infection patients. Health Psychol
18. Marks IM, Kenwright M, McDonough M, et al. Saving clinicians' time by delegating routine aspects of therapy to a computer: a randomized controlled trial in phobia/panic disorder. Psychol Med
19. Marks IM, Mataix-Cols D, Kenwright M, et al. Pragmatic evaluation of computer-aided self-help for anxiety and depression. Br J Psychiatry
20. Osgood-Hynes DJ, Greist JH, Marks IM, et al. Self-administered psychotherapy for depression using a telephone-accessed computer system plus booklets: an open US-UK study. J Clin Psychiatry
21. Williamson DA, Martin PD, White MA, et al. Efficacy of an Internet-based behavioral weight loss program for overweight adolescent African-American girls. Eat Weight Disord
22. Bara-Crril N, Williams CJ, Pombo-Carril MG, et al. A preliminary investigation into the feasibility and efficacy of a CD-ROM-based cognitive-behavioral self-help intervention for bulimia nervosa. Int J Eat Disord
23. Glasgow RE, La Chance PA, Toobert DJ, et al. Long-term effects and costs of brief behavioural dietary intervention for patients with diabetes delivered from the medical office. Patient Educ Couns
24. King DK, Estabrooks PA, Strycker LA, et al. Outcomes of a multifaceted physical activity regimen as part of a diabetes self-management intervention. Ann Behav Med
25. Greist JH, Marks IM, Baer L, et al. Behavior therapy for obsessive-compulsive disorder guided by a computer or by a clinician compared with relaxation as a control. J Clin Psychiatry
26. Casazza K, Ciccazzo M. Improving the dietary patterns of adolescents using a computer-based approach. J Sch Health
27. Goodman J, Blake J. Nutrition education: a computer-based education program. J Health Care Poor Underserved
. 2005;16(Suppl A):118-127.
28. Rost K, Nutting PA, Smith J, et al. Designing and implementing a primary care intervention trial to improve the quality and outcome of care for major depression. Gen Hosp Psychiatry
29. Hollis JF, Lichtenstein E, Vogt TM, et al. Nurse-assisted counseling for smokers in primary care. Ann Intern Med
30. Stevens VJ, Severson H, Lichtenstein E, et al. Making the most of a teachable moment: a smokeless-tobacco cessation intervention in the dental office. Am J Public Health
31. Cornuz J, Zellweger JP, Mounoud C, et al. Smoking cessation counseling by residents in an outpatient clinic. Prev Med
32. Hartmann KE, Thorp JM, Pahel-Short L, et al. A randomized controlled trial of smoking cessation intervention in pregnancy in an academic clinic. Obstet Gynecol
33. Ockene JK, Adams A, Hurley TG, et al. Brief physician- and nurse practitioner-delivered counseling for high-risk drinkers: does it work? Arch Intern Med
34. Senft RA, Polen MR, Freeborn DK, et al. Brief intervention in a primary care setting for hazardous drinkers. Am J Prev Med
35. Ockene IS, Hebert JR, Ockene JK, et al. Effect of physician-delivered nutrition counseling training and an office-support program on saturated fat intake, weight, and serum lipid measurements in a hyperlipidemic population: Worcester Area Trial for Counseling in Hyperlipidemia (WATCH). Arch Intern Med
36. Calfas KJ, Long BJ, Sallis JF, et al. A controlled trial of physician counseling to promote the adoption of physical activity. Prev Med
37. Gonzalez-Willis A, Rafi I, Boekeloo B, et al. Using simulated patients to train physicians in sexual risk assessment and risk reduction. Acad Med
38. Kiene SM, Fisher JD, Fisher WA. Linking prevention with care: HIV risk reduction interventions in clinical settings for persons living with HIV. In: Kalichman S, ed. Positive Prevention: Reducing HIV Transmission Among People Living With HIV-AIDS
. New York: Kluwer Academic/Plenum Publishers; 2005:219-244.
39. Turner CF, Ku L, Rogers SM, et al. Adolescent sexual behavior, drug use, and violence: increased reporting with computer survey technology. Science
40. Gribble JN, Miller HG, Rogers SM, et al. Interview mode and measurement of sexual behaviors: methodological issues. J Sex Res
41. Weinhardt LS, Kelly JA, Brondino MJ, et al. HIV transmission risk behavior among men and women living with HIV in 4 cities in the United States. J Acquir Immune Defic Syndr
42. Hedeker H, Gibbons R, Waternaux C. Sample size estimation for longitudinal designs with attrition: comparing time-related contrasts between two groups. Journal of Educational and Behavioral Statistics
43. Agarwal GG, Awasthi S, Walter SD. Intra-class correlation estimates for assessment of vitamin A intake in children. J Health Popul Nutr
44. Sanguanwongse N, Cain KP, Suriya P, et al. Antiretroviral therapy for HIV-infected tuberculosis patients saves lives but needs to be used more frequently in Thailand. J Acquir Immune Defic Syndr
45. Rosenbaum PR, Rubin DB. Reducing bias in observational studies using subclassification on the propensity score. J Am Stat Assoc
46. D'Agostino RB Jr. Propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group. Stat Med
47. Girou E, Brun-Buisson C, Taille S, et al. Secular trends in nosocomial infections and mortality associated with noninvasive ventilation in patients with exacerbation of COPD and pulmonary edema. JAMA
48. Lindenauer PK, Pekow P, Wang K, et al. Lipid-lowering therapy and in-hospital mortality following major noncardiac surgery. JAMA
49. Weiss R. Modeling Longitudinal Data
. New York: Springer; 2005.
50. Bryk AS, Raudenbush SW. Hierarchical Linear Models: Applications and Data Analysis Methods
. Newbury Park, CA: Sage Publications; 1992.
51. Reise SP, Duan N. Multilevel Modeling: Methodological Advances, Issues, and Applications
. Mahwah, NJ: Lawrence Erlbaum Associates; 2002.
52. Jones EF, Forrest JD. Underreporting of abortion in surveys of US women: 1976 to 1988. Demography
53. Turner CF, Danella RD, Rogers SM. Sexual behavior in the United States, 1930-1990: trends and methodological problems. Sex Trans Dis
54. Chewning B, Mosena P, Wilson D, et al. Evaluation of a computerized contraceptive decision aid for adolescent patients. Patient Educ Couns
55. Wisotsky W, Swencionis C. Cognitive-behavioral approaches in the management of obesity. Adolesc Med
56. Guerci B, Drouin P, Grange V, et al. Self-monitoring of blood glucose significantly improves metabolic control in patients with type 2 diabetes mellitus: the Auto-Surveillance Intervention Active (ASIA) study. Diabetes Metab
57. Lightfoot, M, Rotheram-Borus, MJ, Comulada, S, et al. Self-monitoring of behavior as a risk reduction strategy for persons living with HIV. AIDS Care
58. Pequegnat W, Fishbein M, Celentano D, et al. NIMH/APPC workgroup on behavioral and biological outcomes in HIV/STD prevention studies: a position statement. Sex Transm Dis
60. Herbst JH, Sherba RT, Crepaz N, et al. A meta-analytic review of HIV behavioral interventions for reducing sexual risk behavior of men who have sex with men. J Acquir Immune Defic Syndr
© 2010 Lippincott Williams & Wilkins, Inc.