Introduction
Brisk progress in HIV therapeutics has been facilitated by well-organized, randomized, controlled clinical trial (RCT) networks developed to test the safety and efficacy of candidate antiretroviral drugs and regimens. The Adult AIDS Clinical Trial Group (ACTG) and the Terry Beirn Community Programs for Clinical Research on AIDS (CPCRA) networks, both funded by the National Institute for Allergy and Infectious Diseases (NIAID), are the largest HIV clinical trial groups in the United States. However, compared with the high rates of virologic response to highly active antiretroviral therapies (HAART) seen in these HIV RCTs, up to 50% of patients in clinic-based and prospective cohort studies fail to achieve initial or sustained responses to HAART [1–3], with survival reductions in those without durable viral suppression [1]. Differences between trial findings and clinical experience have raised concerns regarding the comparability or generalizability of HIV RCTs to actual patient care (‘real-world') settings.
Selective eligibility criteria for RCTs may limit the ability to extrapolate findings to all groups of patients [4]. In the HIV clinical trial setting, under-representation of women, ethnic minorities and intravenous drug users (IDUs) has been recognized as a persistent problem [5–9]. Although difficulties in recruitment and retention of such groups are often cited for their under-representation, RCT enrollment criteria may set up structural obstacles to equal participation. Complete information on the eligibility criteria and study enrollment procedures for clinical trials will allow HIV clinicians to assess the extent to which trial findings pertain to their patients.
Concerns regarding the generalizability of RCT findings in other chronic disease states, secondary to strict inclusion and exclusion criteria, frequent the literature [10–16]. Moreover, RCT publications may not report the complete methodologic information required to assess the selectivity of trial enrolment [17,18]; less than one-third (28%) of clinical trials report the proportion of participants screened who were trial-eligible per the enrollment criteria [19]. Even fewer provide a comparison of characteristics between excluded and included participants [20–23]. Failure to catalog the full list of eligibility criteria used for trial enrollment in subsequent publications is a topic that is generally not addressed for RCTs. Both barriers to and rates of participation in HIV clinical trials have been reported [7], but a systematic evaluation of the impact of both published and actual eligibility methods on the generalizability of HIV RCT findings has yet to be performed.
The first major aim of this study was to apply the exclusion criteria of the major ACTG and CPCRA treatment trials to a large observational cohort study of HIV-infected women in order to assess the representativeness of this population in the literature. The second major aim was to look at gaps in exclusion criteria reporting between the original protocols and subsequent journal publications of these trials.
Methods
We obtained access to full and updated protocols for ACTG and CPCRA trials through direct contact with study leaders. We examined a total of 32 phase II (efficacy only) and phase III RCTs on HIV management published between 1994 and 2004 (20 ACTG trials [24–43] and 12 CPCRA trials [44–55]), for which both the original trial protocols and journal publications were available (ACTG and CPCRA protocol titles, numbers, and dates included after each cited publication in Reference section). Every major HIV trial from these networks during this 10-year period was included. We then calculated the overall proportion of participants in the largest US-based longitudinal cohort of HIV-positive women who would have been excluded from each RCT based on eligibility criteria in the trial protocol versus criteria in the corresponding publication. We used the Women's Interagency HIV Study (WIHS) cohort for these calculations because it is sociodemographically and racially representative of HIV-infected women in the US [56,57] and because its only exclusion criterion for original enrollment was age less than 13 years.
When applying the eligibility criteria for each trial, we only considered general exclusion and inclusion criteria, such as clinical criteria (patient demographics, concurrent or past illnesses, including opportunistic infections and malignancies, life expectancy, active symptomatology, including malabsorption, and substance use), general laboratory parameters, concurrent medications, and pregnancy/lactation status. We did not include eligibility criteria inherent to a trial's study question or involving history or type of antiretroviral exposure, CD4 cell counts, or HIV viral loads. To further allow the most conservative estimate of excluded women possible, we did not include the risk of pregnancy as an exclusion criterion in trials studying agents not known to be teratogens. Instead, we assumed, along with the authors of most of the HIV RCTs, that two forms of adequate birth control methods could be verified for trial participants. Enrollment criteria for the 32 RCTs were summarized in the manner exemplified for ACTG trial 388 [40] and CPCRA trial 064 [53] in Table 1.
;)
Table 1: This table illustrates, for one AIDS Clinical Trial Group trial (ACTG 388) and one Community Programs for Clinical Research on AIDS trial (CPCRA 064), the list of ‘background' eligibility criteria listed in the original protocol for the trial and the resultant publication.
We applied the enrollment criteria listed in the 32 original RCT protocols and subsequent publications to 1717 WIHS participants for which data was available as of April 2003, using historical data when required. To allow a conservative estimate of excluded WIHS participants, if a data point relevant to a particular exclusion criterion was missing or not captured in the cohort database, the assumption was made that the participant would be included in the RCT. To compare the impact of the eligibility criteria listed in the trial publication to the enrollment criteria in the actual protocol, we calculated an ‘exclusion ratio’ for each trial. The exclusion ratio is defined as the number of WIHS women excluded from a trial based on publication criteria divided by the number excluded by protocol criteria. The average of these ratios was then determined for the ACTG trials, the CPCRA trials, and all trials combined.
Results
Description of enrollment criteria
The background eligibility criteria from the protocols and publications were categorized and listed as in Table 1, which illustrates how the enrollment criteria in protocols and publications were compared for one ACTG trial and one CPCRA trial. As appropriate, inclusion criteria were reversed in order to list all the eligibility criteria as exclusion criteria for our analyses. Clinical criteria listed in the original protocols included subjective assessments by the clinical investigator of the candidate's general health or his/her ability to comply with the study protocol. Examples of subjective criteria from various RCT protocols are listed in Table 2 and include statements granting investigators the authority to dismiss trial candidates for presumed non-compliance, medical contraindications or substance abuse. These subjective criteria were listed in the actual protocols for eight ACTG and 10 CPCRA trials, but not one of these criteria was described in the published reports from these trials.
Table 2: List of all the enrollment criteria in the 32 AIDS Clinical Trial Group (ACTG) and Community Programs for Clinical Research on AIDS (CPCRA) trial protocols that relied on the judgement of the individual investigator.
First aim: percentage of WIHS excluded from HIV treatment RCTs
Based on the protocols of the 20 ACTG trials, 28.4 to 67.6% (median 50.6%) of the WIHS cohort would have been excluded from participation per the eligibility criteria (Fig. 1): 20.7 to 62.3% (median 39.5%) excluded based on clinical criteria; 2.0 to 14.9% (median 9.0%) based on laboratory criteria; 0 to 24.8% (median 7.3%) based on medication criteria; and 0 to 2.6% (median 2.6%) based on pregnancy or lactation status. Based on the publications of these ACTG trials, 0 to 62% (median 21.2%) of WIHS would have been excluded from participation based on the listed eligibility criteria: 0 to 55% (median 16.8%) based on clinical criteria; 0 to 14.9% (median 3.7%) based on laboratory criteria; 0 to 20.2% (median 0%) based on medication criteria; and 0 to 2.6% (median 2.6%) based on pregnancy/lactation.
Fig. 1: Percentages of women's Interagency HIV Study (WIHS) participants excluded from AIDS Clinical Trial Group (ACTG) trials. Shaded columns show the percentages of WIHS participants excluded based on background enrollment criteria in protocols (ascending order). Solid portions of the bars indicate the proportion of patients excluded based only on publication criteria. The bar below the x-axis shows the exclusion ratio for each trial. *Trials where the protocols included subjective eligibility criteria for which numbers of excluded participants could not be quantified.
The enrollment criteria for CPCRA protocols were generally less stringent and excluded a lower percentage of WIHS participants than the ACTG protocols. Based on the eligibility criteria listed in the 12 CPCRA trial protocols, 0 to 41.8% (median 16.1%) of the WIHS cohort would have been excluded from participation (Fig. 2): 0 to 36.7% (median 12.9%) based on clinical criteria; 0 to 9.4% (median 4.1%) based on laboratory criteria; 0% based on medication criteria; and 0 to 2.6% (median 2.6%) based on pregnancy/lactation. Based on the criteria listed in the CPCRA publications, 0 to 38.9% (median 13.7%) of the WIHS cohort would have been excluded from participating: 0 to 34.3% (median 10.1%) based on clinical criteria; 0 to 9.4% based on laboratory criteria (median 1.5%); none for medication criteria; and 0 to 2.6% (median 2.6%) based on pregnancy/lactation.
Fig. 2: Percentages of women's Interagency HIV Study (WIHS) participants excluded from Community Programs for Clinical Research on AIDS (CPCRA) trial. Shaded columns show the percentages of WIHS participants excluded based on background enrollment criteria in protocols (ascending order). Solid portions of the shaded bars indicate the proportion of patients excluded based only on publication criteria. The bar below the x-axis shows the exclusion ratio for each trial. *Trials where the protocols included subjective eligibility criteria for which numbers of excluded participants could not be quantified.
Table 3 illustrates the effects on cohort exclusion for specific clinical, laboratory and medication eligibility criteria frequently employed for HIV trial enrollment. For instance, excluding patients with an ongoing or past history of grade 2 or above peripheral neuropathy (41% of trials) would eliminate 25.7% of WIHS participants and excluding patients with elevated total bilirubin (47% of trials) would eliminate 5.6%. Stratification of the cohort into self-identified race/ethnicities (African-American, Latina, Caucasian and other) revealed no significant race-based differences in the percentage excluded (data not shown).
Table 3: Percentage of Women's Interagency HIV Study (WIHS) women (n = 1717) excluded from HIV treatment trials due to specific criteria.
Second aim: calculating gap in eligibility criteria reporting between protocols and publications
Figure 1 illustrates the gaps in eligibility criteria reporting between protocols versus publications for each ACTG trial. For the ACTG trials, the mean of the exclusion ratios (publication/protocol) was 0.51, indicating that, on average, only half of the women actually excluded by protocol criteria seemed to be excluded based on publication criteria. Based on exclusion ratios, only four (243, 306, 347 and 343) of the 20 ACTG trials listed the full set of enrollment criteria in their subsequent publications. Half of the ACTG trials listed fewer than 50% of the eligibility criteria for the RCT in their publications; 30% (six of 20) disclosed less than a quarter of the actual enrollment criteria in their publications. Of note, eight out of 20 of the ACTG protocols contained subjective, investigator-dependent eligibility criteria (not revealed in publications), for which we could not quantify the percentage of WIHS with potential for exclusion.
Figure 2 shows the same data for the CPCRA trials. A greater proportion of CPCRA trials (seven of 12) listed the full set of evaluable enrollment criteria in the publications, although 10 out of 12 of these trials contained subjective exclusion criteria in protocols that were not disclosed in publications. For the CPCRA trials, the mean of the exclusion ratios (publication/protocol) was 0.75, which means that, on average, one-quarter of the women excluded by actual protocol criteria would have been deemed eligible by publication criteria.
For all the treatment trials combined, the average of the exclusion ratios was 0.60. Therefore, the number of women in the WIHS cohort ineligible for trial participation per publication criteria averaged only 60% of those actually excluded based on the protocols.
Discussion
Our report examines the background eligibility criteria of 32 seminal HIV treatment RCTs in the ACTG and CPCRA networks to assess disclosure rates of enrollment criteria reporting. Our first major aim, to examine the impact of clinical trial eligibility criteria on the generalizability of trial findings, has been studied for other disease states. Our second major aim was unique in that this study is the first systematic evaluation of gaps in reporting between clinical trial protocols and publications for any clinical condition. To assess the impact of incomplete reporting in major HIV RCTS, we analyzed the percentage of a representative cohort of HIV-infected women who would have been excluded from trial participation based on criteria listed in protocols and publications. We determined that a range of 0 to 67.6% (median 42%) of participants in the WIHS observational cohort would have been excluded from participating in these influential HIV RCTs per the original protocols. However significant disparities in eligibility criteria reporting between protocols and publications were found. A reader referring to methods in HIV trial publications would have underestimated the percentage of WIHS participants actually excluded from these trials (based on protocol criteria) by 40%. Furthermore, 83% of CPCRA trials and 40% of the ACTG trials listed subjective, investigator-dependent, eligibility criteria in the original protocols (Table 2) that were not mentioned in the publications.
Impact of the percentage of WIHS excluded from HIV treatment RCTs (first aim)
Although our work is one of largest such studies in the HIV treatment literature, concerns regarding the generalizability of RCT findings due to restrictive eligibility criteria have been raised for multiple other chronic medical conditions [13,20,58–60], including cancer treatment [14], lipid-lowering therapy [15], cardiovascular trials [16], and depression [11]. One group recently examined the enrollment criteria of the 20 controlled trials that determine the current guidelines for the management of hypertension [10]. When the eligibility criteria for these trials were applied to the general US hypertensive population, it was found that 42.5% of the general population would have been excluded from participation in these trials. These analyses have prompted re-evaluation of the necessity [61,62], and even the ethical implications [63], of over-rigorous enrollment criteria for RCTs.
As patients are excluded from a clinical trial setting, a progressively smaller subset of the reference population is captured, compromising trial generalizability. Our analysis did not examine the impact of HIV RCT eligibility criteria on the exclusion of HIV-infected men, so we cannot comment on whether enrollment criteria in these HIV clinical trials would have differentially excluded women. Lower participation of women in HIV RCTs has been recognized as a persistent problem in the HIV literature, however; a meta-analysis of 49 RCTs of antiretroviral efficacy in HIV-infected adults performed between 1990 and 2000 found that the proportional mean of women in these trials was merely 12.25%, with just two of the trials able to support any analysis of results by sex [64]. Given that our analysis establishes that over half (median 50.6%) of the largest cohort of HIV-infected women in the US would have been excluded from participating in 20 key ACTG trials based on protocol enrollment criteria, re-examination of these criteria may lead to increased participation of women in HIV RCTs.
The percentage of WIHS participants excluded from the HIV RCTs examined was not secondary to the risk of pregnancy, an oft-cited reason for differential exclusion of women from major treatment trials. Except for studies of known teratogens [29], these HIV RCTS did not exclude women based on the risk of pregnancy, but rather stressed the requirement for adequate contraception during participation of the trial. We therefore chose not to exclude any women of childbearing potential in the WIHS cohort from the HIV RCTs and found that only 2.6% of WIHS women were either pregnant or lactating at the time of this analysis. Therefore, pregnancy or the risk of pregnancy could not account for the high percentages of WIHS women excluded from ACTG clinical trials based on criteria in the original protocols.
Impact of incomplete eligibility criteria reporting (second aim)
Direct comparison of the effects of applying published versus protocol enrollment criteria for HIV RCTs demonstrated significant differences in the percentage of WIHS participants eligible for trial participation. Incomplete reporting of eligibility criteria may reflect restrictions imposed by the journal format, such as word-count limitations, for the publication of RCT findings. We examined the HIV trial descriptions posted on the NIH clinical trials website (www.clinicaltrials.gov) and found the listing of eligibility criteria to be generally more complete than in publications, but less complete than in protocols (data not shown). Therefore, no publicly available listing of complete selection criteria for HIV RCTs exists. Since most of the HIV RCT publications understate the extent of selection criteria, the reader may believe that study results are more widely generalizable than they actually are. The Consolidated Standards of Reporting Trials (CONSORT) statement in 1996 [17] identifies key information about entry criteria (including inclusion and exclusion criteria), randomization methods, patient follow-up, and outcome determination that should be reported for every RCT publication. Our analysis demonstrates that current reporting of eligibility criteria, at least for these HIV clinical trials, may not be meeting these standards.
Significantly, none of the reports that examine eligibility criteria in RCTs for other disease states [10,11,13–16,20,58–60] assess the disparity between enrollment criteria listed in protocols versus publications. One study did examine the reporting of eligibility criteria in trials that formed the basis of eight ‘Clinical Alerts’ issued by the NIH between January 1988 and September 1994 (including two HIV treatment trials) and found that only 63% of the protocol eligibility criteria were reported in the accompanying journal articles [65], similar to our proportion of 60%. However, this report did not examine the impact such gaps in disclosure would mean for eligibility of a representative population to participate in the clinical trials, nor did it examine the systematic gap in reporting for a single disease state.
Limitations of our analysis
One limitation of our analysis is that only protocols for NIH-funded HIV RCTs for which publications were available (up to April 2004) could be examined. Newer protocols may have adopted less restrictive exclusion criteria, and full disclosure of protocol eligibility in the future will allow assessment of any such changes. This paper comments on the impact of exclusion criteria on the possible participation of a representative cohort of US women. We are undertaking a comparison between the percentage of participants excluded from WIHS and the male Multicenter AIDS Cohort Study (MACS) to determine if exclusion criteria may eliminate participants differentially by sex. Finally, the self-reported nature of active diseases or symptoms in the WIHS cohort, such as active opportunistic or other infections, or the presence of peripheral neuropathy, may have over-estimated their prevalence, although self-report presumably plays a large role in clinical trial enrollment as well.
Suggestions for improvement
One possible modification to HIV trial enrollment procedures would be to minimize broad and/or arbitrary eligibility criteria. A broad definition of one exclusion criterion alone had a pronounced impact on the potential generalizability of RCT results: mild neuropathy (greater than grade I neuropathy) excluded more than one-quarter of women in WIHS in 41% of the HIV RCTs (Table 3). A narrowing of this exclusion criterion to grade III neuropathy would have eliminated only 8.3% of the cohort, resulting in an RCT participant pool that would have been more representative of WIHS subjects. Likewise, arbitrary cutoffs for laboratory criteria should be examined carefully (Table 3). The requirement of an absolute neutrophil count (ANC) of > 1000 × 106 cells/l for participation in 11 of 32 trials would exclude 6.2% of WIHS women, whereas the use of a more clinically-meaningful cutoff of > 500 × 106 cells/l would exclude < 1%. Since mild neutropenia is common among African Americans [66], careful reconsideration of this exclusion criterion may result in trial diversification.
Another revision to trial enrollment procedures would be the minimization of the broadest selection criteria, namely those employing ‘investigator judgment’, to determine eligibility. A survey of cancer investigators examined enrollment decisions based on subjective eligibility criteria, such as estimation of substance abuse, and determined that these criteria predictably lead to more variable enrollment decisions than objective criteria [67]. Investigator judgement as to whether a patient can adhere to the study protocol based on social or psychological factors may introduce social biases into HIV RCTs. Provider assessment of adherence to antiretroviral therapy, for example, can be inaccurate [68], especially when assumptions about patient compliance in light of certain social conditions, such as homelessness or poverty, are made [69]. Social challenges, such as lack of insurance, substance abuse, depression, trading sex for money or drugs, housing, debt or child care issues, and violence, are proportionally higher in HIV-infected women than men [70], which may result in selective exclusion of women from RCTs based on these subjective criteria.
One of the objectives of the NIAID in funding the CPCRA was to recruit previously under-represented HIV-infected patients into the clinical trial setting [71]. Since minorities are over-represented among HIV-infected women in comparison with men [72], greater recruitment of women may simultaneously improve racial disparities in RCTs. Although objective exclusion criteria do not exclude WIHS participants in CPCRA trials (Fig. 2) to the same extent as ACTG trials (Fig. 1), the CPCRA trial protocols include more subjective criteria (Table 2), for which it is fair to conjecture that the exclusions may be sizable. It is likely that the elimination of investigator discretionary criteria would result in more diverse study population for RCTs. In the setting of HIV research, where women and ethnic minorities are increasingly represented in the afflicted population, the call for diversity is especially pressing.
Finally, full disclosure of the enrollment criteria of RCTs for HIV and other disease states should be mandated in publications. Space limitations to publishing the full list of eligibility criteria can be managed through weblinks from the electronic publication (and URL listings in the paper article) to the original RCT protocols.
Conclusion
We found that significant under-reporting of the full list of HIV trial enrollment criteria in publications may have a significant impact on the generalizability of a trial's findings to a representative HIV-infected population. Furthermore, structural barriers to participation in HIV clinical trials through restrictive eligibility criteria may lead to under-representation of certain HIV-infected groups. A critical appraisal of the necessity and effects of each enrollment criterion should be performed. Subjective, investigator-dependent, eligibility criteria should be minimized. In the few circumstances when it is not feasible to eliminate these or otherwise broaden eligibility, and representative participants cannot be enrolled in RCTs, alternative research methods, such as observational cohort studies, may be required to supplement knowledge on treatment effectiveness in the ‘real world' setting. Standardization and openness of clinical trial reporting will allow clinicians to more fully assess the applicability of a trial finding to an individual patient and allow researchers and policymakers to expand scientific investigation for all representative populations with HIV.
Acknowledgements
We wish to thank staff of NIAID, CPCRA, (especially David Cohn MD and Eric Daniels) and ACTG for their notable co-operation with this study; and Virginia Ernster PhD (Department of Epidemiology, University of California, San Francisco) for her helpful comments on this paper.
Sponsorship: Data in this manuscript were collected by the Women's Interagency HIV Study (WIHS) Collaborative Study Group. The WIHS is funded by the National Institute of Allergy and Infectious Diseases with supplemental funding from the National Cancer Institute, the National Institute on Drug Abuse (UO1-AI-35004, UO1-AI-31834, UO1-AI-34994, UO1-AI-34989, UO1-AI-34993, and UO1-AI-42590). Funding was also provided by the National Institute of Child Health and Human Development (UO1-CH-32632) and the National Center for Research Resources (MO1-RR-00071, MO1-RR-00079, MO1-RR-00083). M.G. was supported in part for preparation of the manuscript by a Mentored Clinical Scientist Development Program Award (K12 AR47659) from the National Institutes of Health.
References
1. Lucas GM, Chaisson RE, Moore RD. Survival in an urban
HIV-1 clinic in the era of highly active antiretroviral therapy: a 5-year cohort study. J Acquir Immune Defic Syndr 2003; 33:321–328.
2. Anon.
Analysis of HIV-1 clinical trials: statistical magic? The AVANTI Steering Committee.
Lancet 1999;
353:2061–2064.
3. Deeks S, Hecht F, Swanson M, Elbeik T, Loftus R, Cohen P,
et al.
HIV RNA and CD4 cell count response to protease inhibitor therapy in an urban AIDS clinic: response to both initial and salvage therapy. AIDS 1999; 13:F35–F43.
4. Sulkowski MS. Defining the standard of care: randomized controlled trials for the treatment of hepatitis C in the
HIV-infected person. Hepatology 2004; 39:906–908.
5. el-Sadr W, Capps L. The challenge of minority recruitment in
clinical trials for AIDS. JAMA 1992; 267:954–957.
6. Stone VE, Mauch MY, Steger K, Janas SF, Craven DE. Race, gender, drug use, and participation in AIDS
clinical trials. Lessons from a municipal hospital cohort. J Gen Intern Med 1997; 12:150–157.
7. Gifford AL, Cunningham WE, Heslin KC, Andersen RM, Nakazono T, Lieu DK,
et al. Participation in research and access to experimental treatments by
HIV-infected patients. N Engl J Med 2002; 346:1373–1382.
8. Diaz T, Chu SY, Sorvillo F, Mokotoff E, Davidson AJ, Samuel MC,
et al. Differences in participation in experimental drug trials among persons with AIDS. J Acquir Immune Defic Syndr Hum Retrovirol 1995; 10:562–568.
9. Hankins C, Lapointe N, Walmsley S. Participation in
clinical trials among women living with
HIV in Canada. Canadian Women's
HIV Study Group. CMAJ 1998; 159:1359–1365.
10. Pedone C, Lapane KL.
Generalizability of guidelines and physicians’ adherence. Case study on the Sixth Joint National Commitee's guidelines on hypertension. BMC Public Health 2003; 3:24.
11. Zimmerman M, Chelminski I, Posternak MA. Exclusion criteria used in antidepressant efficacy trials: consistency across studies and representativeness of samples included. J Nerv Ment Dis 2004; 192:87–94.
12. Heiat A, Gross CP, Krumholz HM. Representation of the elderly, women, and minorities in heart failure
clinical trials. Arch Intern Med 2002; 162:1682–1688.
13. Sweeney KG, Gray DP, Steele R, Evans P. Use of warfarin in non-rheumatic atrial fibrillation: a commentary from general practice. Br J Gen Pract 1995; 45:153–158.
14. Fossa SD, Skovlund E. Selection of patients may limit the
generalizability of results from cancer trials. Acta Oncol 2002; 41:131–137.
15. Lloyd-Jones DM, O'Donnell CJ, D'Agostino RB, Massaro J, Silbershatz H, Wilson PW. Applicability of cholesterol-lowering primary prevention trials to a general population: the Framingham heart study. Arch Intern Med 2001; 161:949–954.
16. Sharpe N.
Clinical trials and the real world: selection bias and generalisability of trial results. Cardiovasc Drugs Ther 2002; 16:75–77.
17. Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I,
et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA 1996; 276:637–639.
18. Huwiler-Muntener K, Juni P, Junker C, Egger M. Quality of reporting of randomized trials as a measure of methodologic quality. JAMA 2002; 287:2801–2804.
19. Gross CP, Mallory R, Heiat A, Krumholz HM. Reporting the recruitment process in
clinical trials: who are these patients and how did they get there? Ann Intern Med 2002; 137:10–16.
20. Britton A, McKee M, Black N, McPherson K, Sanderson C, Bain C. Choosing between randomised and non-randomised studies: a systematic review, Chapter 4 and Appendix 4. Health Technol Assess 1998; 2(i–iv):1–124.
21. Horwitz RI, Viscoli CM, Clemens JD, Sadock RT. Developing improved observational methods for evaluating therapeutic effectiveness. Am J Med 1990; 89:630–638.
22. Klein R, Moss S. A comparison of the study populations in the Diabetes Control and Complications Trial and the Wisconsin Epidemiologic Study of Diabetic Retinopathy. Arch Intern Med 1995; 155:745–754.
23. Moore D, Goodall R, Ives N, Hooker N, Gazzard B, Easterbrook P. How generalizable are the results of large randomized controlled trials of antiretroviral therapy?
HIV Med 2000; 1:149–154.
24. van der Horst CM, Saag MS, Cloud GA, Hamill RJ, Graybill JR, Sobel JD,
et al. Treatment of cryptococcal meningitis associated with the acquired immunodeficiency syndrome. National Institute of Allergy and Infectious Diseases Mycoses Study Group and AIDS
Clinical Trials Group. N Engl J Med 1997; 337:15–21 [ACTG protocol 159 (1992): A randomized double blind protocol comparing amphotericin b with flucytosine to amphotericin b alone followed by a comparison of fluconazole and itracaonzole in the treatment of acute cryptococcal meningitis.].
25. Krown SE, Li P, Von Roenn JH, Paredes J, Huang J, Testa MA. Efficacy of low-dose interferon with antiretroviral therapy in Kaposi's sarcoma: a randomized phase II AIDS
clinical trials group study. J Interferon Cytokine Res 2002; 22:295–303 [ACTG protocol 206 (1992): A randomized Phase II trial to determine the safety, tolerance and efficacy of two doses of interferon alpha combined with didanosine in patients with Kaposi's sarcoma.].
26. Benson CA, Williams PL, Currier JS, Holland F, Mahon LF, MacGregor RR,
et al. A prospective, randomized trial examining the efficacy and safety of clarithromycin in combination with ethambutol, rifabutin, or both for the treatment of disseminated Mycobacterium avium complex disease in persons with acquired immunodeficiency syndrome. Clin Infect Dis 2003; 37:1234–1243 [ACTG protocol 223 (1994): A Phase II/III prospective, multicenter, randomized, controlled trial comparing the safety and efficacy of three clarithromycin-containing combination drug regimens for the treatment of disseminated MAC disease in persons with AIDS.].
27. Chirgwin K, Hafner R, Leport C, Remington J, Andersen J, Bosler EM,
et al. Randomized phase II trial of atovaquone with pyrimethamine or sulfadiazine for treatment of toxoplasmic encephalitis in patients with acquired immunodeficiency syndrome: ACTG 237/ANRS 039 Study. AIDS
Clinical Trials Group 237/Agence Nationale de Recherche sur le SIDA, Essai 039. Clin Infect Dis 2002; 34:1243–1250 [ACTG protocol 237 (1994): A Phase II randomized open-label trial of atovaquone plus pyrimethamine and atovaquone plus sulfadiazine for the treatment of acute toxoplasmic encephalitis.].
28. Hall CD, Dafni U, Simpson D, Clifford D, Wetherill PE, Cohen B,
et al. Failure of cytarabine in progressive multifocal leukoencephalopathy associated with human immunodeficiency virus infection. AIDS
Clinical Trials Group 243 Team. N Engl J Med 1998; 338:1345–1351 [ACTG protocol 243 (1994): A Phase II multicenter study comparing antiretroviral therapy alone to antiretroviral therapy plus cytosine arabinoside for the treatment of progressive multifocal leukoencephalopathy (PML) in human immunodeficiency virus infected subjects.].
29. Jacobson JM, Greenspan JS, Spritzler J, Ketter N, Fahey JL, Jackson JB,
et al. Thalidomide for the treatment of oral aphthous ulcers in patients with human immunodeficiency virus infection. National Institute of Allergy and Infectious Diseases AIDS
Clinical Trials Group. N Engl J Med 1997; 336:1487–1493 [ACTG protocol 251 (1994): Thalidomide for treatment of oral and esophageal aphthous ulcers and
HIV viremia in patients with
HIV infection.].
30. Friedland GH, Pollard R, Griffith B, Hughes M, Morse G, Bassett R,
et al. Efficacy and safety of delavirdine mesylate with zidovudine and didanosine compared with two-drug combinations of these agents in persons with
HIV disease with CD4 counts of 100 to 500 cells/mm3 (ACTG 261). ACTG 261 Team. J Acquir Immune Defic Syndr 1999; 21:281–292 [ACTG protocol 261 (1994): A Phase II double-blind study of delavirdine (DLV) in combination with zidovudine (ZDV) and/or didanosine (ddI) versus ZDV and ddI combination therapy.].
31. Kuritzkes DR, Marschner I, Johnson VA, Bassett R, Eron JJ, Fischl MA,
et al. Lamivudine in combination with zidovudine, stavudine, or didanosine in patients with
HIV-1 infection. A randomized, double-blind, placebo-controlled trial. National Institute of Allergy and Infectious Disease AIDS
Clinical Trials Group Protocol 306 Investigators. AIDS 1999; 13:685–694 [ACTG protocol 306 (1995): A Phase II, randomized study of the antiviral activity and resistance interactions of 3TC + ZDV, D4T, or DDI vs. monotherapy DDI or D4T in
HIV-infected individuals with 200–600 cells/mm
3 CD4 cells and no previous nucleoside experience.].
32. Lederman MM, Connick E, Landay A, Kuritzkes DR, Spritzler J, St Clair M,
et al. Immunologic responses associated with 12 weeks of combination antiretroviral therapy consisting of zidovudine, lamivudine, and ritonavir: results of AIDS
Clinical Trials Group Protocol 315. J Infect Dis 1998; 178:70–79 [ACTG protocol 315 (1996): A pilot study to evaluate the immunologic consequences of highly active antiretroviral therapy (HAART) consisting of ritonavir (ABT-538), zidovudine (ZDV), and lamivudine (3TC) in moderately advanced,
HIV-1 disease.].
33. Hammer SM, Squires KE, Hughes MD, Grimes JM, Demeter LM, Currier JS,
et al. A controlled trial of two nucleoside analogues plus indinavir in persons with human immunodeficiency virus infection and CD4 cell counts of 200 per cubic millimeter or less. AIDS
Clinical Trials Group 320 Study Team. N Engl J Med 1997; 337:725–733 [ACTG protocol 320 (1996): A randomized, double-blind, Phase III study of indinavir sulfate (IDV) with open-label ZDV or stavudine (d4T) and lamivudine (3TC) in subjects with
HIV infection with CD4 CELL Counts ≤ 200 cells/mm
3 and ≥ 3 months of prior zidovudine.].
34. Havlir DV, Marschner IC, Hirsch MS, Collier AC, Tebas P, Bassett RL,
et al. Maintenance antiretroviral therapies in
HIV infected patients with undetectable plasma
HIV RNA after triple-drug therapy. AIDS
Clinical Trials Group Study 343 Team. N Engl J Med 1998; 339:1261–1268 [ACTG protocol 343 (1996): A prospective randomized double blind trial of three maintenance regimens for
HIV infected patients receiving induction therapy with zidovudine, lamivudine, and indinavir.].
35. Murphy RL, Gulick RM, DeGruttola V, D'Aquila RT, Eron JJ, Sommadossi JP,
et al. Treatment with amprenavir alone or amprenavir with zidovudine and lamivudine in adults with human immunodeficiency virus infection. AIDS
Clinical Trials Group 347 Study Team. J Infect Dis 1999; 179:808–816 [ACTG protocol 347 (1997): A randomized, double-blind phase 2 study of VX-478 (amprenavir) monotherapy vs VX-478 plus ZDV plus 3TC in
HIV infected individuals.].
36. Gulick RM, Hu XJ, Fiscus SA, Fletcher CV, Haubrich R, Cheng H,
et al. Randomized study of saquinavir with ritonavir or nelfinavir together with delavirdine, adefovir, or both in human immunodeficiency virus-infected adults with virologic failure on indinavir: AIDS
Clinical Trials Group Study 359. J Infect Dis 2000; 182:1375–1384 [ACTG protocol 359 (1997): Activity of the soft gelatin capsule of saquinavir (SQV) in combination with ritonavir (RTV) or nelfinavir and combinations of delavirdine (DLV), and/or adefovir dipivoxil in
HIV infected subjects with prior indinavir (IDV) use and viral loads of >/= 2000 and </= 200 000 copies
HIV RNA/ml.].
37. Albrecht MA, Bosch RJ, Hammer SM, Liou SH, Kessler H, Para MF,
et al. Nelfinavir, efavirenz, or both after the failure of nucleoside treatment of
HIV infection. N Engl J Med 2001; 345:398–407 [ACTG protocol 364 (1997): Comparison of the virologic efficacy of nelfinavir and/or DMP-266 (efavirenz) in combination with one or two nucleoside analogs in nucleoside experienced subjects: a rollover study of ACTG 302/303.].
38. Kuritzkes DR, Bassett RL, Johnson VA, Marschner IC, Eron JJ, Sommadossi JP,
et al. Continued lamivudine versus delavirdine in combination with indinavir and zidovudine or stavudine in lamivudine-experienced patients: results of Adult AIDS
Clinical Trials Group protocol 370. AIDS 2000; 14:1553–1561 [ACTG protocol 370 (1997): Virologic and immunologic activity of continued lamivudine (3TC) vs delavirdine (DLV) in combination with indinavir and zidovudine (ZDV) or stavudine (d4T) in 3TC-experienced patients.].
39. Gulick RM, Smeaton LM, D'Aquila RT, Eron JJ, Currier JS, Gerber JG,
et al. Indinavir, nevirapine, stavudine, and lamivudine for human immunodeficiency virus-infected, amprenavir-experienced subjects: AIDS
Clinical Trials Group protocol 373. J Infect Dis 2001; 183:715–721 [ACTG protocol 373 (1997): A Phase II study of 1) amprenavir plus 3TC plus ZDV (or d4T); 2) indinavir plus nevirapine plus 3TC plus stavudine or 3) other treatment regimens in subjects previously treated with amprenavir.].
40. Fischl MA, Ribaudo HJ, Collier AC, Erice A, Giuliano M, Dehlinger M,
et al. A randomized trial of 2 different 4-drug antiretroviral regimens versus a 3-drug regimen, in advanced human immunodeficiency virus disease. J Infect Dis 2003; 188:625–634 [ACTG protocol 388 (1998): A Phase III randomized, controlled trial of EFV or NFV in combination with 3TC/ZDV and IDV in
HIV infected subjects with CD4 ≤ 200 or vl ≥ 80 000.].
41. Hammer S, Vaida F, Bennett K, Holohan M, Sheiner L, Eron J,
et al. Dual vs single protease inhibitor therapy following antiretroviral treatment failure: a randomized trial. JAMA 2002; 288:169–180 [ACTG protocol 398 (1998): A Phase II, randomized trial of amprenavir as part of dual protease inhibitor regimens (placebo-controlled), in combination with abacavir, efavirenz and adefovir dipivoxil in
HIV-infected subjects with prior exposure to approved protease inhibitors and loss of virologic suppression as reflected by a plasma
HIV-1 RNA concentration ≥ 1,000 copies/ml.].
42. Robbins GK, De Gruttola V, Shafer RW, Smeaton LM, Snyder SW, Pettinelli C,
et al. Comparison of sequential three-drug regimens as initial therapy for
HIV-1 infection. N Engl J Med 2003; 349:2293–2303 [ACTG protocol 384 (1998): Study of protease inhibitor and/or non-nucleoside reverse transcriptase inhibitor with dual nucleosides in initial therapy of
HIV infection.].
43. Gulick RM, Ribaudo HJ, Shikuma CM, Lustgarten S, Squires KE, Meyer WA 3rd,
et al. Triple-nucleoside regimens versus efavirenz-containing regimens for the initial treatment of
HIV-1 infection. N Engl J Med 2004; 350:1850–1861 [ACTG protocol 5095 (2001): Phase III, randomized, double-blind comparison of three protease inhibitor-sparing regimens for the initial treatment of
HIV infection.].
44. Abrams DI, Goldman AI, Launer C, Korvick JA, Neaton JD, Crane LR,
et al. A comparative trial of didanosine or zalcitabine after treatment with zidovudine in patients with human immunodeficiency virus infection. The Terry Beirn Community Programs for Clinical Research on AIDS. N Engl J Med 1994; 330:657–662 [CPCRA protocol 002 (1991): Randomized trial of ddI and ddC in
HIV-infected patients who were intolerant or failed ZDV therapy.].
45. Gordin F, Chaisson RE, Matts JP, Korvick JA, Neaton JD, Crane LR,
et al. Rifampin and pyrazinamide vs isoniazid for prevention of tuberculosis in
HIV-infected persons: an international randomized trial. Terry Beirn Community Programs for Clinical Research on AIDS, the Adult AIDS
Clinical Trials Group, the Pan American Health Organization, and the Centers for Disease Control and Prevention Study Group. JAMA 2000; 283:1445–1450 [CPCRA protocol 004 (1994): International, multicenter randomized open-label two-arm (isoniazid versus rifampin/pyrazinamide) comparative trial.].
46. Gordin FM, Matts JP, Miller C, Brown LS, Hafner R, John SL,
et al. A controlled trial of isoniazid in persons with anergy and human immunodeficiency virus infection who are at high risk for tuberculosis. Terry Beirn Community Programs for Clinical Research on AIDS. N Engl J Med 1997; 337:315–320 [CPCRA protocol 005 (1994): Prophylaxis against tuberculosis in patients with
HIV infection and anergy, but with suspected tuberculous infection.].
47. El-Sadr WM, Luskin-Hawk R, Yurik TM, Walker J, Abrams D, John SL,
et al. A randomized trial of daily and thrice-weekly trimethoprim-sulfamethoxazole for the prevention of
Pneumocystis carinii pneumonia in human immunodeficiency virus-infected persons. Terry Beirn Community Programs for Clinical Research on AIDS (CPCRA). Clin Infect Dis 1999; 29:775–783 [CPCRA protocol 006 (1994): Compare the safety and efficacy of two dosage regimens (daily and thrice weekly) of trimethoprim-sulfamethoxazole (TMS) in the prevention of PCP in high-risk
HIV-infected patients.).
48. Saravolatz LD, Winslow DL, Collins G, Hodges JS, Pettinelli C, Stein DS,
et al. Zidovudine alone or in combination with didanosine or zalcitabine in
HIV-infected patients with the acquired immunodeficiency syndrome or fewer than 200 CD4 cells per cubic millimeter. Investigators for the Terry Beirn Community Programs for Clinical Research on AIDS. N Engl J Med 1996; 335:1099–1106 [CPCRA protocol 007 (1994): To compare ZDV alone with ZDV given as combination therapy with either ddI or ddC with respect to efficacy and safety in
HIV-infected individuals with CD4 < 200 cells/mm
3.].
49. Benson CA, Williams PL, Cohn DL, Becker S, Hojczyk P, Nevin T,
et al. Clarithromycin or rifabutin alone or in combination for primary prophylaxis of Mycobacterium avium complex disease in patients with AIDS: A randomized, double-blind, placebo-controlled trial. The AIDS
Clinical Trials Group 196/Terry Beirn Community Programs for Clinical Research on AIDS 009 Protocol Team. J Infect Dis 2000; 181:1289–1297 [CPCRA protocol 009 (1994): A prospective randomized comparative study of the safety and efficacy of clarithromycin versus rifabutin vs the combination of clarithromycin plus rifabutin for the prevention of MAC in
HIV-infected patients with CD4 < 100/mm
3.].
50. Shlay JC, Chaloner K, Max MB, Flaws B, Reichelderfer P, Wentworth D,
et al. Acupuncture and amitriptyline for pain due to
HIV-related peripheral neuropathy: a randomized controlled trial. Terry Beirn Community Programs for Clinical Research on AIDS. JAMA 1998; 280:1590–1595 [CPCRA protocol 022 (1994): To evaluate the separate and combined efficacy of a standard acupuncture regimen and amitriptyline on the relief of pain due to peripheral neuropathy and on the quality of life of
HIV-infected patients.].
51. Brosgart CL, Louis TA, Hillman DW, Craig CP, Alston B, Fisher E,
et al. A randomized, placebo-controlled trial of the safety and efficacy of oral ganciclovir for prophylaxis of cytomegalovirus disease in
HIV-infected individuals. Terry Beirn Community Programs for Clinical Research on AIDS. AIDS 1998; 12:269–277 [CPCRA protocol 023 (1995): To evaluate the safety and efficacy of oral ganciclovir for prophylaxis against CMV retinal and gastrointestinal mucosal disease in
HIV-infected patients in severe immunosuppression.].
52. El-Sadr WM, Burman WJ, Grant LB, Matts JP, Hafner R, Crane L,
et al. Discontinuation of prophylaxis for
Mycobacterium avium complex disease in
HIV-infected patients who have a response to antiretroviral therapy. Terry Beirn Community Programs for Clinical Research on AIDS. N Engl J Med 2000; 342:1085–1092 [CPCRA protocol 048 (1999): A randomized, double-blind, placebo-controlled trial of prophylaxis for disseminated
Mycobacterium avium complex disease and bacterial pneumonia versus deferred prophylaxis in
HIV-infected patients who experience rebound in CD4+ cell count due to active antiretroviral therapy.].
53. Lawrence J, Mayers DL, Hullsiek KH, Collins G, Abrams DI, Reisler RB,
et al. Structured treatment interruption in patients with multidrug-resistant human immunodeficiency virus. N Engl J Med 2003; 349:837–846 [CPCRA protocol 064 (2000): A randomized study of a prescribed 4-month structured treatment interruption (STI) followed by initiation of a new antiretroviral regimen versus immediate initiation of a new antiretroviral regimen in
HIV-infected patients with multidrug resistance (MDR) virus.].
54. Baxter JD, Mayers DL, Wentworth DN, Neaton JD, Hoover ML, Winters MA,
et al. A randomized study of antiretroviral management based on plasma genotypic antiretroviral resistance testing in patients failing therapy. CPCRA 046 Study Team for the Terry Beirn Community Programs for Clinical Research on AIDS. AIDS 2000; 14:F83–F93 [CPCRA protocol 046 (1997): A pilot study of the short-term effects of antiretroviral management based on plasma genotypic antiretroviral resistance testing (GART) compared with antiretroviral management without plasma GART.].
55. Fisher EJ, Chaloner K, Cohn DL, Grant LB, Alston B, Brosgart CL,
et al. The safety and efficacy of adefovir dipivoxil in patients with advanced
HIV disease: a randomized, placebo-controlled trial. AIDS 2001; 15:1695–1700 [CPCRA protocol 039 (1997): A Phase III, randomized, double blind, placebo-controlled study of the safety and efficacy of adefovir dipivoxil (bis-POM PMEA) in prolonging survival in
HIV-infected individuals with a CD4+ cell count of < 100 cells/mm
3 or a lifetime nadir of < 50.].
56. Barkan SE, Melnick SL, Preston-Martin S, Weber K, Kalish LA, Miotti P,
et al. The Women's Interagency
HIV Study. Epidemiology 1998; 9:117–125.
57. Women's Interagency
HIV Study (WIHS) website:
https://statepiaps.jhsph.edu/wihs/
58. Mant D. Can randomised trials inform clinical decisions about individual patients? Lancet 1999; 353:743–746.
59. McKee M, Britton A, Black N, McPherson K, Sanderson C, Bain C. Methods in health services research. Interpreting the evidence: choosing between randomised and non-randomised studies. BMJ 1999; 319:312–315.
60. Britton A, McKee M, Black N, McPherson K, Sanderson C, Bain C. Threats to applicability of randomised trials: exclusions and selective participation. J Health Serv Res Policy 1999; 4:112–121.
61. Yusuf S, Held P, Teo KK, Toretsky ER. Selection of patients for randomized controlled trials: implications of wide or narrow
eligibility criteria. Stat Med 1990; 9:73–83, discussion 83–86.
62. Chalmers TC. Ethical implications of rejecting patients for
clinical trials. JAMA 1990; 263:865.
63. Weijer C. Selecting subjects for participation in clinical research: one sphere of justice. J Med Ethics 1999; 25:31–36.
64. Pardo M, Ruiz M, Gimeno A, Navarro L, Garcia A, Tarazona M,
et al.
Gender bias in clinical trials of AIDS drugs.
XIV International AIDS Conference. Barcelona, July, 2002 [abstract WePeB5964].
65. Shapiro SH, Weijer C, Freedman B. Reporting the study populations of
clinical trials. Clear transmission or static on the line? J Clin Epidemiol 2000; 53:973–979.
66. Reed WW, Diehl LF. Leukopenia, neutropenia, and reduced hemoglobin levels in healthy American blacks. Arch Intern Med 1991; 151:501–505.
67. Weijer C, Freedman B, Shapiro S, Fuks A, Skrutkowska M, Sigurjonsdottir M. Assessing the interpretation of criteria for clinical trial eligibility: a survey of oncology investigators. Clin Invest Med 1998; 21:17–26.
68. Bangsberg D, Hecht F, Clague H, Charlebois E, Ciccarone D, Chesney M,
et al. Provider assessment of adherence to
HIV antiretroviral therapy. J Acquir Immune Defic Syndr 2001; 26:435–442.
69. Bangsberg D, Hecht F, Charlebois E, Zolopa A, Holodiny M, Sheiner L,
et al. Adherence to protease inhibitors,
HIV-1 viral load, and development of drug resistance in an indigent population. AIDS 2000; 14:357–366.
70. Hader SL, Smith DK, Moore JS, Holmberg SD.
HIV infection in women in the United States: status at the Millennium. JAMA 2001; 285:1186–1192.
71. Fauci AS. AIDS–challenges to basic and clinical biomedical research. Acad Med 1989; 64:115–119.
72.
HIV/AIDS Surveillance Supplemental Report – Cases of HIV Infection and AIDS in the United States, by Race/Ethnicity, 1998–2002.
http://www.cdc.gov/hiv/stats/Cases_HIV-Infection_AIDS-by-Race_Ethnicity_1998-2002.pdf 2004
