Factors associated with the successful modification of antiretroviral therapy
Weidle, Paul J.a,b; Lichtenstein, Kenneth A.c; Moorman, Anne C.a; Bargen, Jennifer C. Vona; Greenberg, Kenneth S.c; Palella, Frank J. Jrd; Holmberg, Scott D.*; for the HIV Outpatient Study Investigators
From the aEpidemiology Branch, Division of HIV/AIDS Prevention Surveillance and Epidemiology, National Center for HIV, STD, and TB Prevention, and bEpidemic Intelligence Service, Epidemiology Program Office, Centers for Disease Control, Atlanta, GA, USA; cRose Medical Center, Denver, CO, USA; and dNorthwestern University Medical School, Chicago, IL, USA.
*The investigators participating in the HIV Outpatient Study are listed in the Appendix.
This work was presented at the Thirty-sixth Annual Meeting of the Infectious Disease Society of America, November, 1998, Denver, Colorado [abstract 445].
Sponsorship: The HIV Outpatient Study was supported by cooperative agreement UC64/CCU5096889-03 between the Centers for Disease Control and Prevention and the Health Research Network of APACHE Medical Systems, Inc., McLean, VA, USA.
Correspondence to: Paul J. Weidle, PharmD, MPH, Division of HIV/AIDS Prevention, National Center for HIV, STD, and TB Prevention, Centers for Disease Control and Prevention, 1600 Clifton Road, MS E-45, Atlanta, GA 30333, USA. Tel: +1 404 639 6155; fax: +1 404 639 6127; e-mail: email@example.com
Requests for reprints to: National Center for HIV, STD, and TB Prevention, Office of Communications, Centers for Disease Control and Prevention, Mail Stop E-06, Atlanta, GA 30333, USA.
Received: 15 October 1999; accepted: 23 December 1999.
Objectives: To assess the characteristics of medication regimen modification and the influence of a commercial genotypic resistance assay on the short-term (3–12 weeks) viral load response (≥ 0.5 log reduction) in HIV-1-infected patients extensively treated with antiretroviral therapy (ART).
Methods: A nested cohort study was performed in two clinics from the HIV Outpatient Study of 96 persons with a HIV-1 viral load of 104 log copies/ml or greater taking at least two antiretroviral medications.
Results: Successful modification was associated with adding at least two new medications [relative risk (RR), 1.5; 95% confidence interval (CI), 1.1–2.2], adding a drug from a previously unused class of agents (RR, 2.0; CI, 1.4–2.9), the initiation of a non-nucleoside reverse transcriptase inhibitor (NNRTI) (RR, 1.7; CI, 1.2–2.4), but not substituting a protease inhibitor or the use of a commercial genotypic resistance assay.
Conclusion: Incorporating a drug from a previously unused class or changing at least two new medications, but, within the confines of this study, not using a commercial genotypic resistance assay, was associated with the successful modification of ART as measured by a reduction in viral load.
Combination antiretroviral therapy (ART) has been highly effective in reducing morbidity and mortality in persons infected with HIV-1 , but many patients have received sequential monotherapy or suboptimal combination therapy for years before the advent of more potent therapies [2–4]. Consequently, viruses with differing degrees of resistance to current antiretroviral medications have been selected, compromising future benefit [5–9]. A conventional approach to choosing a combination of medications that will have activity against HIV in a person in whom current ART is failing has been empirically to modify ART, that is, to change most or all drugs to drugs that the patient has not taken [4,10–12]. Some evidence has been presented in persons with moderate antiretroviral experience that use of the genotypic resistance assay may be useful in directing changes in therapy when accompanied by expert advice  or other tightly circumscribed circumstances . The full utility of these tests in typical clinical situations, however, remains uncertain [12,15–20]. We examined the characteristics of the modification of ART and the influence of a genotypic resistance assay to direct modifications on the short-term viral-load response in HIV-1-infected patients who have been extensively treated with ART.
As described elsewhere, the HIV Outpatient Study (HOPS) is an ongoing study that collects summaries of physician–patient interactions and data on the course of disease for more than 4500 HIV-infected, non-hospitalized individuals who have been seen at a total of approximately 67 000 outpatient visits at nine clinics since 1992 . Our analysis includes data on patients seen at two clinics participating in HOPS from November 1996 to November 1997.
Patients eligible for inclusion had to: (i) be taking two or more antiretroviral medications; (ii) have a plasma HIV-1 RNA titer of 104 copies/ml or greater within 12 weeks of a modification of ART; and (iii) have a HIV-1 RNA titer available for evaluation of response 3–12 weeks after the modification, and no subsequent modifications before the follow-up HIV-1 RNA titer. The practice at one site was to use the results of a genotypic resistance assay to aid in the decision for modification. Mutations in amino acids in the pol or pro genes, thought to be associated with the clinical resistance of HIV-1 to antiretroviral drugs, were determined by standard sequencing methods at one commercial laboratory (Specialty Laboratories, Santa Monica, CA, USA). At the second site, the general practice was not to obtain this test for use in clinical decision making.
Information collected on all patients comprised demographics, pertinent laboratory test results, and complete ART history before the modification. The time span on ART was calculated, and an index of intensity of previous ART (the cumulative time on all antiretroviral medications divided by the span of time on ART). For example, for a person who had been taking two drugs without interruption since ART was begun, the ART intensity index would be 2.0; for a patient taking one drug for half of the time and two drugs for half of the time, the ART intensity index would be 1.5.
The primary endpoint for the analysis was a reduction in viral load of at least 0.5 log within 3–12 weeks after the modification.
Response (≥ 0.5 log reduction in viral load) was modelled as a dichotomous variable. For bivariate analysis, the relative risk (RR) and corresponding 95% confidence intervals (CI) of achieving a desired response were calculated by using the Cochran–Mantel–Haenszel test. To test for statistical significance, the Wilcoxon two-sample test was used for continuous covariates and the chi-square or Fisher's exact test for categorical covariates and dichotomous variables that corresponded to: (i) the start of two new medications; (ii) the start of a new medication from a class of drugs not used in the premodification therapy; (iii) the start of any new nucleoside reverse transcriptase inhibitor (NRTI); (iv) the start of any new non-nucleoside reverse transcriptase inhibitor (NNRTI); (v) the start of any new protease inhibitor; and (vi) the use of a genotypic resistance assay. Because treatment with a protease inhibitor before modification would be expected to influence the response, stratified analyses were performed on this variable to assess for confounding.
Logistic regression multivariate analysis using a backward stepwise elimination for covariates of interest was carried out to determine their association with response in the presence of the other covariates. All these models included as covariates the use of the genotypic resistance assay and the following premodification characteristics: the use of a NNRTI, the use of a protease inhibitor, viral load, CD4 cell count, and ART intensity. The modification characteristics (as listed above), were added to separate logistic models. In all models, two-way interaction terms were included for the use of a genotypic resistance assay with each of the covariates. The dichotomous variable for the use of a protease inhibitor in the regimen before modification was left in all models because it was believed to have a confounding effect.
All statistical analyses were performed by using SAS, version 6.12 (SAS Institute Inc., Cary, NC, USA).
A total of 96 patients were studied, whose ART modifications were made between November 1996 and November 1997 (Table 1). The group had received extensive previous treatment as evidenced by a previous median ART time of 1460 days (range, 112–3458 days) and a median ART intensity index of 1.74 (range, 0.25–3.64).
Before modification, 30 patients (31%) were taking two medications, 50 (52%) were taking three medications, 14 (15%) were taking four medications, and two (2%) were taking five medications. After modification four patients (4%) were taking two medications, 63 (66%) were taking three medications, 27 (28%) were taking four medications, and two (2%) were taking five medications. The pre- and post-modification regimens by drug class are shown in Table 2. Overall, modification of the regimen for the 96 patients resulted in the use of a new NRTI for 52 (54%), a new NNRTI for 28 (29%) and a new protease inhibitor for 62 (65%). For 56 (58%) patients, the modification included at least two new medications. All of the patients started on a new NNRTI were previously naive to this class of drugs, so this term reflects the initiation of a NNRTI not a switch from one to another. A medication from a new class of drugs was added for 47 (49%) patients: a NNRTI for 25 patients, a protease inhibitor for 19 patients, and both for three patients.
Genotypic resistance assay
A genotypic resistance assay was performed for all 44 (46%) patients at one site. Of 44 isolates tested for mutations in the polymerase gene associated with resistance to NRTI, resistance to zidovudine was reported for 36 (82%), lamivudine for 36 (82%), didanosine/zalcitabine for nine (21%), stavudine for seven (16%), combined zidovudine plus lamivudine for 11 (25%), and combined zidovudine plus didanosine/zalcitabine for one (2%). Of the 31 isolates tested for mutations in the protease gene associated with resistance to protease inhibitors, resistance to saquinavir was reported for 16 (52%), indinavir for nine (29%), ritonavir for four (13%), and nelfinavir for two (7%). Neither of the two isolates tested for mutations in the polymerase gene associated with NNRTI resistance were reported as being resistant to nevirapine or delavirdine.
An assessment of viral susceptibility was available for 80% of premodification and 76% of postmodification medications. As summarized from the laboratory reports, the patients' HIV isolates were sensitive to 46% and resistant to 34% of premodification medications. After modification the isolates were sensitive to 64% and resistant to 12% of the medications.
Viral load response
The overall median change in the viral load after modification was −0.7255mulog5mucopies/ml. A reduction of at least a 0.5 log in viral load was demonstrated in 56 (58%) of 96 patients. Viral load became undetectable in 15 (16%) of 96.
Responders and non-responders did not differ in the time known to be HIV positive or in baseline viral load (Table 3). However, the baseline CD4 cell count was higher in responders and, although there was a trend for CD4 cell nadir to be higher in responders, it was not statistically significant (median, 126 versus 71;P = 0.09). The influence of previous therapy revealed that, although not statistically significant, taking a protease inhibitor or a NNRTI in the regimen before the modification were associated with a poorer response than was not taking a drug from the respective classes. Further examination of ART history revealed that responders had a lower median overall previous ART intensity than non-responders. Subanalysis revealed that responders had a lower median previous NNRTI intensity than had non-responders (P = 0.008) and a lower previous protease inhibitor intensity (P = 0.02), but a similar NRTI intensity (P = 0.19).
Relationship of regimen modification to viral load response
In a bivariate analysis involving all 96 patients, those in whom at least two new medications were added were significantly more likely to achieve the desired viral load response (RR, 1.5; CI, 1.1–2.2;P = 0.03). Likewise, patients who began taking a NNRTI or a protease inhibitor (not having taken a drug from the respective class in the premodification regimen) achieved similar results. Analysis of individual drug classes revealed that the initiation of a NNRTI, but not any new NRTI or any new protease inhibitor was also independently associated with a viral load response. Of the 28 patients started on a NNRTI, 16 (80%) of the 20 on a protease inhibitor before the modification achieved a viral load response, as did seven (89%) of the eight who were not on a protease inhibitor before the modification.
Relationship of a genotypic resistance assay to viral load response
The overall median change in viral load in those patients who had a genotypic resistance assay was −0.530 log copies/ml compared with −0.980 log copies/ml in those who did not (P = 0.26). A response (a reduction of at least a 0.5 log in viral load) was observed in 24 (55%) of 44 patients who had a genotypic resistance assay, compared with 32 (62%) of 52 in those who did not (RR, 0.9; CI, 0.6–1.3;P = 0.49). Adjusted for protease inhibitor use just before modification, the estimate of RR changed slightly, but was not statistically significant (adjusted RR, 1.0; CI, 0.7–1.6;P = 0.85).
Relationship of CD4 cell response to viral load response
A CD4 cell count was available on 90 individuals 3–12 weeks after modification of therapy. The overall median change in CD4 cell count from baseline was +17.5 cells/mm3 (interquartile range, −11−−+50). Individuals who had a viral load response had a greater CD4 increase than those who did not (median, +34.55mucells/mm3; interquartile range, +5.5−−+65 versus median, −1.5; interquartile range, −35,−−+28; P<0.001).
By the use of separate multivariate logistic regression analyses, the characteristics of modification that were associated with a response were: (i) the use of at least two new medications; (ii) the use of a medication from a new class of drugs, and (iii) the initiation of a NNRTI (Table 3). The following characteristics of modification were not associated with a response in the models: (i) the use of any new NRTI; and (ii) the use of any new protease inhibitor.
A lower previous ART intensity was associated with a response and higher CD4 cell count before modification was of borderline significance in all models. Use of the genotypic resistance assay, viral load before modification, the use of a NNRTI before modification, or any of the interactions between covariates and the use of the genotypic resistance assay were not associated with response in any of the models. The results of the five logistic regression models, adjusted for previous ART intensity, CD4 cell count before modification, and the use of a protease inhibitor before modification are shown in Table 3.
In patients with extensive antiretroviral treatment, responses to modifications in therapy, as measured by a reduction in viral load between 3 and 12 weeks, were more likely when two or more medications were replaced or patients who had not been taking a NNRTI or a protease inhibitor began to take one of these drugs. These results support the recommendations of the International AIDS Society  and the US Department of Health and Human Services  for changing ART in patients who are failing it; that is, ideally to replace the regimen completely with different drugs to which the patient is naive, but, if that is not possible, with at least two new medications.
This primary analysis was to compare the first modification in therapy during the time period of observation. It was decids.
This primary analysis was to compare the first modification in therapy during the time period of observation. It was decided, a priori, not to use a very stringent definition of response, such as a reduction in viral load to undetectable levels, because such responses are often not obtained in a group of heavily treated patients . Although our definition of a viral load reduction may appear to be modest, such changes in viral load, if sustained, may translate into clinical benefit . However, it is unlikely that such a modest decline in viral load after the failure of one of a series of sequential modifications of therapy will be sufficient for a durable suppression of viral replication, or will prevent the development of further resistance to the new medications started . However, such responses may be what was commonly attainable with the strategies used during the period of observation.
The standard practice during much of the observation period was to use dual NRTI for the initiation of therapy , but by mid-1997 the standard initial therapy was what is commonly referred to as `highly active antiretroviral therapy' (HAART), which comprised dual NRTI plus either a protease inhibitor or a NNRTI . During this time period the commercial use of a first generation genotypic resistance assay was being incorporated into some clinical practices. These data provide no evidence that the use of this commercial genotypic resistance assay during this time period was associated with a better outcome; as measured by short-term reductions in viral load, in patients having ART modifications.
The lack of benefit of a commercial genotypic resistance assay in directing modifications must be interpreted cautiously. First, more of those patients whose ART modifications were partly guided by a genotypic resistance assay had been taking a protease inhibitor, and were thus less likely to respond to a modification of therapy. Several separate analyses controlling for this confounding variable statistically and analyses comparing only those with strictly comparable previous ART did not change the result. That is, the use of a genotypic resistance assay was not associated with a discernible benefit (RR always near 1.0). However, there is no reliable way to control all potential biases. Also, although unavoidable, this study was performed during a time of marked evolution in ART, and the sequential addition during the observed time period on the commercial genotypic resistance assay reports of resistance to NRTI, protease inhibitors, and NNRTI. Finally, most subjects had advanced HIV infection and received extensive previous ART, thus limiting the choices of ART likely to be effective. We therefore think that whereas our study detected no benefit from the use of a genotypic resistance assay and may not apply to patients with less advanced disease and treatment, this issue clearly requires the continuing study it is receiving.
Our findings must also be interpreted in the context of the particular time during which the study was conducted, November 1996 to November 1997. Many of these patients had been taking ART since the era of zidovudine monotherapy, through the period of sequential monotherapy, followed by the period of dual NRTI, and into the current era of HAART. With only three drug classes available and the periodic marketing of new drugs within these classes, the options for these patients may have been exhausted. In addition, our calculation of the antiretroviral intensity index is a measure of the total accumulated drug history of patients in this study from that time period. Analyses using total time on ART therapy instead of the intensity index produced similar overall results (data not shown). However, either measure is an arbitrary method to estimate previous drug exposure and thus chance for development of resistance. Neither measure calculated in patients in 1996 or 1997 who had been on therapy for a period of years would necessarily be comparable to patients who started therapy more recently with aggressive three or more drug therapy. Such therapy would lead to a higher cumulative time on therapy and the intensity index, but there may be a different relationship between the measure and response to a change in therapy.
Evidence was found, in outpatient clinic practices of patients with extensive previous ART treatment and a viral load of 104 log copies/ml or greater, that changing two or more medications and using a medication from a new class of drugs; whether the decisions were made empirically or were partly guided by the results of a commercial genotypic resistance assay, was associated with short-term reductions in viral load. These results support recommendations and clinical decision-making practices for individuals who are not responding to ART; that is, ideally to replace the regimen completely with different drugs to which the patient is naive, but, if not possible, with at least two new medications.
1. Palella FJ, Delaney KM, Moorman AC. et al
. Declining morbidity and mortality among patients with advanced human immunodeficiency virus infection.
N Engl J Med 1998, 338: 853 –860.
2. State-of-the-Art Conference on Azidothymidine Therapy for Early HIV Infection. Am J Med
3. Sande MA, Carpenter CCJ, Cobbs G, Holmes KK, Sanford JP. Antiretroviral therapy for adult HIV-infected patients.
:Recommendations from a state-of the-art conference.
JAMA 1993, 270: 2583 –2589.
4. Carpenter CCJ, Fischl MA, Hammer SC. et al
. Antiretroviral therapy for HIV infection in 1996.
:Recommendations of an international panel.
JAMA 1996, 276: 146 –154.
5. Condra JH, Schleif WA, Blahy OM. et al
. In vivo emergence of HIV-1 variants resistant to multiple protease inhibitors.
Nature 1995, 374: 569 –571.
6. Brun-Vezinet F, Boucher C, Loveday C. et al
. HIV-1 viral load, phenotype, and resistance in a subset of drug-naive participants from the Delta trial
. Lancet 1997, 350: 983 –990.
7. Shafer RW, Winters MA, Palmer S, Merigan TC. Multiple concurrent reverse transcriptase and protease mutations and multidrug resistance of HIV-1 isolates from heavily treated patients.
Ann Intern Med 1998, 128: 906 –911.
8. Japour AJ, Welles S, D'Aquila RT. et al
. Prevalence and clinical significance of zidovudine resistance mutations in human immunodeficiency virus isolated from patients after long-term zidovudine treatment.
J Infect Dis 1995, 171: 1172 –1179.
9. Rey D, Hughes M, Pi J, Winters M, Merigan RC, Katzenstein DA. HIV-1 reverse transcriptase codon 215 mutation in plasma RNA: immunologic and virologic responses to zidovudine.
J Acquired Immune Defic Syndr Hum Retrovirol 1998, 17: 203 –208.
10. Carpenter CCJ, Fischl MA, Hammer SC. et al
. Antiretroviral therapy for HIV infection.
:Updated recommendations of the International AIDS Society – USA panel.
JAMA 1997, 277: 1962 –1969.
11. Carpenter CCJ, Cooper DA, Fischl MA. et al
. Antiretroviral therapy in adults.
:Updated recommendations of the International AIDS Society – USA panel.
JAMA 2000, 283: 381 –390.
12. Centers for Disease Control and Prevention. Report of the NIH panel to define principles of therapy of HIV infection and guidelines for the use of antiretroviral agents in HIV-infected adults and adolescents.
Available at http://www.hivatis.org
. Accessed 28 January 2000. [Originally published in MMWR
13. Baxter JD, Mayers DL, Wentworth DN, Neaton JD, Merigan TC, and the Study Team for the Terry Beirn Community Programs for Clinical Research on AIDS (CPCRA). Final results of CPCRA 046: a pilot study of antiretroviral management based on plasma genotypic antiretroviral resistance testing (GART) in patients failing antiretroviral therapy. [Abstract]
In:Program and Abstracts of 3rd International Workshop on HIV Drug Resistance and Treatment Strategies
. San Diego, CA: 61. Available from International Medical Press, Atlanta, GA, USA.
14. Durant J, Clevenbaugh P, Halfon P. et al
. Drug-resistance genotyping in HIV-1 therapy: the VIRADAPT randomised controlled trial.
Lancet 1999, 353: 2195 –2199.
15. Hirsch MS, Conway B, D'Aquila RT. et al
. Antiretroviral drug resistance testing in adults with HIV infection.
:Implications for clinical management.
JAMA 1998, 279: 1984 –1991.
16. Schinarzi RF, Larder BA, Mellors JW. Mutations in retroviral genes associated with drug resistance.
Int Antiviral News 1996, 4: 95 –107.
17. Deeks SG, Abrams DI. Genotypic-resistance assays and antiretroviral therapy.
Lancet 1997, 349: 1489 –1490.
18. Condra JH. Resisting resistance: maximizing the durability of antiretroviral therapy.
Ann Intern Med 1998, 128: 951 –954.
19. Perrin L, Telenti A. HIV treatment failure: testing for HIV resistance in clinical practice.
Science 1998, 280: 1871 –1873.
20. Fallon J. Time to genotype for selection of antiretroviral regimens in previously treated patients?
Lancet 1999, 353: 2173 –2174.
21. Lucas GM, Chaisson RE, Moore RD. Highly active antiretroviral therapy in a large urban clinic: risk factors for virologic failure and adverse drug reactions.
Ann Intern Med 1999, 131: 81 –87.
22. Kaufmann D, Pantaleo G, Sudre P, Telenti A, for the Swiss HIV Cohort Study. CD4-cell count in HIV-1-infected individuals remaining viraemic with highly active antiretroviral therapy (HAART).
Lancet 1998, 351: 723 –724.
23. Colgrove R, Japour A. A combinatorial edge: reverse transcriptase fidelity, total body viral burden, and the implications of multiple-drug HIV therapy for the evolution of antiviral resistance.
Antiviral Res 1999, 41: 45 –56.
The HIV Outpatient Study Investigators, 1998–January 1999
The HOPS ivestigators include the following investigators and sites: A.C. Moorman, J.C. Von Bargen, P.J. Weidle and S.D. Holmberg, Division of HIV/AIDS Prevention, National Center for HIV, STD, and TB Prevention (NCHSTP), Centers for Disease Control and Prevention (CDC), Atlanta, GA, USA. K.C. Wood, F.D. Fett and R.K. Baker, Health Research Network of APACHE Medical Systems, Inc., McLean, VA, USA; F.J. Palella, J.S. Chmiel, C. Chan and J. Arnold, Northwestern University Medical School, Chicago, IL, USA; K.A. Lichtenstein, K.S. Greenberg, B. Young, B. Widdick, C. Stewart and P. Zellner, Columbia Rose Medical Center, Denver, CO, USA; B.G. Yangco, K. Halkias and C. Lapierre, Infectious Disease Research Institute, Tampa, FL, USA; D.J. Ward and C. Owens, Washington, DC, USA; J. Fuhrer, L. Ording-Bauer, R. Kelly and M. Nekola, State University of New York (SUNY), Stony Brook, NY, USA; E.M. Tedaldi and S. Smith, Temple University Hospital, Philadelphia, PA, USA; J.B. Marzouk, R.T. Phelps, and M. Rachel, Adult Immunology Clinic, Oakland, CA, USA; R.E. McCabe and M. Rachel, Fairmont Hospital, Oakland, CA, USA. Cited Here...
This article has been cited 14 time(s).
Journal of Acquired Immune Deficiency Syndromes
Comparison of different treatment regimens for the emergence of new resistance under therapy
Journal of Acquired Immune Deficiency Syndromes, 27(4):
The effect of number of mutations and of drug-class sparing on virological response to salvage genotype-guided antiretroviral therapy
Antiviral Therapy, 8(6):
Clinical Infectious Diseases
Extent of human immunodeficiency virus type 1 drug resistance as a predictor of virological failure after genotype-guided treatment switch
Clinical Infectious Diseases, 33(5):
Journal of Molecular BiologyCrystal structures of clinically relevant Lys103Asn/Tyr181Cys double mutant HIV-1 reverse transcriptase in complexes with ATP and non-nucleoside inhibitor HBY 097Journal of Molecular Biology
Virological response to salvage therapy in HIV-infected persons carrying the reverse transcriptase K65R mutation
Antiviral Therapy, 12(8):
Medizinische KlinikDrug resistance in antiretroviral therapy of HIV infectionMedizinische Klinik
Journal of Acquired Immune Deficiency Syndromes"Discordant" increases in CD4 cell count relative to plasma viral load in a closely followed cohort of patients initiating antiretroviral therapyJournal of Acquired Immune Deficiency Syndromes
AIDS Patient Care and Stds
Theory-guided, empirically supported avenues for intervention on HIV medication nonadherence: Findings from the healthy living project
AIDS Patient Care and Stds, 17():
International Journal of Infectious DiseasesAntiretroviral treatment outcome following genotyping in Thai children who failed dual nucleoside reverse transcriptase inhibitorsInternational Journal of Infectious Diseases
Durability and predictors of success of highly active antiretroviral therapy for ambulatory HIV-infected patients
antiretroviral therapy; genotypic; modification; resistance
© 2000 Lippincott Williams & Wilkins, Inc.
What does "Remember me" mean?
By checking this box, you'll stay logged in until you logout. You'll get easier access to your articles, collections,
media, and all your other content, even if you close your browser or shut down your
To protect your most sensitive data and activities (like changing your password),
we'll ask you to re-enter your password when you access these services.
What if I'm on a computer that I share with others?
If you're using a public computer or you share this computer with others, we recommend
that you uncheck the "Remember me" box.
Highlight selected keywords in the article text.
Data is temporarily unavailable. Please try again soon.