Secondary Logo

Journal Logo

Clinical Science

Development and Validation of HIV-ASSIST, an Online, Educational, Clinical Decision Support Tool to Guide Patient-Centered ARV Regimen Selection

Maddali, Manoj V. MDa; Mehtani, Nicky J. MD, MPHb; Converse, Caro BAc; Kapoor, Sunaina MD, MPHd; Pham, Paul PharmDe; Li, Jonathan Z. MDf; Shah, Maunank MD, PhDe

Author Information
JAIDS Journal of Acquired Immune Deficiency Syndromes: October 1, 2019 - Volume 82 - Issue 2 - p 188-194
doi: 10.1097/QAI.0000000000002118

Abstract

BACKGROUND

There are currently an estimated 1.1 million persons living with HIV (PLWH) in the United States.1 Because of advances in antiretroviral (ARV) tolerability and potency, more people are living longer with HIV.2–5 Providing optimal care for these persons requires providers trained in and dedicated to HIV practice.6–8

Delivery of excellent lifelong care for PLWH is challenging, requiring up-to-date knowledge of HIV medicine, including selection and monitoring of ARV therapy. In addition to the 30+ ARV drugs in 7 different mechanistic classes currently approved by the US Food and Drug Administration (FDA), there are also a wide array of fixed-dose ARV combinations.9 The selection of an optimal ARV regimen is a nuanced medical decision, with current guidelines advocating for individualized therapy that accounts for multiple patient and viral characteristics.10,11 Factors that must be considered include previous ARV treatment, adherence, HIV resistance mutations, viral load, comorbidities (eg, renal function and cardiovascular disease), and drug interactions. Ultimately, the selection and management of appropriate ARV regimens requires a detailed understanding of current treatment and prevention principles. Improved education for potential HIV providers is thus of critical concern.

Although there are many existing HIV educational resources, there are very few interactive tools that allow for patient-specific decision support and tailored educational material.12,13 Current guidelines are comprehensive, but can be time-consuming and challenging to navigate.10,11 For example, when selecting a regimen for ARV initiation or modification, many patients do not fall neatly into categories that have been explicitly studied. Although data may be available related to risks associated with some comorbidities, resistance mutations, or comedications individually (eg, cardiovascular disease or drug resistance to some ARV classes), there is less literature available on how to integrate these attributes and comprehensively approach patients with more than one such modifying factor.

Our objective was to develop an online tool (HIV-ASSIST, https://www.hivassist.com) to provide patient-specific clinical decision support while enhancing provider education and knowledge. Through HIV-ASSIST, we hoped to develop an objective, quantifiable approach to ARV selection that weighs and presents information on each of these factors in a transparent manner to assist clinicians in making treatment decisions.

METHODS

We developed HIV-ASSIST (https://www.hivassist.com) as an online, interactive, educational tool to inform clinical decision-making for ARV selection. HIV-ASSIST was developed using widely available open-source software including PHP (Zend Technologies; version 5.6), MySQL (Oracle Corporation; version 5), Drupal (version 7), and jQuery (version 1.10). HIV-ASSIST (version 1.1.0) is currently available online and is free of charge to use.

Algorithm Development (Multiple-Criteria Decision Analysis)

In developing HIV-ASSIST, we used a multiple-criteria decision analysis (MCDA) framework, in which the set of available therapeutic options is finite and known.14,15 To implement this, we built a value measurement model to mathematically capture the clinical ARV selection process. Our goal was to generate a numerical aggregate value to allow for comparison and ranking of all ARV regimen choices for a particular patient-case scenario. In constructing a multi-attribute utility function, we applied a compositional approach that generated and subsequently combined separate estimates of scores and utility weights for individual elements in the clinical decision-making process. We acknowledged the presence of multiple decision-outcome goals (ie, multiple-criteria) in development of the value measurement model. Within the desirable objectives of ARV selection, we considered achievement of virologic suppression to have primacy, with secondary goals to maximize tolerability, minimize drug-interactions and side-effects, and maximize adherence (eg, small pill size, number, and dosing frequency).

We defined our set of ARV regimen options to be all permutations of 2-, 3-, and 4-drug combinations of FDA-approved ARVs (doravirine, enfuviritide, and ibalizumab were not included in the initial version). We then defined a list of factors related to ARV selection, drawing upon national guidelines and clinical experience. We constructed utility weights (or utility functions resulting in utility weights) for each individual ARV and ARV regimen in relation to our predefined goals of ARV selection. Initial valuation of utility weights was based on the development team's clinical experience, with iterative refinement based on user feedback and expert informants at Johns Hopkins Hospital (JHH) and Brigham and Women's Hospital (BWH). We subsequently developed a multi-attribute utility function that yielded a numeric value (“Weighted Score”) for each of the possible treatment options, with lower scores representing greater composite value or overall favorability of that particular ARV regimen. HIV-ASSIST incorporates a web-based user interface where clinicians can enter patient-specific variables and are presented an ordered “rank” list of treatment options that are based on the HIV-ASSIST Weighted Scores.

Validation Study

We conducted a validation survey by distributing 10 hypothetical patient-case scenarios of varying degrees of clinical complexity (see Table 1, Supplemental Digital Content, https://links.lww.com/QAI/B348, ARV-naive and -experienced, with and without comorbidities/comedications, and with and without ongoing viremia) to a convenience sample of experienced HIV providers not previously involved in the HIV-ASSIST algorithm development. The survey participants included HIV providers at JHH, BWH, the University of California San Francisco, and Massachusetts General Hospital, who were asked to select an ARV regimen for each scenario by free response (without presentation or knowledge of HIV-ASSIST results). We report on concordance of participant responses with the top 5 ARV regimens ranked by HIV-ASSIST; for purposes of analysis, cobicistat- and ritonavir-boosted PI regimens were counted only once among ranked outputs. As a secondary measure of agreement, we calculated the difference between the “Weighted Score” of the top-ranked HIV-ASSIST ARV regimen and participant-selected regimen, defining a difference of <1.0 as good agreement, 1.1–2.0 as moderate agreement, and >2.0 as poor agreement. Following each case scenario, survey participants were presented the 5 top-ranked HIV-ASSIST outputs (in a randomized order) and asked to indicate if any were considered medically contraindicated (see Supplementary Content, https://links.lww.com/QAI/B348).

RESULTS

Algorithm Development

1a. Algorithm Inputs (Defining Attributes for a Multi-Attribute Utility Function)

Based on a review of current guidelines, we identified 9 key patient- and virus-specific attributes for consideration in ARV selection.10,11 We additionally identified several patient-independent attributes related to ARV tolerability (eg, pill size or dosing frequency and side effects), efficacy, and barrier to resistance (Fig. 1).

F1
FIGURE 1.:
Schematic of HIV-ASSIST multi-utility attribute function. *Patient-independent attributes.**Applies greater numeric penalty to regimens not meeting these criteria. †All ARVs were assigned a numeric utility score representing the degree of preference or relative contraindication when administered with a given comorbidity or comedication.

1b. Individual ARV Evaluation

Each identified attribute was considered a separate dimension affecting an individual ARV's activity or tolerability. A particular ARV's utility weight incorporated degree of activity based on known or suspected genotypic HIV mutations, drawing upon a numerical scale defined by the Stanford HIV Drug Resistance Database.16 Individual ARV utility weights were also influenced by a patient's comorbidities or comedications and were either penalized or excluded based on unfavorable interactions with a given comorbidity or comedications. Utility weights related to comorbidities or comedications were based on the team's interpretation of degree of impact on tolerability or other secondary outcomes of interest (eg, need for dosage adjustments or monitoring) from data provided about drug interactions and comorbidities within DHHS guidelines, a publicly available HIV drug interaction database, and input from an experienced HIV pharmacist.10,17 Tropism and HLA-B5701 status were similarly incorporated as utility functions that either allowed or excluded maraviroc or abacavir, respectively, from final consideration. Additional utility weight considerations included pill burden, side-effect profile, and relative barrier to resistance. An individual ARV's net utility weight combined these attributes and represented its composite drug activity and tolerability (Fig. 1).

Multi-Drug Regimen Evaluation

After generating composite utility weights for individual ARVs, we constructed additional utility functions to evaluate multi-drug ARV regimens across all attribute dimensions simultaneously (Fig. 1). Utility weights associated with independent attributes such as viral load, CD4 cell count, number of active drugs, pill burden, and dosing frequency were combined with utility weights of each individual ARV within a particular regimen to result in a numerical aggregate score (“Weighted Score”). This multi-attribute function was calibrated to generate a Weighted Score of 1.0 for regimens recommended “for most people” by DHHS guidelines for ARV-naive individuals without any comorbidities or comedications.10 Regimens with lower aggregate Weighted Scores are interpreted as being more preferred for the composite objective of achieving viral suppression and maximizing tolerability and adherence. By contrast, higher Weighted Scores (eg, >2.0) can be considered less preferred in relation to this composite objective or as representing regimens with poor evidence or evidence against usage. Additional algorithm details are available; see Supplemental Digital Content; https://links.lww.com/QAI/B348, shows the HIV-ASSIST input page and results for the default patient-case scenario (ARV-naive without any comorbidities or comedications).

Educational Content and Algorithm Transparency

To couple real-time clinical decision support with medical education, we created “Education Sheets” summarizing guideline recommendations and clinic trial evidence for all ARV regimens displayed by HIV-ASSIST. Educational material also included dosing and administration information, including adjustments for renal or hepatic insufficiency. To ensure transparency of all algorithms as an educational tool, and recognizing that clinicians may weigh some considerations (eg, the impact of a drug interaction) differently from the developers of HIV-ASSIST, all utility weights and functions involved in constructing given regimen's final ranking and Weighted Score are made available to the user (see Supplemental Digital Content, https://links.lww.com/QAI/B348).

Validation Study

Table 1 (see Supplemental Digital Content, https://links.lww.com/QAI/B348) shows results of a validation study to assess concordance between HIV-ASSIST algorithm-generated ranked outputs and ARV prescribing preferences for a cohort of 17 experienced HIV providers (8 from JHH, 3 from BWH/Massachusetts General Hospital, and 6 from University of California San Francisco; most of whom had 10+ years of experience managing HIV). Among the 4 hypothetical case scenarios of ARV-naive patients, we found 99% (67/68, range 94%–100% for each case individually) concordance between provider-free responses and the 5 top-ranked HIV-ASSIST outputs (see Table 1, Supplemental Digital Content, https://links.lww.com/QAI/B348, Scenarios #1–4). For all cases involving ARV-naive patients, there was good agreement between the top-ranked HIV-ASSIST regimen and the participant response based on the study-defined measure of agreement. The only discordant response featured an ARV-naive patient with tuberculosis on rifampin, isoniazid, pyrazinamide, and ethambutol (scenario #4). For this scenario, HIV-ASSIST prioritized EFV/TDF/FTC as a single-tablet, once-daily regimen easily administered with directly observed therapy. The discordant response was attributable to its inclusion of TAF, which HIV-ASSIST excluded due to potential drug interactions between TAF and rifampin (while some data exists on using this combination, HIV-ASSIST algorithms drew upon current guidelines at the time of the study that recommended against usage).18

Among 3 cases of ARV-experienced patients with viremia and different degrees of resistance (see Table 1, Supplemental Digital Content, https://links.lww.com/QAI/B348, Scenarios #5–7), there were 20 different ARV regimens suggested by the study participants, demonstrating heterogeneity in clinical practice. Among these 3 cases, 24% (12/49; range 13%–38% individually) of participants selected the same regimen as the top HIV-ASSIST ranked output, and 84% (41/49, range 69%–94% individually) selected a regimen concordant with the 5 top-ranked outputs. The higher Weighted Scores of HIV-ASSIST outputs (0.5–1.95, 2.4–3, 1.3–3.4, for Scenarios #5–7, respectively) reflect relatively poorer ARV regimen utility scores compared with options available for the ARV-naive cases. Nonetheless, the median differences in Weighted Scores comparing the top-ranked HIV-ASSIST output and participant selections for these 3 cases were 0.5 (IQR 0.5–1.5, good-to-moderate agreement), 0 (IQR 0–0.6, good agreement), and 0.65 (IQR 0–1.95, good-to-moderate agreement), respectively. Of note, 2 responses (one in scenario #5 and #7, respectively) were excluded from statistical analysis for containing doravirine, which was not included in HIV-ASSIST at the time of the study.

Among the remaining discordant responses across the 3 scenarios of ARV-experienced and viremic patients (8/49; 16%), BIC/TAF/FTC was chosen among 5 of 8 responses (63%; 3/16 and 2/16 for scenario #5 and #7, respectively). In both scenarios, BIC/TAF/FTC was the sixth-ranked HIV-ASSIST output, with a lower net utility weight partially attributable to numeric penalizations for current lack of data of BIC/TAF/FTC in patients with drug resistance or history of treatment failure. One discordant response in scenario #6 (1/8; 13%) was attributable to DTG + TAF/FTC, which was numerically penalized by HIV-ASSIST for having less than 2 fully active drugs in the setting of extensive Nucleoside Reverse Transcriptase Inhibitor resistance.

We evaluated participant approaches to ARV simplification for virally-suppressed patients in Scenarios #8–10 (see Table 1, Supplemental Digital Content, https://links.lww.com/QAI/B348). Each case yielded between 3 and 8 discrete ARV regimens selected by the 17 respondents. Overall, there was 88% (44/50; range 81%–100% individually) concordance between these responses and the 5 top-ranked outputs of HIV-ASSIST. Among regimens not excluded by HIV-ASSIST, there was a median difference in Weighted Score comparing the top-ranked HIV-ASSIST output and participant selection of 0 (IQR 0, good agreement), 0 (IQR 0–0.5, good agreement), and 0.15 (IQR 0.1–0.8, good agreement), for Scenarios #8–10 respectively. As above, we excluded one response containing doravirine in scenario #8 from statistical analysis. Among the remaining discordant-free responses (6/50; 12%), 3 were attributable to participant selection of a regimen containing once-daily DRV/c in the setting of PI mutations (scenario #8), which are excluded by HIV-ASSIST based on current DHHS guidelines advocating for twice-daily dosing of DRV in such situations.10 Furthermore, in scenario #10 (featuring multi-class resistance with Nucleoside Reverse Transcriptase Inhibitor, Non-Nucleoside Reverse Transcriptase Inhibitor, and Integrase Strand Transfer Inhibitor mutations), 2 participant responses (DTG/RPV and DTG bid + DRV/c) were considered to have fewer than 2 active drugs, resulting in numerical penalizations.

We additionally asked participants to indicate whether they considered HIV-ASSIST outputs to be medically contraindicated, along with their rationale (see Supplemental Digital Content, https://links.lww.com/QAI/B348). Across all scenarios, 5% (9/170 responses) of the top-ranked HIV-ASSIST outputs were considered to be medically contraindicated (0% for scenarios #1–3 and #9–10) by participants. Among cases in which at least one participant considered the top HIV-ASSIST output to be medically contraindicated (scenarios #4–8), 13%–69% of other respondents considered the same top-ranked HIV-ASSIST output to be their preferred regimen (see Table 2, Supplemental Digital Content, https://links.lww.com/QAI/B348).

DISCUSSION

ARV selection—like most nuanced decisions in clinical medicine—involves a complex interaction of guideline interpretation, internal algorithms, heuristics, and, most importantly, shared decision-making with the patient. Current guidelines advocate for an individualized approach to ARV selection, incorporating a variety of patient- and virus-related characteristics that can be challenging to implement in a busy clinical practice. With recent FDA approval of multiple new ARVs and more in the developmental pipeline, future HIV providers will be faced with more treatment considerations and therapeutic options. There thus exists a need for real-time, patient-tailored decision support and education to support providers in comparing and choosing between various ARV treatment options. We describe the systematic development of an online, easy-to-use tool to support clinicians in tailored, individualized ARV selection across a wide range of clinical scenarios; results of our study suggest that HIV-ASSIST outputs are strongly concordant with practice patterns of experienced HIV clinicians at major academic centers for both ARV-naive and experienced patient scenarios.

Decision analysis is a quantitative and systematic framework that allows for comparison of several uncertain outcomes (eg, viral suppression, tolerability, and drug interactions) involving multiple potential trade-offs.19 Decision analysis can be used to guide management options in both patient (individual-level) and public health settings. For example, Markov models can be used to mathematically analyze clinical problems with continuous risk or iterative decisions (eg, anticoagulation in a patient with both embolic and hemorrhagic risk).20 Alternatively, health economists frequently rely on decision-tree models to guide population-level policy decisions, often to maximize quality-adjusted life-years and cost-effectiveness.21 However, such models are not easily applied to individual-level decisions with a large number of inputs, treatment options, and objectives. Guidelines developed by panels of experts offer one framework for establishing norms in ARV treatment selection for a variety of scenarios. However, these guidelines are subject to variable interpretation, with more precise recommendations for ARV-naive patients and less discrete recommendations for most other clinical permutations (eg, among ARV-experienced, viremic patients with drug resistance “a new regimen should include at least 2, and preferably 3, fully-active agents”).10 This heterogeneity of guideline interpretation was evident in our survey results, particularly for situations of ARV-experienced patients. There is currently no single accepted framework to help providers quickly navigate the relative strengths or potential negative health effects of an increasing number of potential ARV choices.

MCDA is a framework that allows for comparison of multiple options when there are multiple objectives to satisfy (ie, viral suppression and tolerability).14,15 We applied MCDA principles to create a clinical decision support tool to assist HIV providers by generating a weighted utility score that can be used to rank nearly all possible multi-drug ARV combinations under a variety of user-defined scenarios. Our approach is novel in attempting to objectively quantify the various components of the ARV decision-making process, allowing synthesis of multiple considerations into a single aggregated value. Our pilot study comparing HIV-ASSIST outputs (ranked according to aggregate “Weighted Score”) with those of a cohort of experienced HIV clinicians suggests a high degree of concordance (approaching 100%) for scenarios involving ARV-naive patients. Among treatment-experienced patients, we found wide provider heterogeneity in ARV-prescribing preferences. This was evidenced by no single ARV regimen being chosen by more than 38% of respondents across 3 scenarios describing individuals with current viremia. In this context, our overall results suggest that HIV-ASSIST ranked outputs are in very good agreement with provider preferences of experienced clinicians at 4 academic institutions, for a variety of complex case scenarios. Our results also suggest widely variable provider practices, with regimens considered “contraindicated” by some respondents, yet chosen as the regimen of choice by a similar or larger number of respondents (see Supplemental Digital Content, https://links.lww.com/QAI/B348). The transparency of HIV-ASSIST algorithms can help clinicians elucidate specific areas of heterogeneity in decision-making practices.

Recent studies have shown that more than half of HIV care in the US is provided by primary care practitioners.22–25 As PLWH live longer, there is projected to be an increasing demand for HIV and primary care services.4,8,22 Unfortunately, recent forecasts suggest declining numbers of primary care physicians and HIV providers.26,27 Furthermore, a survey of Internal Medicine program directors showed that only a minority believed their graduates had the skills to be primary providers for PLWH.28 Consequently, HIV care in the US is often managed by clinicians from multiple specialties, many of whom have not had formal training in HIV management.22 HIV-ASSIST was created in part to couple decision support with educational content (such as summaries of clinical trial evidence and a “narrative report” describing algorithm rationale for any specific ARV regimen) and allow for integration of effective and efficient learning of HIV management principles, while simultaneously enhancing patient care (see Supplemental Digital Content, https://links.lww.com/QAI/B348).

Our work has several limitations. First, HIV-ASSIST represents a tool to aid decision-making and is not a substitute for clinical judgement. There may be additional patient- or virus-related considerations not fully captured in current algorithms. Second, there is no currently accepted comprehensive utility index related to composite objectives (viral suppression and tolerability). Uncertainty and variability in provider preferences could influence the relative concordance of HIV-ASSIST results with individual provider practices. Nonetheless, HIV-ASSIST presents each regimen's component utility scores for users to explore and evaluate for themselves, consistent with good practice guidance for decision analysis (see Figure 3, Supplemental Digital Content, https://links.lww.com/QAI/B348).14,15 Third, we considered incorporation of ARV costs into the utility scores, but chose not to do so in the initial version, largely because of national variability in cost-sharing among patients, insurance programs, AIDS Drug Assistance Programs, and other payers; consequently it remained unclear how clinicians (and patients) are incorporating cost considerations into individual-level decisions. Our validation study also has several limitations. Survey responses may have been influenced by regional and institutional variability in HIV practice patterns. Our validation study may not be representative of practice patterns in other areas, to the extent that practices in other areas differ. Furthermore, a minority of respondents suggested newer agents (such as doravirine) that are not yet included in HIV-ASSIST algorithms. There were also differences in interpretation of some survey instructions; for example, several respondents interpreted “medical contraindication” as less preferred regimens (eg, because of increased pill burden; see Table 2, Supplemental Digital Content, https://links.lww.com/QAI/B348). Finally, in our iterative algorithm development process and validation study, we encountered lack of consensus on best approaches to many common clinical scenarios. For example, there was heterogeneity in management of virus with M184V mutations, the preferred number of active drugs, usage of DTG/RPV for treatment simplification, dosing frequency of DTG or DRV in the setting of Integrase Strand Transfer Inhibitor or PI mutations, respectively, and usage of BIC/TAF/FTC in combination with other drugs or in patients with a history of treatment failure. To accommodate some differences in provider approaches, we incorporated several user-defined inputs that allow the user to prioritize regimens with smaller pills, once-daily dosing, and 3 active drugs (Fig. 2A).

F2
FIGURE 2.:
A, HIV-ASSIST input page with sample inputs. B, HIV-ASSIST results with default inputs (ARV-naive patient with viremia and no resistance, without any comorbidities or comedications).

Our approach to constructing HIV-ASSIST has several important strengths. We are among the first to attempt to synthesize the various dimensions of ARV decision-making using a quantifiable and systematic process. Second, we refined algorithms iteratively over several years based on scientific literature, guidelines, and expert input. Our validation study results suggest that HIV-ASSIST outputs are largely consistent with experienced HIV providers at major academic centers for a variety of patient scenarios. Our results also identified several important areas of provider heterogeneity that should be addressed in future clinical studies. Finally, our approach has sought to couple longitudinal medical education with decision support and represents an innovative strategy to address important projected gaps in the HIV workforce. Future directions and challenges include addition of newer ARV agents (and costs), incorporation into the electronic medical record, development of a mobile platform, and evaluation of impact (of HIV-ASSIST implementation) on provider knowledge and patient outcomes.

ACKNOWLEDGMENTS

The authors thank Paul Sax and Dan Kuritzkes for their guidance in refining HIV-ASSIST algorithms. The authors thank Jacob Coleman and Laura Platt for their help in developing educational content.

REFERENCES

1. HIV in the United States and Dependent Areas. Centers for Disease Control and Prevention. Available at: https://www.cdc.gov/hiv/statistics/overview/ataglance.html. Accessed February 25, 2019.
2. HIV Surveillance Reports. Centers for Disease Control and Prevention. Available at: https://www.cdc.gov/hiv/library/reports/hiv-surveillance.html. Accessed on February 25, 2019.
3. Justice AC. HIV and aging: time for a new paradigm. Curr HIV/AIDS Rep. 2010;7:69–76.
4. Samji H, Cescon A, Hogg RS, et al. Closing the gap: increases in life expectancy among treated HIV-positive individuals in the United States and Canada. PLoS One. 2013;8:e81355.
5. Deeks SG, Lewin SR, Havlir DV. The end of AIDS: HIV infection as a chronic disease. Lancet. 2013;382:1525–1533.
6. Grossman HA. Addressing the need for HIV specialists: the AAHIVM perspective. AIDS Read. 2006;16:479–486.
7. Landovitz RJ, Desmond KA, Gildner JL, et al. Quality of care for HIV/AIDS and for primary prevention by HIV specialists and nonspecialists. AIDS Patient Care STDS. 2016;30:395–408.
8. Weiser J, Beer L, West BT, et al. Qualifications, demographics, satisfaction, and future capacity of the HIV care provider workforce in the United States, 2013–2014. Clin Infect Dis. 2016;63:966–975.
9. Antiretroviral Drugs Used in the Treatment of HIV Infection. US Food and Drug Administration (FDA). Available at: https://www.fda.gov/forpatients/illness/hivaids/ucm118915.htm. Accessed February 25, 2019.
10. Guidelines for the Use of Antiretroviral Agents in Adults and Adolescents Living With HIV. US Department of Health and Human Services (DHHS). Available at: https://aidsinfo.nih.gov/contentfiles/lvguidelines/adultandadolescentgl.pdf. Accessed February 25, 2019.
11. Saag MS, Benson CA, Gandhi RT, et al. Antiretroviral drugs for treatment and prevention of HIV infection in adults: 2018 recommendations of the international antiviral society-USA panel. JAMA. 2018;320:379–396.
12. Decision Support Tools. Clinical Care Options HIV. Available at: https://www.clinicaloptions.com/HIV/Tools. Accessed February 25, 2019.
13. Eron JK; D. inPractice HIV. inPractice; 2019: Available at: https://www.inpractice.com/Textbooks/HIV.aspx. Accessed February 25, 2019.
14. Thokala P, Devlin N, Marsh K, et al. Multiple criteria decision analysis for health care decision making--an introduction: report 1 of the ISPOR MCDA emerging good practices task force. Value Health. 2016;19:1–13.
15. Marsh K, IJzerman M, Thokala P, et al. Multiple criteria decision analysis for health care decision making--emerging good practices: report 2 of the ISPOR MCDA emerging good practices task force. Value Health. 2016;19:125–137.
16. HIV Drug Resistance Database. Stanford University. 2019; Available at: https://hivdb.stanford.edu/. Accessed February 25, 2019.
17. HIV Drug Interactions: Interaction Checker. University of Liverpool. 2019; Available at: https://hiv-druginteractions.org/. Accessed February 25, 2019.
18. Cerrone MAO, Neary M, Marzinke M, et al. Rifampin effect on tenofovir alafenamide (TAF) plasma/intracellular pharmacokinetics. Paper presented at: Conference on Retroviruses and Opportunistic Infections (CROI); 2018; Boston, MA.
19. Detsky AS, Naglie G, Krahn MD, et al. Primer on medical decision analysis: Part 1—Getting started. Med Decis Making. 1997;17:123–125.
20. Sonnenberg FA, Beck JR. Markov models in medical decision making: a practical guide. Med Decis Making. 1993;13:322–338.
21. Mathematical and Computational Simulation of the Behavioral and Social Drivers of Population Health. Available at: https://www.ahrq.gov/professionals/education/curriculum-tools/population-health/orr.html. Accessed February 25, 2019.
22. First Look at the HRSA HIV Workforce Study: American Academy of HIV Medicine (AAHIVM); 2016.
23. Weddle A. HIV Clinic Capacity and Workforce Challenges: Results of a Survey of Ryan White Part C Programs. Paper presented at: National Summit on HIV Diagnosis, Prevention and Access to Care; Arlington, VA; 2008.
24. Landon BE, Wilson IB, McInnes K, et al. Physician specialization and the quality of care for human immunodeficiency virus infection. Arch Intern Med. 2005;165:1133–1139.
25. Cheng QJ, Engelage EM, Grogan TR, et al. Who provides primary care? An assessment of HIV patient and provider practices and preferences. J AIDS Clin Res. 2014;5:366.
26. Petterson SM, Liaw WR, Phillips RL Jr, et al. Projecting US primary care physician workforce needs: 2010–2025. Ann Fam Med. 2012;10:503–509.
27. Analysis of the NRMP-SMS Match for Infectious Disease for 2016-17 Appointment Year and Trends Over-time: Infectious Diseases Society of America (IDSA); 2016.
28. Adams J, Chacko K, Guiton G, et al. Training internal medicine residents in outpatient HIV care: a survey of program Directors. J Gen Intern Med. 2010;25:977–981.
Keywords:

HIV; antiretroviral therapy; clinical decision support tool; multiple-criteria decision analysis

Supplemental Digital Content

Copyright © 2019 Wolters Kluwer Health, Inc. All rights reserved.