Patients’ and Nephrologists’ Evaluation of Patient-Facing Smartphone Apps for CKD : Clinical Journal of the American Society of Nephrology

Journal Logo

Original Articles: Chronic Kidney Disease

Patients’ and Nephrologists’ Evaluation of Patient-Facing Smartphone Apps for CKD

Singh, Karandeep1,2,3; Diamantidis, Clarissa J.4,5; Ramani, Shreyas3; Bhavsar, Nrupen A.4; Mara, Peter2; Warner, Julia2; Rodriguez, Jorge6; Wang, Tianshi3; Wright-Nunes, Julie2

Author Information
Clinical Journal of the American Society of Nephrology 14(4):p 523-529, April 2019. | DOI: 10.2215/CJN.10370818
  • Open
  • Infographic
  • SDC

Abstract

Introduction

Patient-centered care is at the forefront of CKD management. Nearly every aspect of CKD management relies heavily on patient self-management in collaboration with providers, including medication and dietary adherence, self-monitoring of BP, and daily physical activity. Patient education and self-monitoring have historically relied on face-to-face visits with physicians, handouts, and written diaries. There is growing evidence that incorporating technology, specifically smartphone-based applications (apps), can help support self-management in CKD and chronic disease more generally (1–3). Randomized, controlled trials support the use of mobile health apps in the self-management of diabetes and obesity, yet few studies have formally evaluated apps in the context of kidney disease (4,5). One study evaluating a smartphone-based self-management system in usual care for patients with CKD showed reductions in both systolic and diastolic BP in patients using the smartphone-based intervention, with 80% of patients self-monitoring their BP on a regular basis (3). Additionally, in this study, providers perceived that patients using the smartphone app were more empowered in their CKD care than patients who did not use it.

Patients are already using apps focused on their health. Over three quarters of Americans own smartphones, and smartphone ownership is rising among older adults (46%) and those with low household incomes (67%). The number of health-related mobile apps exceeds 318,000 (6,7). In a survey of United States adults conducted by Pew Research in 2013, 69% reported keeping track of at least one health indicator, such as weight, diet, exercise routine, or symptoms, through their phones (8). Individuals with chronic illnesses are more likely to track their health information and more likely to report that tracking has a direct effect on health compared with healthy adults.

Unfortunately, nephrologists face many challenges when guiding patients to apps appropriate for their CKD self-management needs. First, nephrologists may not be familiar with which apps to recommend and may not be engaged in supporting patients in the use of technology. Second, Pew Research also shows that patients and caregivers do not routinely interact with their providers using apps. Only 18% of individuals report that they share data that they are tracking through technology with their health care provider (8). Additionally problematic is that the apps described in the scientific literature are not usually available to patients on the app stores, which creates a large barrier to providing evidence-based care (9). Third, the majority of apps that track health information may not be safe. For example, some apps do not include alerts for patients when health parameters are dangerously out of range as in the case of extremely high BP (10). Fourth, clinician evaluations of app quality correlate poorly with those from nonclinicians and consumers more broadly, which leads to additional uncertainty about which apps providers should recommend (10). Two systematic reviews of more traditional patient-centered education and support interventions showed that most education/support interventions are heterogeneous and are not necessarily replicable, with clinical effect that is often equivocal (11,12). As such, it is critical that we explore any and all means to support patients’ learning about their CKD diagnosis and engagement in behaviors needed to stay healthy.

To date, there have been no systematic evaluations of CKD smartphone apps with respect to patient engagement, patient and provider evaluations of quality, usability, or safety. The purpose of this study is to identify and systematically evaluate patient-facing Apple iOS and Google Android smartphone apps across the spectrum of CKD care addressing these key areas. By including apps across the continuum of CKD care (predialysis, ESKD, and transplant), this study is designed to be an important step toward elucidating apps that can serve as additional resources for patients and providers to optimize patient support and kidney disease outcomes.

Materials and Methods

Identifying CKD Apps

We identified apps targeting patients with CKD by conducting a search of the US Apple App Store and Google Play Store. We searched for the following four phrases on both app stores: “kidney disease,” “renal,” “dialysis,” and “kidney transplant.” We considered the first 50 apps for each search term on each app store. The search was conducted on November 21, 2016.

Selecting Apps for Review

We removed duplicates when a single app was identified through multiple search terms. We then excluded apps that were not related to kidney disease, were not patient facing, or had been last updated before 2014. We considered apps to be patient facing if the description of the app identified patients as a potential target audience (even if the app also had a clinician-facing component). Both the iOS and Android operating systems underwent significant interface changes in late 2013 (iOS 7 and Android 4.4). We, therefore, chose to remove apps updated before 2014 to eliminate stagnant apps, consistent with prior research in this area (10).

Evaluating Apps

An assessment form was designed and user tested to collect quantitative data on each app in the areas of clinical focus, patient engagement, quality, safety, and cost. We adapted a previously described framework for assessment of mobile health apps to account for kidney disease–specific content areas (10,13). Each app was reviewed by a nephrologist (K.S., C.J.D., and J.W.-N.), an experienced patient with kidney disease (P.M. and J.W.), and a research staff member (S.R., N.A.B., J.R., and Srishti Agrawal). Apps available for both iOS and Android were reviewed on either device. Either iPhones or iPads were used for iOS reviews, and a second generation Google Nexus 7 was used for Android reviews.

Apps were reviewed in their initial downloaded form without making in-app purchases. In-app purchases consist of additional features that can be purchased from within the app. In addition to using the app, reviewers gathered information from the app store pages on the iOS and Android stores and from the app’s website. If these sources did not provide evidence for a particular feature, we assumed that the feature was absent. App evaluations were carried out between January 23, 2017 and August 22, 2017.

The complete evaluation forms are available here: https://kdpsingh.lab.medicine.umich.edu/datasets/ckd-apps-dataset. We ascertained information from apps related to the following areas (text in parentheses indicates which members of the research team evaluated each aspect).

General App-Related Information (Staff).

We ascertained app store ratings, type of developer (individual, nonprofit, for profit, or government agency), app cost, in-app advertising, privacy policy, whether a login was required to use the app, and connectivity with BP monitors and weight scales using Bluetooth technology.

General Medical Information (Nephrologist).

We determined the kidney disease population being targeted (CKD, hemodialysis, peritoneal dialysis, or kidney transplant recipient) and whether the app referenced scientific literature or clinical guidelines (e.g., Kidney Disease Outcomes Quality Initiative and Kidney Disease Improving Global Outcomes).

Patient Engagement (Areas 1–4: Patient and Nephrologist; Areas 5–7: Staff).

We evaluated the intended audience for the app (adults with CKD, children with CKD, caregivers, providers, or the general public) and how useful the app was for engaging patients in one of seven areas: (1) providing educational information; (2) reminding or alerting patients; (3) tracking, displaying, and summarizing health information; (4) providing guidance on the basis of information entered by the patients; (5) enabling communication of health information with others; (6) obtaining support from peers through social networks; or (7) providing rewards to motivate and support behavior change. For areas 1–4, we asked questions about specific CKD-related content areas. For instance, 19 questions assessed education related to CKD-specific dietary recommendations, hypertension, electrolyte abnormalities, and others (full list is in Supplemental Table 1). We considered functionality to be present for areas 1–4 only if patients and nephrologists agreed.

Quality (Patient and Nephrologist).

We assessed the quality of the app through a single question with slight variations for patients and nephrologists: “How likely is it that you would recommend this app to (another patient/your patient)?” This is derived from the net promoter score question, which was originally formulated as a way of measuring customer satisfaction or loyalty as it relates to a company or a product (14,15). The net promoter score is widely used by Fortune 1000 companies and has been used as a measure of health app quality in prior work (10,16).

Usability (Patient).

We asked patient reviewers to complete a system usability scale, which is a validated tool to evaluate usability (17).

Safety (Nephrologist).

For apps that track health information, we asked nephrologists to enter extremely abnormal values (e.g., extremely high BP) that would be indicative of danger to a patient’s health. We considered apps to be unsafe if they did not respond at all to entry of this information.

Statistical Analyses

We used descriptive summary statistics to evaluate the results. We evaluated relationships between clinician ratings, app store ratings, number of app store ratings, usability, and patient ratings using a Spearman rank correlation test. All analyses were performed in R 3.4.1 (R, Vienna, Austria). The study data are available here: https://kdpsingh.lab.medicine.umich.edu/datasets/ckd-apps-dataset.

Results

Overall, our search strategy identified 174 unique apps on Android and 165 unique apps on iOS (Figure 1). After excluding apps that were not related to kidney disease, were not patient facing, were last updated before 2014, or could not be downloaded, five multiplatform apps, 12 Android-only apps, and 11 iOS-only apps remained (Tables 1 and 2). The majority of iOS apps were not rated by consumers, and the largest group of Android apps was rated between four and five of five stars (Table 1). Two apps offered in-app purchases. Individual developers created the majority of the apps, and the majority of apps were free to download. The largest group of apps was aimed at nondialysis, nontransplant CKD care and intended for use primarily by adults or their caregivers (Table 3).

fig1
Figure 1.:
We identified 200 Android and 178 iOS apps using 4 search terms and narrowed this to 12 Android apps, 11 iOS apps, and 5 dual-platform apps after applying inclusion/exclusion criteria. Flowchart of how applications were selected for final evaluation by patients and providers.
Table 1. - Characteristics of applications evaluated in the study
Characteristic Number of Applications Percentage or Range
Platform
 iOS and Android 5 18%
 iOS only 11 39%
 Android only 12 43%
Application store ratings
 iOS
  No rating 12 75%
  0–1
  >1–2
  >2–3 1 6%
  >3–4 1 6%
  >4–5 2 13%
 Android
  No rating 2 12%
  0–1
  >1–2
  >2–3 2 12%
  >3–4 6 35%
  >4–5 7 41%
Median number of ratings
 iOS 0 0–204
 Android 2 0–38
Number of downloads (Android only)
 10–49 1 6%
 50–99 1 6%
 100–499 4 24%
 500–999 4 24%
 1000–4999 5 29%
 5000–9999 1 6%
 10,000 or more 1 6%
Type of developer
 Individual 17 61%
 For-profit organization 7 25%
 Nonprofit organization 4 14%
Price to download
 Free 21 75%
 $0.99 2 7%
 $1.99 2 7%
 $2.99 or more 3 10%

Table 2. - All 28 applications evaluated in this study
Application Name Developer
C.A.P.D. Helper eeighon
Dialysis Finder Renalsite
Dialysis Maps a Pablo Perez
DialysisFinder a MaGIC-X
Diet a pmbsdung
Health Rank a StatWhiz Ventures LLC
How to Prevent Kidney Stones: Diet Guide and Healthy Recipes, CKD a WindyApp Studio
Kidney Conditions & Treatment Dimple Shah
Kidney Disease 101: Kidney Stones, Healthy Diet and CKD WhaleParadise Labs
Kidney Disease & Symptoms a thadanporn boonmarkmee
Kidney Disease Assistant a Deerane Tech.
Kidney Diseases Facts: Urology Health Eval Tips Tool, Simulations Guide and Behavior with Renal & Dialysis Glossary Review! Michael Quach
Kidney Disorders a Dilip J Bhagora
Kidney Failure Information a thadanporn boonmarkmee
Kidney Health a Smartphone Software Development, LLC
Kidney Stones Guide—How to Prevent & Treat Kidney Stones Symptoms a nipon phuhoi
My Dialysis Dictionary Pro DailyHemo
My Kidneys, My Health Handbook Digital Noir
Our Journey with Peritoneal Dialysis Net-Craft.com Inc.
Polycystic Kidney Information a thaweepong kongkratin
Potassium Counter Plus 250 Heart Healthy Foods First Line Medical Communications Ltd
Record Weight for Dialysis Patients Apollonious
Renal Disease Kidney Diet Tips SendGroupSMS.com Bulk SMS Software
Renal Function Calculator oxdb.net
Renal Kidney Failure Index Cal a Droid Blaster
Score Index for Haemodialysis a AndroidHub
Syck Anouk Stein, M.D.
Transplant Hero—Medication Reminder for Transplant Patients Transplant Hero LLC
aApplication is no longer available for download as of January 20, 2019.

Table 3. - Target population for applications
Target Population Number of Applications Percentage (of 28) a
Which CKD populations?
 CKD (nondialysis, nontransplant) 11 39
 Hemodialysis 6 21
 Peritoneal dialysis 6 21
 Kidney transplant recipient 2 7
 Others (e.g., nephrolithiasis) 8 29
Which type of patient is the application targeting?
 Adults with kidney disease 24 86
 Children with kidney disease 9 32
 Clinicians 8 29
 Caregivers 19 68
 General public (e.g., screening or prevention applications) 15 54
aThese are not mutually exclusive categories, and therefore, the percentages may exceed 100%.

Patient Engagement

We found 15 apps that educate patients, one that reminds or alerts patients, seven that track health information, one that guides patients in response to tracked information, four that enable sharing of health information, five that connect patients to peers via social networks, and two that reward patients for positive health behaviors. We identified seven apps that were deemed to be high performing (greater than or equal to four of five) by both nephrologists and patients on at least one functionality related to how the app engaged patients (Table 4). The high-performing apps covered the entire spectrum of CKD, including two apps focused primarily on earlier stages of CKD (Kidney Disease Assistant and My Kidneys, My Health Handbook), one app focused on hemodialysis (Record Weight of Patients on Dialysis), two apps focused on peritoneal dialysis (C.A.P.D. Helper and Our Journey with Peritoneal Dialysis), and two apps focused on kidney transplant recipients (Syck and Transplant Hero).

Table 4. - Applications rated greater than or equal to four of five on functionality by patients and nephrologists in any kidney disease content area
Application Name (Developer) Platform Useful for Functionality
C.A.P.D. Helper (eeighon) Android only Tracking BP over time
Tracking weight over time
Keeping an up-to-date list of general medications
Keeping an up-to-date list of dialysis-related medications
Kidney Disease Assistant (Deerane Tech.) iOS only Tracking creatinine or GFR over time
Tracking proteinuria over time
Tracking BP over time
My Kidneys, My Health Handbook (Digital Noir) Android and iOS Education about CKD
Education about hypertension
Education about diabetes and/or glucose control
Education about heart disease
Sharing information via text messaging/email/social media
Our Journey with Peritoneal Dialysis (Net-Craft.com Inc.) Android and iOS Education about peritoneal dialysis
Record Weight for Dialysis Patients (Apollonious) iOS only Tracking and summarizing weight
Syck (Anouk Stein, M.D.) iOS only Education about kidney transplant
Connecting patients with other patients
Transplant Hero—Medication Reminder for Transplant Patients (Transplant Hero LLC) Android and iOS Keeping an up-to-date list of transplant medications
Reminding patients to take medications

Quality

Patient ratings of app quality as assessed by the net promoter score question were not correlated with nephrologist ratings (r=0.36; P=0.06). Patient ratings of quality did not correlate with consumer ratings (r=0.34; P=0.18) or the number of consumer ratings (r=−0.10; P=0.61) as shown in Supplemental Figure 1.

Four apps were rated greater than or equal to eight (of ten) on the net promoter score question by patients (How to Prevent Kidney Stones, Transplant Hero, Dialysis Maps, and Dialysis Dictionary). Of these, only one was rated greater than or equal to eight by a nephrologist (Transplant Hero).

Usability

Usability was moderately correlated with patients’ overall rating of the app’s quality (r=0.46; P=0.01) as shown in Supplemental Figure 2.

Safety

Of the seven apps focused on tracking health information, only two (C.A.P.D. Helper and Score Index for Haemodialysis) generated an alert or some form of response on entry of extremely abnormal values that may be considered dangerous or life threatening.

Discussion

We used an established app evaluation framework, adapted with a CKD focus, to assess the patient engagement, quality, usability, and safety of apps supporting self-management of patients with CKD (10). Our approach used multiple stakeholders, including patients with kidney disease, nephrologists, and staff with expertise in information technology, to evaluate different aspects of each app. We found that most of the 28 evaluated apps were designed by individuals, were free to download, targeted adult patients with CKD not on dialysis and their caregivers, and focused on patient education. Patients, nephrologists, and consumers generally did not agree on app quality, potentially reflecting competing perspectives on what makes an app useful. Patient usability was related to app quality, affirming the value of usability assessments of apps. We identified safety as a major concern, because only two of the seven apps with tracking functionality alerted users on entry of extremely abnormal values.

Whereas prior research has shown physician and consumer ratings of apps to be poorly correlated (10), this is the first study to show that patient ratings of apps correlate poorly with both physician ratings and consumer ratings. This is not surprising, because focus groups show that patients with CKD and nephrologists often disagree on which aspects of their care are most important (18–21). Disagreement on the quality of the apps may similarly reflect competing priorities for patients and providers. This has important implications, because national efforts to evaluate apps—led in part by the American Medical Association—have focused on the content quality and security rather than patient perceptions of usefulness (22). This is also relevant in CKD, where prior efforts to evaluate apps solely on the basis of content alone led to 67 apps being recommended, none of which were considered high quality by patients or providers in our study and only one of which was rated as having high patient engagement in at least one area (My Kidneys, My Health Handbook) (23). Some commercial efforts to evaluate health apps have included a patient-derived score, but one popular app review service only included a single app in the “kidney disease” category (24). Incorporating patient input into evaluation of apps may complement efforts to assess content as physicians and medical professional societies consider which self-management tools to recommend to patients with CKD.

Usability and user satisfaction are among the most common end points measured in the scientific literature, and this study affirms their importance in the assessment of apps (9). We found usability to be closely tied to patients’ overall assessment of app quality. This is not entirely surprising, because the first question on the system usability scale (“I think that I would like to use this system frequently”) has some similarities to the net promoter score question used to assess quality (“How likely is it that you would recommend this app to another patient?”). Given that usability is commonly measured and reported in the literature (25–27), it has the potential to serve as a surrogate marker for app quality as long as it is assessed by patients.

Safety has previously been identified as an area of concern for mobile health apps more generally (10), and our work largely confirms this to be true in CKD. Patients using apps to track important health measures may not always know which values are dangerous or require additional attention. When apps do not respond when patients enter abnormal values, this has the potential to severely erode the trust that providers may place in apps. Coupled with the fact that most of the apps that we evaluated were developed by individuals rather than well known organizations, physicians are unlikely to implicitly trust the quality of apps, which likely is in part responsible for the low rates of adoption of some of these apps. Although our initial search did capture several apps developed by the National Kidney Foundation, these were excluded either due to lack of updates or in one case, due to a reviewer being unable to download the app.

Our study has several limitations. First, it was not designed to comprehensively review all apps for kidney disease but rather, was designed to evaluate those apps most likely to have high effect on patients. In particular, our search strategy was not designed to identify apps solely focused on complications of CKD (e.g., hypertension) if the app description did not explicitly mention CKD. Second, the evaluative team was small and may not be reflective of all stakeholders who may interface with patients with CKD. Our methods included both patients and physicians in the review process, offering key perspectives from two of the most critical stakeholders involved in patient-centered CKD management. Third, our app search was conducted in late 2016, after which new apps have entered the market, and some have been removed from the app store. We have made our study data available to facilitate ongoing evaluation of apps in the face of these changes. Lastly, the study team was composed of individuals who likely have a higher degree of health literacy (i.e., patients who are considered advocates willing to participate in this research study) and e-literacy (i.e., patients and providers in this study with interest in apps evaluation) than the general population.

Despite the limitations, we believe that our study has important implications. Although the apps that we evaluated are not necessarily a representative sample of all CKD apps, the fact that the majority focused on predialysis CKD is encouraging given that the majority of Americans with CKD—approximately 30 million people—are not on dialysis. Focusing patient-facing tools on the early stages of kidney disease affords a potential to prevent progression to ESKD. We identified seven apps with at least one high-quality area of patient engagement as determined by both patients and nephrologists—these apps span the entire spectrum of kidney disease and may serve as a template for new apps targeting patients with CKD. There are a number of engagement areas for which we found no apps at all or no high-quality apps, suggesting ample opportunities for patient-facing engagement efforts through apps. One particularly surprising finding is that the majority of apps evaluated focus primarily on patient education rather than tracking of health information, which differs from prior findings in CKD (10). This may be explained by the fact that our study evaluated more CKD apps overall compared with prior work, thus casting a wider net for apps that is more representative of the current marketplace.

In conclusion, only a small subset of CKD apps is highly rated by both patients and nephrologists, and patients’ impressions of app quality are not directly linked to consumer app ratings or nephrologist impressions.

Disclosures

K.S. reports other funding from Blue Cross Blue Shield of Michigan outside the submitted work. S.R. reports other funding from Caravan Health during the conduct of the study. J.W.-N. serves as a nonpaid member of the board of trustees for the American Kidney Fund and intermittently advises work with the National Kidney Disease Education Program.

C.J.D., N.A.B., P.M., J.W., J.R., and T.W. have nothing to disclose.

Published online ahead of print. Publication date available at www.cjasn.org.

See related Patient Voice, “Patient Perspective of Smartphone-Based Apps for CKD Self-Care,” and editorial, “Got CKD? There's an App for That!,” on pages and , respectively.

Acknowledgments

We thank Srishti Agrawal for her assistance with reviews.

K.S. received support from grant 5K12 DK111011 from the National Institute of Diabetes and Digestive and Kidney Diseases. C.J.D. received support from grants K23 DK099385 and R01 DK93938 from the National Institute of Diabetes and Digestive and Kidney Diseases. J.W.-N. received support from grants K23 DK097183 and R01 DK115844 from the National Institute of Diabetes and Digestive and Kidney Diseases.

The funding source had no role in the study design, conduct, analysis, or decision to submit the manuscript.

Supplemental Material

This article contains the following supplemental material online at http://cjasn.asnjournals.org/lookup/suppl/doi:10.2215/CJN.10370818/-/DCSupplemental.

Supplemental Figure 1. Correlation of evaluations (A) between nephrologists and patients, (B) between consumers and patients, and (C) between application store popularity and patients.

Supplemental Figure 2. Correlation of usability as assessed by patients and overall patient recommendation.

Supplemental Table 1. The κ and percentage agreement for application functionality (present or absent) as assessed by patient and nephrologist reviewers.

References

1. Aitken M, Gauntlet C: Patient Apps for Improved Healthcare: From Novelty to Mainstream, Parsippany, NJ, IMS Institute for Healthcare Informatics, 2013, pp 1–65
2. IMS Institute for Healthcare Informatics: Patient Adoption of mHealth, Parsippany, NJ, IMS Institute for Healthcare Informatics, 2015, pp 1–63
3. Ong SW, Jassal SV, Miller JA, Porter EC, Cafazzo JA, Seto E, Thorpe KE, Logan AG: Integrating a smartphone-based self-management system into usual care of advanced CKD. Clin J Am Soc Nephrol 11: 1054–1062, 201627173169
4. Quinn CC, Clough SS, Minor JM, Lender D, Okafor MC, Gruber-Baldini A: WellDoc mobile diabetes management randomized controlled trial: Change in clinical and behavioral outcomes and patient and physician satisfaction. Diabetes Technol Ther 10: 160–168, 200818473689
5. Carter MC, Burley VJ, Nykjaer C, Cade JE: Adherence to a smartphone application for weight loss compared to website and paper diary: Pilot randomized controlled trial. J Med Internet Res 15: e32, 201323587561
6. Pew Research Center: Demographics of Mobile Device Ownership and Adoption in the United States, 2017. Available at: http://www.pewinternet.org/fact-sheet/mobile/. Accessed January 29, 2017
7. IQVIA: The Growing Value of Digital Health, 2017. Available at: https://www.iqvia.com/-/media/iqvia/pdfs/institute-reports/the-growing-value-of-digital-health.pdf. Accessed August 30, 2018
8. Pew Research Center: Tracking for Health, 2013. Available at: http://www.pewinternet.org/2013/01/28/tracking-for-health/. Accessed January 6, 2016
9. Singh K, Drouin K, Newmark LP, Filkins M, Silvers E, Bain PA, Zulman DM, Lee J-H, Rozenblum R, Pabo E, Landman A, Klinger EV, Bates DW: Patient-facing mobile apps to treat high-need, high-cost populations: A scoping review. JMIR Mhealth Uhealth 4: e136, 201627993761
10. Singh K, Drouin K, Newmark LP, Lee J, Faxvaag A, Rozenblum R, Pabo EA, Landman A, Klinger E, Bates DW: Many mobile health apps target high-need, high-cost populations, but gaps remain. Health Aff (Millwood) 35: 2310–2318, 201627920321
11. Mason J, Khunti K, Stone M, Farooqi A, Carr S: Educational interventions in kidney disease care: A systematic review of randomized trials. Am J Kidney Dis 51: 933–951, 200818440681
12. Lopez-Vargas PA, Tong A, Howell M, Craig JC: Educational interventions for patients with CKD: A systematic review. Am J Kidney Dis 68: 353–370, 201627020884
13. Singh K, Drouin K, Newmark LP, Rozenblum R, Lee J, Landman A, Pabo E, Klinger EV, Bates DW: Developing a framework for evaluating the patient engagement, quality, and safety of mobile health applications. Issue Brief (Commonw Fund) 5: 1–11, 201626934758
14. NICE Satmetrix: The Net Promoter Score and System, 2012. Available at: http://www.netpromoter.com/why-net-promoter/know. Accessed September 21, 2015
15. Kinney WC: A simple and valuable approach for measuring customer satisfaction. Otolaryngol Head Neck Surg 133: 169–172, 200516087007
16. Bloomberg: The Inventor of Customer Satisfaction Surveys Is Sick of Them Too, 2016. Available at: https://www.bloomberg.com/news/articles/2016-05-04/tasty-taco-helpful-hygienist-are-all-those-surveys-of-any-use. Accessed December 2, 2018
17. Brooke J: SUS: A retrospective. J Usability Stud 8: 29–40, 2013
18. Tiu H, Fagerlin A, Roney M, Kerr E, Ojo A, Rothman E, Nunes JW: Provider perspectives on chronic kidney disease diagnosis delivery. Clin Nephrol 89: 205–213, 201829057734
19. Rubin HR, Jenckes M, Fink NE, Meyer K, Wu AW, Bass EB, Levin N, Powe NR: Patient’s view of dialysis care: Development of a taxonomy and rating of importance of different aspects of care. CHOICE study. Choices for healthy outcomes in caring for ESRD. Am J Kidney Dis 30: 793–801, 19979398123
    20. Song MK, Lin FC, Gilet CA, Arnold RM, Bridgman JC, Ward SE: Patient perspectives on informed decision-making surrounding dialysis initiation. Nephrol Dial Transplant 28: 2815–2823, 2013
      21. Lewis AL, Stabler KA, Welch JL: Perceived informational needs, problems, or concerns among patients with stage 4 chronic kidney disease. Nephrol Nurs J 37: 143–148, 201020462074
      22. The Health app guidelines—Xcertia, 2018. Available at: http://xcertia.org/the-guidelines/. Accessed July 12, 2018
      23. Lee Y-L, Cui Y-Y, Tu M-H, Chen Y-C, Chang P: Mobile health to maintain continuity of patient-centered care for chronic kidney disease: Content analysis of apps. JMIR Mhealth Uhealth 6: e10173, 201829678805
      24. AppScript: AppScript | Discover, Deliver & Track Digital Health, 2014. Available at: https://www.appscript.net/. Accessed July 13, 2018
      25. Diamantidis CJ, Ginsberg JS, Yoffe M, Lucas L, Prakash D, Aggarwal S, Fink W, Becker S, Fink JC: Remote usability testing and satisfaction with a mobile health medication inquiry system in CKD. Clin J Am Soc Nephrol 10: 1364–1370, 201526220816
      26. Diamantidis CJ, Zuckerman M, Fink W, Aggarwal S, Prakash D, Fink JC: Usability testing and acceptance of an electronic medication inquiry system for CKD patients. Am J Kidney Dis 61: 644–646, 201323157933
        27. De Silva L, Diamantidis CJ, Prakash D, Zuckerman M, Weir MR: Usability of mobile technology to screen for drug-drug interactions in kidney transplant patients. Am J Nephrol 40: 97–104, 201425059671
        Keywords:

        mobile health; chronic kidney disease; end stage kidney disease; Smartphone; Self Care; Methyltestosterone; Malus; kidney transplantation; Patient Participation; blood pressure; dialysis; Mobile Applications; kidney; Renal Insufficiency, Chronic; Chronic Disease; Telemedicine; Exercise

        Copyright © 2019 by the American Society of Nephrology