Secondary Logo

Journal Logo

Primers in Clinical and Translational Research

Adventures in Developing an App for Covert Hepatic Encephalopathy

Bajaj, Jasmohan S1

Author Information
Clinical and Translational Gastroenterology: April 2017 - Volume 8 - Issue 4 - p e85
doi: 10.1038/ctg.2017.14
  • Free



Chronic complex conditions have long been major causes of morbidity and mortality in the developed world and with increasing affluence are increasing in their prevalence worldwide ( Physical and psycho-social impairments conspire to worsen the prognosis.1 However, treatments focus on the medical perspective, rather than taking a holistic view of the patient experiences or patient-reported outcomes (PROs).2 The quantification of these PROs has been started by the NIH PROMIS group ( for the general population, but this may not apply to specialized diseases. One of the major chronic diseases is cirrhosis, which represent the end-stage of liver fibrosis. Most cirrhotic patients have co-morbid conditions that worsen the disease progression, or are direct complications of the disease process itself. Temporal trends in cirrhosis management have shown an improvement in overall survival but PRO analyses continue to show an immense psycho-social burden.3, 4 Although complications such as variceal bleeding are increasingly being controlled, hepatic encephalopathy (HE) remains an important issue. The diagnostic and treatment strategies for the overt form are generally well-outlined and agreed upon.5 However, it is the silent epidemic of covert hepatic encephalopathy (CHE) found in the majority of tested cirrhotic patients, which is an unmet need (Table 1). CHE is associated with a higher progression to OHE, results in hospitalizations and death and is perhaps the single biggest contributor to impaired PROs in cirrhosis.6 These include daily function, driving, socio-economic status as well as caregiver burden.5 Most clinicians and researchers agree that CHE is important but logistic concerns prevent them from regular testing.7 This is important because its treatment can improve PROs and medical outcomes.8 Treatments for CHE are not cost-effective without testing given the adverse events, adherence issues, and expense.9 Therefore treating every single cirrhotic patient for presumed CHE is not appropriate. Importantly, there is no laboratory or radiological covariate or physical sign that reliably points towards CHE similar to those recommended for other cirrhosis complications, i.e., hepatocellular cancer, esophageal varices etc. Part of the problem is the overabundance of testing strategies to diagnose CHE5 that have failed to “cross-over” into clinical practice.10 Therefore CHE represents an opportunity to increase the reach of tests that could benefit patients if applied as point-of-care. This example illustrates a condition where there is agreement that the condition is clinically relevant and logistic barriers prevent the adoption of this where it would help the maximum number of patients. Clinical situations similar to these are where newer methodology can have the most impact and should be sought out in other spheres of GI and Hepatology.

Table 1
Table 1:
Needs assessment using CHE as an example


Before embarking on a plan, it is important to determine whether the proposed testing strategy has a theoretical basis for use in the situation of interest. For CHE, we used the Stroop test due to several reasons. It has been used since the early 1930s to diagnose cognitive dysfunction.11 While a relatively simple paper-pencil test, its difficulty levels can be modulated towards higher and lower-functioning individuals. Cognitive domains engaged during Stroop testing are psychomotor speed, attention, and cognitive flexibility, most of which are also interrogated in the validated tests for CHE such as the Psychometric Hepatic Encephalopathy Score (PHES).12 Since the PHES is copyrighted, converting it into an electronic interface would have required multiple permissions as well as a large standardization sample similar to its paper-pencil version. Also in the US, several components of the PHES are copyrighted and cannot even be ordered by a non-psychologist. Therefore a testing strategy that studies similar domains but a different approach was chosen. There is also precedent for the study of Stroop in HE in prior studies using paper-pencil and computerized administrations with good outcomes.13 It was likely that that similar to most CHE tests, this would be a sensitive but not a specific test. Also it was recognized that patients with red-green color blindness would not be candidates and similar to prior reports, there would be correlation but not high concordance between different CHE testing strategies.14 Therefore for CHE, there was precedent that it would be helpful, which should be the case when defining tests/approaches for other conditions.


The smartphone and tablet revolution has truly modernized the patient–clinician interface.15 Also in order to advance the acceptance and knowledge of CHE in the younger generation of clinicians and researchers, a tool that runs on platforms familiar with current investigators is necessary.16 The need for such a tool was also informed by the author's interaction with several researchers and audience members, who bemoaned the lack of an App that could be administered and interpreted directly. A prior computerized non-App strategy, the inhibitory control test (ICT), was also studied by the authors extensively as an alternative to traditional testing.17 This was made available for free but requires highly functional patients and was ultimately difficult for many subjects with cirrhosis. Also the ICT was not in App form, making it difficult to apply within the clinic, which also informed our strategies. An overall analysis of new technology development steps are in Table 2.

Table 2
Table 2:
Steps for validating new technology


The first step is always to study available Apps in the market and test their face validity, agreement across several platforms and acceptability of this strategy to our patients. This is presumed to be challenging since most cirrhotic patients are >45 years and not as exposed to technology compared to people with other chronic diseases such as inflammatory bowel disease. We, however, had prior experience with ICT, which required familiarity with computers, so were confident that an App would work. For the first investigation, we chose to study not one but two gold standards, ICT and PHES, against a commercially available App.18 This App was meant to be a game, had not been validated and was not customized for CHE. The first investigation was to perform this in healthy controls and cirrhotic patients in a cross-sectional manner. We found good sensitivity, specificity, and patient acceptance, and surprisingly found that the time required rather than the accuracy determined the ultimate differentiation between CHE and no-CHE patients. Therefore the first step was completed.


Given the initial success of the Stroop in an App platform separating CHE and no-CHE patients, we had to create a custom-built App called the EncephalApp that focused on CHE.19 This was performed by enlisting an App development company and consulting regarding further optimization. In our case, we used Mobelux based in Richmond due to their reputation as well as their proximity to our campus ( The company and creators had several conferences to streamline the App. These included (1) creation of a script of instruction to be read verbatim at each point of the App (2) formalizing the practice runs and not allowing subjects to go onto the testing unless the practice runs were completed (3) reduce the inter-run time to make the App (4) make data entry and settings user-friendly and (5) to devise a novel method to directly transmit results via an Excel spreadsheet to any e-mail of the user's choosing. The total administration time was kept ranging between 3 and 5 min with five runs in the easy Off and relatively hard On state (Figure 1a and b). This was then beta-tested several times by at least three research coordinators for their input regarding administration ease and then administered to several healthy volunteers to get their initial feedback. After the initial kinks were removed, we were then ready to test this in a separate group of cirrhotic patients and controls.

Figure 1
Figure 1:
Presentation of the EncephalApp in the Off state (a) and On state (b) as presented to the user.


The next step was, in addition to testing the streamlined App, to evaluate it against clinically relevant outcomes that would encourage its use. For this again a fresh batch of cirrhotic subjects and age-matched healthy controls were recruited and also given the gold standard tests.19 Test/retest reliability and administration via phone vs. tablet were also measured. The results showed good test/retest reliability and equivalence regarding mode of administration. The App was also found to correlate with driving simulator performance and responded to underlying changes in patients' status with worsening after transjugular intra-hepatic porto-systemic shunting (TIPS) and improvement after hyponatremia correction. Age-based cut-offs were established but these needed further confirmation. The results confirmed prior experience that time rather than accuracy was the differentiator between CHE and no-CHE patients.


Further studies were needed with a larger sample size to predict the ultimate goal, development of OHE. This required four centers across the USA, with >800 new subjects.20 Of these there were 300 controls and rest were cirrhotic patients. The App was tested against PHES and ICT, and norms for all three modalities were created as a result for the USA. There was between site variability in the AUC for CHE detection using the App. However, the App results were able to predict the development of OHE within 6 months independent of MELD score and prior OHE status. Age, gender, and education-adjusted norms were also created. After these results the App was released for free on iOS since there was demonstration of its scientific validity in several hundred subjects. These results have prompted the translation of the App into several languages, including Mandarin, French, German, Arabic, Spanish, and Thai. The results of these initial experiences outside the USA are now coming out and will be presented in at the 17th ISHEN conference in India.


The first part of deciding what is abnormal is to define what is “normal”, which may not be as easy as it sounds.21 Norms for PHES, ICT, and EncephalApp Stroop, along with detailed methodology for administration and interpretation are on the website The App was then also released on Android platforms after the initial iOS version was well-received. The Android version has greatly increased the reach of the App. Although it is important to discuss the App and its results through educational conferences, ultimately the scientific performance is what helps the uptake. Till date, the EncephalApp has been studied in more >1,350 subjects published in four papers and has generated >10 scientific abstracts and presentations.18, 19, 20, 22 The field of CHE research is moving towards improving the reach of these tests with newer versions of EEG and use of quality-of-life instruments to help define this.23, 24 At present in our center, EncephalApp testing through medical assistants and nurses is provided as standard of care to eligible subjects. The time for interpretation and administration is reimbursable as allowed for cognitive impairment detection.


For any new technology or approach a detailed analysis between needs of the clinician, research, and patient population is needed. As Albert Einstein said “make things as simple as possible, but not simpler”. Therefore acceptability and user-friendliness of a new technique needs to be balanced with its ultimate utility. A tilt towards reducing time required may reduce its discriminating capability and vice-versa. Ultimately a balance between scientific evidence and user-friendliness will guide the uptake of any new technology or approach.


Guarantor of the article: Jasmohan S. Bajaj, MD, FACG.

Specific author contributions: Jasmohan Bajaj was responsible for all aspects of this publication.

Financial support: This was partly supported by VA Merit review CX10076 and NIH RO1DK089713 to JSB.

Potential competing interests: Jasmohan S. Bajaj holds the copyright to EncephalApp but it is freely available for download without financial attribution to JSB.

1. The World Health OrganizationInternational Classification of Impairments, Disabilities, and Handicaps. World Health Organization: Geneva, 1980.
2. Revicki DA, Erickson PA, Sloan JA et al. Interpreting and reporting results based on patient-reported outcomes. Value Health 2007; 10 (Suppl 2): S116–S124.
3. Kanwal F. Decreasing mortality in patients hospitalized with cirrhosis. Gastroenterology 2015; 148: 897–900.
4. Bajaj JS, Thacker LR, Wade JB et al. PROMIS computerised adaptive tests are dynamic instruments to measure health-related quality of life in patients with cirrhosis. Aliment Pharmacol Ther 2011; 34: 1123–1132.
5. Vilstrup H, Amodio P, Bajaj J et al. Hepatic encephalopathy in chronic liver disease: 2014. Practice Guideline by the American Association for the Study of Liver Diseases and the European Association for the Study of the Liver. Hepatology 2014; 60: 715–735.
6. Bajaj JS, Cordoba J, Mullen KD et al. Review article: the design of clinical trials in hepatic encephalopathy—an International Society for Hepatic Encephalopathy and Nitrogen Metabolism (ISHEN) consensus statement. Aliment Pharmacol Ther 2011; 33: 739–747.
7. Bajaj JS, Etemadian A, Hafeezullah M et al. Testing for minimal hepatic encephalopathy in the United States: an AASLD survey. Hepatology 2007; 45: 833–834.
8. Prasad S, Dhiman RK, Duseja A et al. Lactulose improves cognitive functions and health-related quality of life in patients with cirrhosis who have minimal hepatic encephalopathy. Hepatology 2007; 45: 549–559.
9. Bajaj JS, Pinkerton SD, Sanyal AJ et al. Diagnosis and treatment of minimal hepatic encephalopathy to prevent motor vehicle accidents: a cost-effectiveness analysis. Hepatology 2012; 55: 1164–1171.
10. Bajaj JS. Diagnosing minimal hepatic encephalopathy: from the ivory tower to the real world. Gastroenterology 2015; 149: 1330–1333.
11. Stroop JR. Studies of interference in serial verbal reactions. J Exp Psychol 1935; 18: 643–662.
12. Weissenborn K, Ennen JC, Schomerus H et al. Neuropsychological characterization of hepatic encephalopathy. J Hepatol 2001; 34: 768–773.
13. Felipo V, Ordono JF, Urios A et al. Patients with minimal hepatic encephalopathy show impaired mismatch negativity correlating with reduced performance in attention tests. Hepatology 2012; 55: 530–539.
14. Goldbecker A, Weissenborn K, Hamidi Shahrezaei G et al. Comparison of the most favoured methods for the diagnosis of hepatic encephalopathy in liver transplantation candidates. Gut 2013; 62: 1497–1504.
15. Spiegel B. 2015 American Journal of Gastroenterology lecture: how digital health will transform gastroenterology. Am J Gastroenterol 2016; 111: 624–630.
16. Rose S. Teaching old dogma with new tricks and technology: educational paradigm shifts in graduate medical education. Clin Transl Gastroenterol 2015; 6: e78.
17. Bajaj JS, Hafeezullah M, Franco J et al. Inhibitory control test for the diagnosis of minimal hepatic encephalopathy. Gastroenterology 2008; 135: 1591–1600 e1.
18. Bajaj JS, Thacker LR, Heuman DM et al. The Stroop smartphone application is a short and valid method to screen for minimal hepatic encephalopathy. Hepatology 2013; 58: 1122–1132.
19. Bajaj JS, Heuman DM, Sterling RK et al. Validation of EncephalApp, smartphone-based stroop test, for the diagnosis of covert hepatic encephalopathy. Clin Gastroenterol Hepatol 2015; 13: 1828–1835 e1.
20. Allampati S, Duarte-Rojo A, Thacker LR et al. Diagnosis of minimal hepatic encephalopathy using stroop EncephalApp: a multicenter US-based, norm-based study. Am J Gastroenterol 2016; 111: 78–86.
21. Montagnese S, De Rui M, Angeli P et al. Neuropsychiatric performance in patients with cirrhosis: who is "normal"? J Hepatol 2017; 66: 825–835.
22. Bajaj JS, Ahluwalia V, Steinberg JL et al. Elderly patients have an altered gut-brain axis regardless of the presence of cirrhosis. Sci Rep 2016; 6: 38481.
23. Nabi E, Thacker LR, Wade JB et al. Diagnosis of covert hepatic encephalopathy without specialized tests. Clin Gastroenterol Hepatol 2014; 12: 1384–1389 e2.
24. Schiff S, Casa M, Di Caro V et al. A low-cost, user-friendly electroencephalographic recording system for the assessment of hepatic encephalopathy. Hepatology 2016; 63: 1651–1659.
© 2017 The Author(s). Published by Wolters Kluwer Health, Inc. on behalf of The American College of Gastroenterology