Journal Logo

Individual QI Projects from Single Institutions

Application of Conjoint Analysis to Improve Reliability of Dietician Consultation in Pediatric Celiac Disease

Kulkarni, Sakil MD; Liss, Kim MD; Samson, Charles M. MD

Author Information
doi: 10.1097/pq9.0000000000000029



Celiac disease is an immune-mediated enteropathy triggered by gluten exposure in susceptible individuals. The childhood prevalence of celiac disease in the United States is about 1%.1 Multiple professional societies have developed guidelines for the diagnosis and management of children with celiac disease.1–3 The only current treatment for celiac disease is lifelong dietary gluten elimination. Due to its challenging nature, there is a significant rate of nonadherence to a gluten-free diet.4,5 Hence, guidelines suggest that ongoing collaboration between patients, parents, pediatric gastroenterologists, and dieticians is critical for the management of patients with pediatric onset celiac disease.1,6 However, these guidelines do not provide specific details on how to best achieve this collaboration and may be dependent upon institution-specific resources.

Conjoint analysis is a market-based research model that has been used extensively to predict consumer preferences.7 In recent years, conjoint analysis has been applied to determine health care providers’ and/or patients’ preferences, with the aim of optimizing resource allocation.8,9 The International Society for Pharmacoeconomics and Outcomes Research established a conjoint analysis task force with a goal of identifying good research practices for conjoint-analysis applications in health care.10 The task force presented its findings in a 10-item checklist including development of research questions, attributes/levels, construction of tasks, experimental design, preference elicitation, instrument design, data collection, statistical analysis, results, and study presentation.10

At baseline, there was a lack in uniformity of outpatient follow-up with either the pediatric gastroenterologist or dietician in newly diagnosed celiac disease patients at our institution. Hence, we used a conjoint analysis to determine attributes which pediatric gastroenterology (GI) providers deemed important in newly diagnosed celiac disease patients’ follow-up care. Based upon the identified attributes, a process change in follow-up care was initiated in an effort to improve reliability of dietary consultation in newly diagnosed celiac disease patients.


Provider Survey and Conjoint Analysis

A brief survey was administered to the 15 providers at our institution (12 physicians and 3 pediatric nurse practitioners) with the aim of identifying their practices when diagnosing and following up with newly diagnosed celiac disease patients. The questions asked were as follows: (1) When would you have newly diagnosed celiac disease patients follow up in clinic?, (2) Would you include a dietician in the first follow-up visit?, (3) How often do you see celiac disease patients in follow-up?, (4) What are the laboratory studies that you would check during the initial follow-up visit?, and (5) Do you repeat celiac serology during follow-up?

Based on this survey, we identified several attributes as being important from the perspective of the providers. These attributes were then used as the basis for a conjoint analysis. A 24-1 fractional factorial study design was used to generate 8 scenarios describing possible follow-up care plans for our conjoint analysis.11,12 This fractional factorial study design permitted analysis of 4 attributes and interactions between attributes while decreasing the number of scenarios required compared with a full factorial study design (for more information on fractional factorial study designs see Moen et al.11, pages 161–202). The 4 attributes studied (2 levels each) were (1) initial follow-up provider (celiac-specific or primary GI provider), (2) interval from diagnosis to follow-up (1 week or 4 weeks), (3) timing of dietary consult (concurrent with GI visit or separate), (4) on-going follow-up provider (celiac-specific or primary GI provider; see Figure, Supplemental Digital Content 1, At the time of the survey, there were no identified “celiac-specific” providers within our pediatric GI division. For other subspecialty GI disorders (inflammatory bowel disease, hepatology, short gut syndrome), disorder-specific providers had been identified at our institution. Although not included in the initial survey, this attribute was included to determine if identifying a “celiac-provider” would be considered as an important attribute at either the first and/or subsequent follow-up visits or if providers preferred to continue following this patient population themselves. The scenarios were presented to providers in a random order and they were asked to force rank scenarios according to their preferences, with rank order 1 being the most ideal follow-up care plan and rank order 8 being the least ideal method of managing a newly diagnosed patient with celiac disease. The different attributes (highlighted in red in Supplemental Digital Content 1, were present in all the scenarios. However, these attributes were not highlighted on the actual document given to the providers and hence were not obvious to the respondents.

Process Change Initiation

At baseline, there was no standardization of follow-up care in newly diagnosed celiac disease patients. In general, most patients were seen by the same provider who saw the patient before diagnosis (defined as primary GI provider). There was no standardization with regard to if and how a patient was seen by a dietitian after diagnosis. Based upon attributes identified through the conjoint analysis, a change of practice was initiated in July 2014 so that all newly diagnosed patients were to be seen at single clinic session held weekly where a dietician would also be present and could be seen concurrently with a GI provider. The primary GI providers could choose to either see their patients at this time (could be outside of their normal clinical session) or a nurse practitioner was available to see the patient if their primary GI provider preferred. Providers were initially informed about this change in practice via e-mail. Starting in July 2014, providers were reminded of this change at the time when one of their patients was diagnosed with celiac disease.

Retrospective Chart Review

After Washington University Human Research Protection Office approval, patients with celiac disease seen in the Washington University Pediatric Gastroenterology clinic between July 2013 and June 2015 were identified by searching for the celiac disease–specific International Classification of Diseases, Ninth Revision (ICD-9) billing diagnosis code (579.0).13

Patients with gastrointestinal complaints that may lead to a diagnosis of celiac disease (abdominal pain, diarrhea, and so on) or those asymptomatic patients at higher risk for celiac disease where serological screening is recommended (e.g., type 1 diabetes mellitus, trisomy 21, first-degree relatives) were evaluated by 1 of the 15 providers at our institution. Evaluation and treatment of subjects with celiac disease were carried out in accordance with the North American Society of Pediatric Gastroenterology, Hepatology and Nutrition guidelines for celiac disease.1 A retrospective chart review was performed to determine if the change in clinical process had affected dietary consultation. Data elements extracted from the electronic medical record included patient’s age at diagnosis, gender, race, symptoms at presentation, date of first office visit, date of endoscopy, date of first follow-up visit, and date of first dietary consultation. As patients with diabetes are seen by a dietician in endocrinology clinic, this represented a separate care process. Thus, these patients were excluded in the analysis.

Statistical Analysis

A conjoint analysis was performed using Study-It 2.0 software (McGraw-Hill, New York City, N.Y.). Effect sizes were determined by taking the difference between the mean rank orders of the 4 survey options when an attribute was at 1 level compared with 4 survey options when the attribute was at the other. Effect sizes and interactions between attributes were plotted on a dot diagram. Mean rank orders were determined for the 4 attributes and displayed on response plot. A geometric cube was used to further assess for interactions between the attributes.11

To determine if the change of follow-up care resulted in more timely dietary consultation, time from diagnosis to outpatient dietary consultation was plotted on an individual-control chart.14 The baseline mean was determined using the 20 points before July 2014 when the change in care process was initiated. To determine if the process change resulted in more reliable dietary consultation, a G-statistical control chart was used to show the number of consecutive newly diagnosed celiac disease subjects before a subject was never evaluated by a dietician.15 The baseline mean was determined by using all available data points before a special cause. Appropriate statistical process control chart rules were used to determine if statistical changes were observed.16


Baseline Characteristics

During the time period between July 2013 and June 2014 (excluding type I diabetes mellitus), 19 patients were diagnosed with celiac disease and 24 patients were diagnosed between July 2014 and June 2015. The diagnosis of celiac disease was established by abnormal duodenal histology. The most common symptom was abdominal pain (38%), followed by poor weight gain (20%). Twenty percentage of the patients were asymptomatic (Table 1).

Table 1.
Table 1.:
Patient Characteristics Before and After the Institution of Change in Practice

Practice Survey

The practice survey administered to the pediatric GI providers at our institution revealed that 20% (3/15) of providers preferred that the first follow-up visit be within 1 week of diagnosis with celiac disease and 47% (7/15) providers thought that the first follow-up visit should be within 2 weeks of diagnosis. The remaining 33% (5/15) were of the opinion that the first follow-up visit could be delayed to 4 weeks after diagnosis. The majority (87%; 13/15) of providers felt that the nature of first follow-up visit should be a concurrent visit with a provider and a dietician.

Conjoint Analysis

Following the practice survey, a conjoint analysis was performed to independently evaluate and prioritize follow-up care attributes. Response plots showed that the mean rank order was more favorable for a concurrent visit with the provider and the dietician (3.2), compared with separate visits with the above-mentioned providers (5.8). With regard to the timing of the first follow-up visit after celiac disease diagnosis, the mean rank order was more favorable for a follow-up visit within 1 week of diagnosis of celiac disease (3.2), compared with a visit within 4 weeks of diagnosis (5.8; Fig. 1). However, no notable change in the rank order was observed between the types of follow-up providers (celiac-specific provider versus the primary GI provider) either at the first or subsequent follow-up visits.

Fig. 1.
Fig. 1.:
Response plots showing mean rank orders for the 4 attributes (1 = most ideal, 8 = least ideal) at 2 levels: timing of dietician consultation (concurrent with GI follow-up versus separate), timing of initial follow-up (within 1 week of diagnosis versus 4 weeks of diagnosis), initial follow-up provider (primary GI provider versus celiac-specific provider), and subsequent follow-up provider (primary GI provider versus celiac-specific provider).

When effect sizes were plotted on a dot diagram (Fig. 2), it was evident that the effect size was higher for concurrent visits (point D; effect size, 2.56) and the time of the visit at 1 week after diagnosis (point T; effect size, 2.59). Neither provider type at initial (point P) or follow-up visit (Point F) nor interactions between attributes (points X1, X2, and X3) had any notable effect size. There were no appreciable differences in the conjoint analysis between physicians and nurse practitioners (data not shown).

Fig. 2.
Fig. 2.:
Dot diagram showing higher effect size for concurrent dietician consultation (D, effect size 2.56) and the timing of the visit at 1 week after diagnosis (T, effect size 2.59). Attributes with lower effect sizes were the type of follow-up provider during initial follow-up visit (P, effect size 0.5) or type of follow-up provider at subsequent follow-up visits (F, effect size 0.5). There were no identified interactions between the different variables tested (points X1, X2, and X3).

The change in mean rank orders when the time of first follow-up was decreased from 4 weeks to 1 week was independent of the type of the provider (celiac-specific provider or primary GI provider). Similarly, the change in mean rank orders when the first follow-up visit changed from a concurrent visit with the dietician and provider to separate visits was also independent of the type of follow-up provider. Differences in mean rank orders were not noticed when the type of follow-up provider changed from the primary GI provider to a celiac-specific provider when the other attributes were kept constant (Fig. 3).

Fig. 3.
Fig. 3.:
Mean rank order cube showing interactions between 3 attributes: initial follow-up provider [primary GI provider (PP) versus celiac-specific provider (CP)], timing of dietician consultation [concurrent with GI follow-up (C) versus separate (S)], and timing of visit (1 week versus 4 weeks after establishing diagnosis of celiac disease).

Improving Dietician Consultation Reliability

To determine the effects of the change in clinical follow-up process, time from celiac disease diagnosis to dietary consultation was plotted on an individual-control statistical process control chart (Fig. 4). A special cause (8 points below the center line) was identified on the individual-control chart beginning on December 8, 2014. The mean time from diagnosis to evaluation by a dietician decreased after the initiation of the quality improvement intervention from 30 days to 20 days (33% reduction). In addition, a special cause (a point above the upper control limit) was observed on December 4, 2014. In review of this patient, the diagnostic endoscopy occurred during a hospital admission and initial outpatient follow-up was scheduled within 4 days of discharge but before pathology results were available. Although pathology results were available for discussion at the first outpatient GI clinic visit, a dietician was not available. The patient saw the dietician during a subsequent outpatient encounter.

Fig. 4.
Fig. 4.:
Individual statistical process control chart showing days from celiac disease diagnosis to evaluation by a dietician. Baseline centerline was determined by the 20 data points before the process change. Two special causes were identified on the run chart: (1) a special cause was noted on December 4, 2014 due to a point above the upper control limit (demonstrated by green-circle), (2) a special cause starting on December 8, 2014 due to 8 consecutive points being below the centerline (demonstrated by red-circle), which resulted in a change in the centerline. The phase 2 centerline was determined using all data points on and after December 8, 2014.

A G-control statistical process control chart was used to assess reliability of dietary consultation. Each point on the G-control chart represents the number of consecutively diagnosed patients who were seen by a dietician before there was a patient who was diagnosed and was never seen by a dietician (Fig. 5). A special cause (a point above the upper control limit) was noted in February 2015, with 12 consecutive subjects being diagnosed with celiac disease before a subject was not seen by a dietician compared with the baseline of 1.9 subjects.

Fig. 5.
Fig. 5.:
G-statistical process chart showing the number of consecutive newly diagnosed celiac disease subjects before a subject was never evaluated by a dietician at our institution.


The results of our conjoint analysis identified that a shorter time from diagnosis to initial follow-up visit and having a concurrent visit with the provider and dietician were attributes which providers identified as more important in the care of newly diagnosed celiac disease patients. Based on these attributes, we redesigned the care process of newly diagnosed celiac disease patients at our institution. We observed improved reliability of dietary consultation, as demonstrated by an increase in the number of consecutively diagnosed celiac disease patients before there was a patient who missed an evaluation by a dietician. The process change also resulted in a decrease in the time from diagnosis to dietary consultation.

Conjoint analysis has been used with success in a variety of settings including market research, health care, and environmental evaluations.7–9,17,18 As our initial provider survey indicated several competing attributes, we used a conjoint analysis to influence the redesign of follow-up care for newly diagnosed celiac disease patients.

Various guidelines emphasize the importance of a dietician in the treatment of celiac disease without providing details on how to best utilize this resource.1,3,19 The American Dietetic Society has published guidelines for the treatment of celiac disease.20 An experienced dietician is important to counsel newly diagnosed celiac disease patients based on their dietary preferences and the latest guidelines about which foods are considered as gluten free. At our institution, we have only 1 such dietician (except for patients with diabetes mellitus) who is experienced in dealing with celiac disease and counseling on the gluten-free diet. To best utilize this resource, we standardized our care process so newly diagnosed patients were seen in 1 pediatric GI clinic session per week, concurrently with the dietician.

Standardization of care has been shown to reduce variations in health-care delivery and improve the quality of health care.21 Our initial survey identified variability in the care of celiac disease patients at our center. Based on the results of the conjoint analysis, we elucidated provider preferences’ how to best to utilize the 1 dietician with expertise in celiac disease. This formed the basis of a quality improvement initiative that improved the reliability of dietary consultation and decreased the variability in the care of celiac disease patients at our center. There are many other examples where standardization of care has led to identification of clinical pathways and along with quality improvement initiatives have contributed to similar results.22,23 A prominent example of this process on a large scale is the Diabetes Quality Improvement Project, which developed and implemented a comprehensive set of national measures for evaluation and quality improvement of diabetic care.23

One of the limitations of the current study is that this is a single-institution study. It is possible that the results of this conjoint analysis may change from 1 institution to another based on preferences of the providers and available resources at a particular institution. However, the principles behind our conjoint analysis can be applied to other institutions, although results may be different. The principles behind our study should also be translatable to other medical conditions. In addition, because only providers participated in the conjoint analysis, we were not able to evaluate patient and/or parent preferences. A separate conjoint analysis could be applied to determine if preferences are different between patients, parents, and providers. Although we did not observe a difference in responses between physicians and nurse practitioners, our sample size may have been insufficient to detect small differences. Our fractional factorial design only permitted for 4 attributes to be studied at 2 levels each. There are many other attributes and at other levels which could have been analyzed. However, we attempted to choose attributes based on our initial survey, which were identified as either important, could compete with other attributes, and/or were limited in availability. Finally, although reliability of consultation was improved, it is not known whether this improvement resulted in better clinical outcomes, dietary adherence, and/or improved patient/parent satisfaction. This could be further studied at a later time.

In conclusion, this study demonstrates how conjoint analysis was used to allocate available resources at our institution to improve care of patients with celiac disease. The QI initiatives instituted during this period are still ongoing with current efforts include the formation of a multidisciplinary celiac disease clinic, which will include a provider with a specific focus in celiac disease care, a nurse, dietician, psychologist, and social worker. The authors are of the opinion that similar approaches as demonstrated in our study could be used to allocate available resources available to improve patients’ care under a variety of different conditions and in a variety of different settings.


We would like to acknowledge Lori Holtz, MD, MSPH (St Louis, MO) for her critical review of this article.


The authors have no financial interest to declare in relation to the content of this article.


1. Hill ID, Dirks MH, Liptak GS, et al.; North American Society for Pediatric Gastroenterology, Hepatology and Nutrition. Guideline for the diagnosis and treatment of celiac disease in children: recommendations of the North American Society for Pediatric Gastroenterology, Hepatology and Nutrition. J Pediatr Gastroenterol Nutr. 2005;40:119.
2. Husby S, Koletzko S, Korponay-Szabó IR, et al.; ESPGHAN Working Group on Coeliac Disease Diagnosis; ESPGHAN Gastroenterology Committee; European Society for Pediatric Gastroenterology, Hepatology, and Nutrition. European Society for Pediatric Gastroenterology, Hepatology, and Nutrition guidelines for the diagnosis of coeliac disease. J Pediatr Gastroenterol Nutr. 2012;54:136160.
3. Snyder J, Butzner JD, DeFelice AR, et al. Evidence-informed expert recommendations for the management of celiac disease in children. Pediatrics. 2016;138:e20153147.
4. Jadresin O, Misak Z, Sanja K, et al. Compliance with gluten-free diet in children with coeliac disease. J Pediatr Gastroenterol Nutr. 2008;47:344348.
5. Kurppa K, Lauronen O, Collin P, et al. Factors associated with dietary adherence in celiac disease: a nationwide study. Digestion. 2012;86:309314.
6. Murch S, Jenkins H, Auth M, et al.; BSPGHAN. Joint BSPGHAN and Coeliac UK guidelines for the diagnosis and management of coeliac disease in children. Arch Dis Child. 2013;98:806811.
7. Mele NL. Conjoint analysis: using a market-based research model for healthcare decision making. Nurs Res. 2008;57:220224.
8. Bridges JF, Kinter ET, Kidane L, et al. Things are looking up since we started listening to patients: trends in the application of conjoint analysis in health 1982-2007. Patient. 2008;1:273282.
9. Taylor WJ. Pros and cons of conjoint analysis of discrete choice experiments to define classification and response criteria in rheumatology. Curr Opin Rheumatol. 2016;28:117121.
10. Bridges JF, Hauber AB, Marshall D, et al. Conjoint analysis applications in health—a checklist: a report of the ISPOR Good Research Practices for Conjoint Analysis Task Force. Value Health. 2011;14:403413.
11. Moen RD, Nolan TW, Provost LP. Quality Improvement through Planned Experimentation. 2012.3rd ed. New York, NY: McGraw Hill.
12. Ryan M, McIntosh E, Shackley P. Using conjoint analysis to elicit the views of health service users: an application to the patient health card. Health Expect. 1998;1:117129.
13. Ludvigsson JF, Pathak J, Murphy S, et al. Use of computerized algorithm to identify individuals in need of testing for celiac disease. J Am Med Inform Assoc. 2013;20(e2):e306e310.
14. Nelson FE, Hart MK, Hart RF. Application of control chart statistics to blood pressure measurement variability in the primary care setting. J Am Acad Nurse Pract. 1994;6:1728.
15. Benneyan JC. Number-between g-type statistical quality control charts for monitoring adverse events. Health Care Manag Sci. 2001;4:305318.
16. Nelson LS. Technical aids: the Shewart control chart—tests for special causes. J Qual Technol. 1984;16(4):237239.
17. Alriksson S, Oberg T. Conjoint analysis for environmental evaluation—a review of methods and applications. Environ Sci Pollut Res Int. 2008;15:244257.
18. Mertens L, Van Cauwenberg J, Ghekiere A, et al. Does the effect of micro-environmental factors on a street’s appeal for adults’ bicycle transport vary across different macro-environments? An experimental study. PLoS One. 2015;10:e0136715.
19. Kagnoff M. AGA Institute Medical Position Statement on the diagnosis and management of celiac disease. Gastroenterology. 2006;131(6):19771980.
20. Kupper C. Dietary guidelines and implementation for celiac disease. Gastroenterology. 2005;128(4 Suppl 1):S121S127.
21. Lavelle J, Schast A, Keren R. Standardizing care processes and improving quality using pathways and continuous quality improvement. Curr Treat Options Peds. 2015;1(4):347358.
22. Clemmer TP, Spuhler VJ, Oniki TA, et al. Results of a collaborative quality improvement program on outcomes and costs in a tertiary critical care unit. Crit Care Med. 1999;27:17681774.
23. Fleming BB, Greenfield S, Engelgau MM, et al. The diabetes quality improvement project: moving science into health policy to gain an edge on the diabetes epidemic. Diabetes Care. 2001;24:18151820.

Supplemental Digital Content

Copyright © 2017 the Author(s). Published by Wolters Kluwer Health, Inc.