What Is Known
- Definitions, outcome measures, and measuring instruments used in randomized controlled trials on infant gastroesophageal reflux disease vary widely.
- Core outcome sets ensure reporting consistency regardless of the primary purpose of a trial.
What Is New
- We developed a core outcome set for infant gastroesophageal reflux disease through an international Delphi study.
- This 9-item core outcome set consists of patient-reported outcomes, clinical outcomes, and adverse events and may have the potential to decrease study heterogeneity and ease comparability of results of future trials.
Gastroesophageal reflux (GER) is the (in)voluntary backflow of stomach contents into the esophagus and occurs physiologically on a daily basis in all infants (1). Gastroesophageal reflux disease (GERD) is defined as GER causing bothersome symptoms and/or complications (2,3).
Despite the importance of symptoms in the definition of GERD, therapeutic trials of medications and surgery for GERD fail to use a common set of outcomes to assess improvement in GERD symptoms (4). Furthermore, bothersome symptoms may differ depending on if the symptoms are assessed by health care providers, patients, or patient's parents (5). Because of this lack of standardization of outcomes, comparisons of therapeutic trials are difficult and the quality of evidence supporting therapeutic interventions is weak (6,7).
To inform clinical practice, future trials that aim to determine benefits and risks of interventions should measure outcomes that are considered to be important to patients and parents and useful to healthcare professionals (HCPs). A way to achieve this is by developing a “core outcome set” (COS) for clinical trials, which represents the agreed minimum set of outcomes that should be measured and reported in trials in a certain area (8). These COS have already been used successfully in other healthcare areas, such as the Outcome Measures in Rheumatology Initiative (OMERACT) (9,10). Patient-reported outcomes (PROs) are increasingly measured in current research. However, only 16% of the currently available COS involved patients and representatives during the COS development process, limiting any value PROs may add to a COS (10).
Currently, no such standardized set of outcome measures including PROs is available for infants with GERD. The aim of our study therefore was to develop a COS for infant GERD by adhering to the OMERACT recommendations and ensuring active involvement of both HCPs and patients and their representatives throughout the whole process. This COS has the potential to improve quality and comparability in future outcome trials on infant GERD, thereby advancing the usefulness of research to inform clinical practice and enhancing patient care.
A COS for infant GERD was developed by adhering to the stepwise approach as advised by the OMERACT 2.0 filter (6,7,9,10,12). The following steps were conducted: collecting important treatment outcomes (step 1) and creating the COS (step 2).
Step 1—Collecting Important Treatment Outcomes
HCPs visiting the 2016 European Society for Pediatric Gastroenterology, Hepatology, and Nutrition (ESPGHAN) annual meeting in Athens, Greece, were invited to participate in the survey. No restrictions were applied as to age, level of expertise of setting of care. HCPs from Australia and the United States of America were invited separately by e-mail to ensure a global representation of HCPs (12,13). Our aim was to include at least 100 HCPs, similar to earlier studies (Supplemental Fig. 1, Supplemental Digital Content, http://links.lww.com/MPG/B544) (12,14). An English-written questionnaire was handed out on paper and participants were asked to list up to 5 primary goals of therapy from their perspective and up to 5 persistent signs or symptoms that would signify inadequate treatment. Answers were processed anonymously. All HCPs were invited to indicate whether they would like to be involved in the further phases of the development of the COS. For further information, the original questionnaires are available from the corresponding author.
Next, parents/caregivers (further referred to as “parents”) from infants with GERD from Europe (Belgium, France, Italy, The Netherlands, and the United Kingdom), Australia, and the USA (n = 20 per country) were invited to participate in the survey. Diagnosis of GERD was made by the infant's treating physician, based on the definition as proposed by ESPGHAN/North American Society for Pediatric Gastroenterology, Hepatology, and Nutrition guidelines. Only infants who were otherwise healthy were included. Our aim was to include at least 140 parents, similar to earlier studies (Supplemental Fig. 1, Supplemental Digital Content, http://links.lww.com/MPG/B544) (12,14). An English-written questionnaire was provided by the infant's treating physician at the inpatient or outpatient clinic or by an independent researcher of our group by telephone after verbal informed consent. The original questionnaire was developed in English and presented to respondents in their native language. Parents were asked to list up 5 aspects of the treatment for GERD that would made them feel their child was being treated adequately/inadequately or that would made them feel comfortable/uncomfortable. Answers were translated back to English by the researchers that questioned the participants and were processed anonymously.
Step 2—Creating the Core Outcome Set
Step 2a—Creating a Short List
The second step in the development of a COS for infant GERD consisted of 4 phases. Listed outcomes by HCPs and parents were classified in predefined domains and grouped in core areas according to the OMERACT conceptual framework (10). To identify responses that were open to interpretation and to make categorization more accurate, responses were independently judged by 2 reviewers (M.M.J.S. and R.R.) and disagreements were adjudicated by discussion and consensus with a third-party arbiter (M.M.T.). When necessary, domains were added according to the outcomes reported by the HCPs and parents.
Reported outcomes were then combined when considered appropriate (by M.M.J.S., M.M.T., and R.R.). Grouped outcomes that were mentioned by >10% of the participants were selected to create 3 different shortlists: 2 based on the answers of HCPs in the inpatient and outpatient settings, respectively, and 1 based on the answers of the parents, regardless of setting (9,11).
Step 2b—Rating Outcomes on the Shortlist
HCPs that had previously agreed to participate in further development of the COS, and members of The Consensus Group on Outcome Measures Made in Paediatric Enteral Nutrition Clinical Trials (COMMENT) Working Group; the ESPGHAN; Neurogastroenterology Motility and Functional Gastrointestinal Disorders Working Group; the North American Society for Pediatric Gastroenterology, Hepatology, and Nutrition; and Australian HCPs with a special interest in pediatric gastroenterology, were invited via e-mail to participate in the second phase and fill out an online questionnaire. We aimed to have at least 50% of the HCPs involved in the first step of the study also involved in the second step. HCPs were asked to rate each outcome for the inpatient and outpatient setting separately on a scale ranging from 1 to 9 (1–3 signifying an outcome of limited importance, 4–6 being important but not critical, and 7–9 being critical) (8,15,16). Consensus to whether an outcome should be included in the COS was defined as 70% or more of the HCPs rating it as 7 to 9 and fewer than 15% rating it as 1 to 3. Consensus that an outcome should not be included in the COS was defined as 70% or more rating it as 1 to 3 and fewer than 15% rating it as 7 to 9 (8,15,16).
A new cohort of 140 parents of infants with GERD (countries similar to those in the first phase of the development process, n = 20 per country) was invited to participate (methods for recruiting and questionnaire-taking as above). Parents were asked to rank the outcomes on the shortlist on a scale of 0 to 4 (0 = “does not make me feel comfortable,” 4 = “makes me feel very comfortable”) and to prioritize these outcomes by selecting the 5 outcomes that made them feel the most comfortable (17).
Step 2c—Creating Top 5 Outcomes for HCPs and Parents
In this phase, the preliminary parental COS was created using the 5 outcomes with the highest rank. Methods for creating the HCPs preliminary COS were as described above. The 2 preliminary COS for parents and HCPs were then combined into 1. Last, following the OMERACT approach, adverse events were put forward to the preliminary COS.
Step 2d—Creating Final Set
In the first consensus meeting, the preliminary COS was discussed between the HCPs with expertise in the field of infant GERD and representatives of pediatric patients with GERD during the 8th European Pediatric Motility Conference in London, March 2018. The final COS was presented to the Neurogastroenterology Motility and Functional Gastrointestinal Disorders Working Group at the 51st ESPGHAN annual meeting in Geneva, May 2018. Consensus was defined as a unanimous decision from the expert panel.
Step 1—Identifying Important Treatment Outcomes
In total, 125 out of 165 HCPs (76%) from 33 different countries of origin completed the first questionnaire. Illegible outcomes were excluded (inpatient, n = 1; outpatient, n = 3). The listed outcomes “diagnostics” (outpatient setting, n = 2; inpatient setting, n = 3), “follow-up” (outpatient setting, n = 1) and “observation” (inpatient setting, n = 6) were also excluded as they were not considered to be appropriate parameters corresponding with measurable effects of therapeutic interventions for GERD. After exclusion, 59 different outcomes were reported for the outpatient setting and 57 for the inpatient setting (Supplemental Tables 1 and 2, Supplemental Digital Content, http://links.lww.com/MPG/B544).
Altogether, 139 out of 143 (97%) parents (66% mothers) from 7 different countries responded to the first questionnaire. Illegible outcomes (n = 3) were excluded and the listed outcomes “diagnostics” (n = 18), “follow-up” (n = 21), and “observation” (n = 7) for the same reason as described above. Listed outcomes were redirected into 1 direction (“adequately treated”). In total, 50 different treatment outcomes were reported (Supplemental Table 3, Supplemental Digital Content, http://links.lww.com/MPG/B544).
Step 2—Creating the Core Outcome Set
Step 2a–c—Creating and Rating Shortlists
Based on the outcomes mentioned by >10% of the HCPs, 2 shortlists (inpatient and outpatient setting) consisting of 9 and 7 outcome measures, respectively, were created (Table 1). These shortlists were rated and prioritized by 83 of the 139 approached HCPs (60%) from primary, secondary, and tertiary care centers originating from 29 different countries.
In total, 127 parents completed the second questionnaire and rated the outcomes on the shortlist. The shortlist resulting from the outcomes mentioned by >10% of parents is displayed in Table 2 and consists of 13 outcome measures. The 5 outcome measures on this shortlist with the highest rank were put forward to the parental preliminary COS.
Step 2d—Creating Final Core Outcome Set
The outcomes listed on the preliminary COS for the HCPs and parents were combined into 1 (Fig. 1—step 1). Figure 1 (steps 1 and 2 a–c) describes the creation of the final COS during the first consensus meeting in London, based on the preliminary COS for the HCPs and parents. Eventually, the final COS consisted of 9 items and was presented during the consensus meeting in Geneva, which consisted of general pediatricians, neonatologists, and mostly pediatric gastroenterologists from European countries, and the USA and Australia (Fig. 1—step 3).
In this study, we developed a 9-item COS for infant GERD by identifying outcomes of importance from HCPs’ and patients’ perspectives worldwide. This COS consisted of a combination of PROs (3 items), clinical outcomes (5 items), and adverse events. Although vomiting and regurgitation could be predominant GERD symptoms, they were not included in the final COS. Reasons for this may be that both HCPs and parents are more concerned about the consequences of GERD, such as impaired growth or reduced intake, instead of GERD-related symptoms as vomiting. Several outcomes included in the current COS, such as crying and adverse events, overlap with outcomes included in recently developed pediatric COS (6,12,13). This may imply that similar aspects of treatment are considered important to parents, regardless of their child's underlying condition. There may thus be room for development of a general COS applicable to all pediatric functional gastrointestinal (GI) conditions that can be expanded with smaller disease-specific outcomes.
Although final agreement on these outcome measures was reached through consensus meetings, individual steps of the process revealed great heterogeneity in outcomes considered to be important by HCPs or parents and patient representatives. In general, parents and representatives considered outcomes associated with quality of life, like reassurance, education, and symptom relief as critical, whereas HCPs gave greater importance to objective clinical outcomes such as weight gain, endoscopic healing, and patient survival. A likely explanation may be that HCPs perceive objective improvement in signs of disease and prevention of complications of disease as their primary roles in the management of infant GERD. Because quality of life-related PROs are more intangible and may be nonspecific for GERD, HCPs place less importance on them. Despite those factors, this COS clearly indicates the importance of these measures to parents, yet none of the currently available randomized controlled trials on the management of infant GERD included PROs such as quality of life or parental satisfaction as outcomes of interest. Second, the OMERACT initiative recommends early involvement of parents, patients, and representatives throughout the COS development to ensure that these PROs truly reflect the patient's perspective (10,18,19). Patients and representatives were, however, involved during the development process of only 16% of the currently available COS (10). This provides implications for future research in this area (2,4). This COS may also be of use in clinical practice to facilitate shared decision making, because the COS is a reflection of outcome measures of greatest importance to HCPs and parents of infants with GERD. Clearly, there is a need for HCPs to provide better education and reassurance tools (20–22). Family-centered services may help define and address the specific goals and expectations of the parents, while simultaneously ensuring optimal clinical care of the infant.
Determining how to measure the outcomes included in this COS will be challenging. Several instruments have already been used in other intervention studies (2). For instance, endoscopy is a validated instrument to measure “Esophagitis,” and the Infant Gastroesophageal Reflux Questionnaire Revised is a reliable instrument to assess GERD symptoms in infants over time and report on treatment outcomes (2,23–26). However, for outcome measures such as “Adequate Relief” and “Sleep Problems,” where more than 1 instrument can be used, it may be difficult to find appropriate validated measurement tools. The Consensus-based Standards for the selection of health Measurement Instruments–COMET guideline, may help to find the most appropriate instruments (27). The adjustment of well-validated measurement tools in other healthcare areas may be a solution. For example, randomized controlled trials on functional GI disorders have shown that a simple dichotomous question (“yes”/”no”) can reliably measure “Adequate Relief,” and validated questionnaires can be used to assess patients’ and parental quality of life (28–31). The first step, however, will be a detailed systematic review on the instruments that can be used to measure each of the agreed outcomes.
This study has several strengths and some limitations. Currently, there is no consensus regarding the best methodology for the development of COS. In accordance to the vast majority of currently available COS, we followed the COMET Initiative guidelines for COS development and used a Delphi survey to obtain information from a global group of participants (8,11,32,33). Other methods include the nominal group technique, a consensus development conference and a semistructured group discussion (34–37). However, these all require face-to-face communication early in the process. The Delphi process avoids the logistical and economic challenges associated with face-to-face meetings (11,32,33). Moreover, it enables participants to vote anonymously; thereby, minimizing the risk of bias resulting from more dominant individuals affecting the opinions of the group (7). It may, however, be possible that an alternative method would have led to a different COS, as was seen in 4 COS in childhood asthma—all of which used a different methodology (6,38–40).
A major strength of this study is that we included participants from 5 different continents and >30 countries. We reached a response rate of 60%, which is similar to previous studies in this field (6,12,13). Second, we actively involved patient representatives during the consensus meeting, which has only been done by a slight minority of currently available COS (10). As we developed a COS for therapeutic trials on infant GER disease, we aimed to include HCPs directly involved in the management of infant GERD. We used fewer primary care and general pediatricians involved in the management of infant GER, because therapy is not needed for these infants. We therefore conducted the survey during the ESPGHAN annual meeting. We primarily included pediatric gastroenterologists and general pediatricians with a special interest in GI problems. It may be possible that there are differences in responses between pediatric gastroenterologists and pediatricians. However, since we included predominantly pediatric gastroenterologists and numbers in the other groups are small, no group-to-group comparison was performed. Last, we used both a 9-point- and 5-point Likert scale to rate the importance of the outcome measures noted in round 2 of the COS development. Reducing the risk of researchers’ drawing conclusions based on their own beliefs, potentially causing bias, is a major advantage of the 9-point Likert scale, and was set for the HCPs in this COS process (8). The 5-point Likert scale creates a clear difference between categories, making it more comprehensible for people who are not experts in the field, and is a powerful scale for parents to rate the outcomes in this COS (41).
A limitation of our study may be that participants from developing countries were under-represented (13%), which could limit the external validity of this COS in a developing country setting (ie, Gross National Income per capita as calculated by the World Bank Atlas method, 2016). Furthermore, there may be a bias for developed countries (42). The included outcome measures in this COS are, however, only the baseline minimum number to include in future studies, and stakeholders can include additional and specific items as required, as some outcome measures may be specific to their culture or their country's health organization. Second, the questionnaire was developed in English and presented to respondents in their native language. The answers were translated back into English by the researchers who had questioned the participants. It is possible that the answers may have been misinterpreted. It should also be noted that, despite our efforts and with the help of our colleagues, some illegible and listed outcomes reported by the participants were not included, which could have induced reporting bias.
In conclusion, we developed a COS for infant GERD consisting of 9 outcome measures that should, at minimum, be measured in future clinical research trials on infant GERD. This COS has the potential to increase comparability of future studies in the field of infant GERD and will therefore allow better evidence-based decision making. It is important to emphasize that this COS is dynamic and will require future review and adjustment. In addition, it will depend on the study population (ie, severity of disease), design of the study, and the intervention assessed which outcomes are appropriate to be included. Implementation of this COS will unavoidably require a clear consensus on how to define GERD in different age groups to obtain homogenous patient populations and facilitate comparison between studies (2,43). The next step will be to identify which measurement tools are required, and which tools are not adequately validated to allow successful implementation (27).
The authors thank all healthcare professionals and parents of infants with GERD for their valuable contributions to this study. The authors would also like to thank the experts who were involved in the survey for this research project and for their assistance in data collection: M. Van Lennep, C. Di Lorenzo, T. Omari, M. Messalska, E. Privat, M.J. Smits, A. Staiano, Z.A. Steur, F.K. Wester-Khawaja, and J. Zeevenhooven. Without their passionate participation and input, the survey could not have been successfully conducted.
1. Holloway RH, Penagini R, Ireland C. Criteria for objective definition of transient lower esophageal sphincter relaxation. Am J Physiol
2. Rosen R, Vandenplas Y, Singendonk M, et al. Pediatric gastroesophageal reflux clinical practice guidelines: Joint Recommendations of the North American Society for Pediatric Gastroenterology, Hepatology, and Nutrition and the European Society for Pediatric Gastroenterology, Hepatology, and Nutrition. J Pediatr Gastroenterol Nutr
3. Sherman PM, Hassall E, Fagundes-Neto U, et al. A Global, evidence-based consensus on the definition of gastroesophageal reflux disease in the pediatric population. Am J Gastroenterol
4. Singendonk MMJ, Brink AJ, Steutel NF, et al. Variations in definitions and outcome measures in gastroesophageal reflux disease: a systematic review
2017; 140:pii: e20164166.
5. Quitadamo P, Urbonas V, Papadopoulou A, et al. Do pediatricians apply the 2009 NASPGHAN–ESPGHAN guidelines for the diagnosis and management of gastroesophageal reflux after being trained? J Pediatr Gastroenterol Nutr
6. Sinha IP, Gallagher R, Williamson PR, et al. Development of a core outcome set for clinical trials in childhood asthma: a survey of clinicians, parents, and young people. Trials
7. Sinha IP, Smyth RL, Williamson PR. Using the Delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies. PLoS Med
8. Williamson PR, Altman DG, Blazeby JM, et al. Developing core outcome sets for clinical trials: issues to consider. Trials
9. Sinha I, Jones L, Smyth RL, et al. A systematic review of studies that aim to determine which outcomes to measure in clinical trials in children. PLoS Med
10. Boers M, Kirwan JR, Wells G, et al. Developing core outcome measurement sets for clinical trials: OMERACT filter 2.0. J Clin Epidemiol
11. Karas J, Ashkenazi S, Guarino A, et al. A core outcome set for clinical trials in acute diarrhoea. Arch Dis Child
12. Steutel NF, Benninga MA, Langendam MW, et al. Developing a core outcome set for infant colic for primary, secondary and tertiary care settings: a prospective study. BMJ Open
13. Kuizenga-Wessel S, Steutel NF, Benninga MA, et al. Development of a core outcome set for clinical trials in childhood constipation: a study using a Delphi technique. BMJ Paediatr Open
14. Karas J, Ashkenazi S, Guarino A, et al. Developing a core outcome measurement set for clinical trials in acute diarrhoea. Acta Paediatr
15. Guyatt GH, Oxman AD, Vist GE, et al. GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. Chinese J Evidence-Based Med
16. GRADE Working Group. Grading quality of evidence and strength of recommendations. BMJ
17. Sinha IA, Altman DG, Beresford MW, et al. Standard 5: selection, measurement, and reporting of outcomes in clinical trials in children. Pediatrics
18. Gargon E, Gurung B, Medley N, et al. Choosing important health outcomes for comparative effectiveness research: a systematic review. Value Heal
19. Wiering B, De Boer D, Delnoij D. Asking what matters: the relevance and use of patient-reported outcome measures that were developed without patient involvement. Health Expect
20. Farrell MF, Frost C. The most important needs of parents of critically ill children: parents’ perceptions. Intensive Crit Care Nurs
21. Fisher MD. Identified needs of parents in a pediatric intensive care unit. Crit Care Nurse
1994; 14:82–90. http://www.ncbi.nlm.nih.gov/pubmed/8194354http://www.ncbi.nlm.nih.gov/pubmed/8194354
. Accessed May 16, 2018.
22. Kirschbaum MS. Needs of parents of critically ill children. Dimens Crit Care Nurs
1990; 9:344–352. http://www.ncbi.nlm.nih.gov/pubmed/2226116http://www.ncbi.nlm.nih.gov/pubmed/2226116
. Accessed May 16, 2018.
23. Lightdale JR, Gremse DA. Section on Gastroenterology, Hepatology, and Nutrition. Gastroesophageal reflux: management guidance for the pediatrician. Pediatrics
24. Vakil N, Van Zanten SV, Kahrilas P, et al. The Montreal definition and classification of gastroesophageal reflux disease: a global evidence-based consensus. Am J Gastroenterol
25. Van Der Pol RJ, Smits MJ, Venmans L, et al. Diagnostic accuracy of tests in pediatric gastroesophageal reflux disease. J Pediatr
26. Orenstein SR. Symptoms and reflux in infants
: Infant Gastroesophageal Reflux Questionnaire Revised (I-GERQ-R)—utility for symptom tracking and diagnosis. Curr Gastroenterol Rep
27. Prinsen CAC, Vohra S, Rose MR, et al. How to select outcome measurement instruments for outcomes included in a “Core Outcome Set”—a practical guideline. Trials
28. Camilleri M, Mangel AW, Fehnel SE, et al. Primary endpoints for irritable bowel syndrome trials: a review of performance of endpoints. Clin Gastroenterol Hepatol
29. Mangel A, Hahn B, Heath A, et al. Adequate relief as an endpoint in clinical trials in irritable bowel syndrome. J Int Med Res
30. Irvine EJ, Tack J, Crowell MD, et al. Design of treatment trials for functional gastrointestinal disorders. Gastroenterology
31. Haywood K, Brett J, Salek S, et al. Patient and public engagement in health-related quality of life and patient-reported outcomes research: what is important and why should we care? Findings from the first ISOQOL patient engagement symposium. Qual Life Res
32. Williamson PR, Altman DG, Bagley H, et al. The COMET handbook: version 1.0. Trials
33. Cole ZD, Donohoe HM, Stellefson ML. Internet-based Delphi research: case based discussion. Environ Manage
34. Haywood KL, Griffin XL, Achten J, et al. Developing a core outcome set for hip fracture trials. Bone Joint J
35. Rider LG, Giannini EH, Brunner HI, et al. International consensus on preliminary definitions of improvement in adult and juvenile myositis. Arthritis Rheum
36. Kirchhof P, Auricchio A, Bax J, et al. Outcome parameters for trials in atrial fibrillation: recommendations from a consensus conference organized by the German Atrial Fibrillation Competence NETwork and the European Heart Rhythm Association. Europace
37. Zannad F, Garcia AA, Anker SD, et al. Clinical outcome endpoints in heart failure trials: a European Society of Cardiology Heart Failure Association consensus document. Eur J Heart Fail
38. Smith MA, Leeder SR, Jalaludin B, et al. The asthma health outcome indicators study. Aust N Z J Public Health
1996; 20:69–75. http://www.ncbi.nlm.nih.gov/pubmed/8799071http://www.ncbi.nlm.nih.gov/pubmed/8799071
. Accessed July 4, 2018.
39. Reddel HK, Taylor DR, Bateman ED, et al. An official American Thoracic Society/European Respiratory Society statement: asthma control and exacerbations: standardizing endpoints for clinical asthma trials and clinical practice. Am J Respir Crit Care Med
40. Busse WW, Morgan WJ, Taggart V, et al. Asthma outcomes workshop: overview. J Allergy Clin Immunol
41. Lozano LM, García-Cueto E, Muñiz J. Effect of the number of response categories on the reliability and validity of rating scales. Methodology
43. Davies I, Burman-Roy S, Murphy MS. Gastro-oesophageal reflux disease in children: NICE guidance. BMJ