The objective of this systematic review is to synthesise the best available evidence on the diagnostic test accuracy (sensitivity and specificity) of Clinical Swallow Assessment (CSA) compared with Video Fluoroscopic Swallow Study (VFSS) in diagnosing oropharyngeal aspiration in children and adults with dysphagia. In this review, VFSS will be the ‘Gold Standard’ or, reference test and CSA will be the index test.
VFSS is the radiographic assessment of swallow function usually performed by a Speech Pathologist and Radiologist. CSA is defined as a non-invasive, non-instrumental assessment of swallow function. It is usually performed by a Speech Pathologist and includes a thorough case history and assessment of oro-motor function. For example, tongue, jaw movement, cough reflex and review of swallow function with a range of food and fluids. A rating is usually provided to describe the level of function and likely presence or absence of aspiration.
Based on the CSA findings the Speech Pathologist recommends specific diet and fluid modifications, feeding techniques and positioning techniques to enable safe oral feeding.
Swallowing is a complex and dynamic process. It involves the precise co-ordination of over 31 paired muscles, six cranial nerves and the central nervous system including the brain stem and cerebral cortex1,2,3, 4. Swallowing in children and adults is divided into four phases: the oral preparatory, oral, pharyngeal and oesophageal5, 3. Disruption to the normal sequence of swallowing during any or all of these phases is termed dysphagia and is defined by American Speech-Language-Hearing Association as “A swallowing disorder. The signs and symptoms of dysphagia vary and may involve the mouth, pharynx, larynx, and/or esophagus”6. Dysphagia can affect infants, children and adults.
The pharyngeal phase of swallowing is of particular interest as it involves structures of the larynx and pharynx which are also used during respiration (breathing) and phonation (talking)4. During the normal process of swallowing the trachea is protected as the bolus passes through the pharynx and enters the oesophagus7. However, patients with dysphagia may experience penetration or aspiration. Penetration is the entry of fluid or food particles into the laryngeal vestibule which does not pass below the level of the true vocal cords8. Aspiration is the entry of fluid or food particles below the level of the true vocal cords8, 9. Silent aspiration is the term used when aspiration occurs without a cough response.
Prompt and accurate diagnosis of aspiration is integral to patient management. It enables immediate intervention to provide the patient with sufficient nutrition and hydration without the risk of aspiration lung disease. If aspiration is not accurately diagnosed appropriate interventions can not be implemented. Aspiration left untreated can have a range of poor health outcomes including: inadequate nutrition and hydration, compromised lung development and pulmonary integrity10, chronic lung disease1 and recurrent respiratory symptoms11.
In infants and young children aspiration episodes may be associated with transient cyanosis, recurrent chest infections or pneumonia12. Prompt diagnosis and treatment may also alleviate further aspiration-related damage to lung growth or function. The diagnosis of aspiration in young infants may be the initial presenting feature of an airway anomaly such as laryngeal cleft12 or a neurological disorder. The diagnosis of aspiration may guide direction for further necessary investigations or interventions.
The swallowing pattern of infants and children is different to adults and the anatomy and physiology of swallowing continues to change from infancy through to adulthood13, 14. The anatomical differences in an infant provide optimal conditions for breast feeding by allowing the infant to use a combination of jaw, cheek and tongue movement to create the sucking rhythm required for breast feeding. The position of the larynx is ‘higher’ than in an older child or adult and provides optimal airway protection during swallowing. As the infant grows their larynx descends and begins to resemble the anatomy of a young child by around 6 months of age14. By three years of age children begin to adopt a swallowing pattern similar to an older child4.
Causes of dysphagia and responses to aspiration are also different in children compared with adults. For example causes of dysphagia in children include: prematurity, reflux, neurological disorder and respiratory difficulties15,14 and symptoms suggestive of aspiration include: upper airway noises, apnoea and cyanosis with feeds18. Causes of dysphagia in adults include: stroke, Parkinson's disease and Alzheimer's disease15 and symptoms suggestive of aspiration include: coughing, extra chewing time and loss of food or fluid from the mouth15.
The presence of neurological conditions may impact on the incidence, symptomatology and diagnosis of oropharyngeal aspiration. Oropharyngeal dysphagia is well documented in people with neurological conditions, (for example cerebral palsy and post-stroke) and such conditions predisposes them to oropharyngeal aspiration1, 19, 14. These conditions can affect the structures and precise neural control required for swallowing14. The cough reflex, in response to aspiration may also be affected14.
Oropharyngeal dysphagia in children without known neurological conditions and without apparent risk factors is not well documented in the literature and few clinical guidelines are available for assessment and treatment in this group20. These children may present with ambiguous or unexplained respiratory symptoms1 and may present as neurologically intact and normally developing children20.
The ‘Gold Standard’ for assessment and diagnosis of oropharyngeal aspiration is a Video Fluoroscopic Swallowing Study (VFSS)10, 7 used frequently as the reference standard in tests of diagnostic test accuracy1, 9, 21, 22. VFSS exposes the patient to radiation, is relatively expensive, requires specialist equipment and staff and is not available to all clinicians21. In contrast, the Clinical Swallow Assessment or ‘Bedside Swallow Assessment’ is a non invasive assessment of swallowing and oral feeding skills and is widely available to all clinicians.
Video Fluoroscopic Swallow Study
VFSS is the radiographic assessment of swallow function with a focus on the oral, pharyngeal and upper oesophageal phase of swallowing The use of fluoroscopy for assessment of swallowing was first mentioned in 189823. At that time the purpose was to assess the oesophageal phase of swallowing. In 1927, Mosher, a laryngologist, used fluoroscopy to further assess features of the pharyngeal swallow. Over time new equipment allowed for a more detailed observation and record of swallow function24 and studies included a wide range of subjects - infants, children and adults with and without medical co morbidities.
The use of VFSS has evolved since this time and the first published protocol for VFSS was by Logeman in 198325. VFSS is now used as a diagnostic tool to assess airway protection in patients with clinical symptoms suggestive of aspiration. VFSS allows the clinician to objectively evaluate all phases of swallowing and diagnose aspiration and silent aspiration1. The use of VFSS also enables diagnosis of post swallow residue within the pharynx which has been linked to pneumonia9. VFSS allows the clinician to trial consistencies and altered patient positioning to establish an eating and drinking program without aspiration11. In addition the VFSS films are recorded and able to be replayed to families which has been shown to increase compliance with clinician recommendations.
The fluids and solids used during the study are radio-opaque, prepared with a contrast agent such as Barium or Omniopaque. These fluids are intended to represent the infant's formula in order to establish fluids that do and do not result in aspiration. The results are presented as a binary outcome whereby the clinician records that the infant, child or adult is aspirating or is not aspirating.
Clinical Swallow Assessment (CSA)
Clinical Swallow Assessment (CSA) is performed by a speech pathologist and the patient in a natural setting such as home or outpatient clinic. It usually involves a thorough case history, cranial nerve assessment and review of the patient eating and drinking their usual food and fluid21, 26. Speech Pathologists use a range of signs and symptoms during clinical swallow assessment as indicators of dysphagia including cough, wheeze and voice changes18. The assessment guides recommendations for ‘safe swallowing’ and the speech pathologist will often recommend a modified diet, fluids or feeding strategies. Clinical swallow assessment is inexpensive and does not require additional specialist staff (such as radiographer and radiologist) or specialist equipment. It can be repeated frequently for patients with rapidly changing dysphagia27.
For the purpose of this study, CSA is defined as the assessment of swallowing by a Speech Pathologist or Occupational Therapist using fluids and solids of varying viscosity. The results are recorded using a pre-determined checklist which includes variables frequently considered for assessment of swallow function such as: respiration changes, voice changes and presence or absence of coughing21. The results are presented as a binary outcome whereby the clinician records that the infant, child or adult is aspirating or is not aspirating.
Some studies have shown clinical swallow assessment is a poor diagnostic tool for assessment of aspiration10 particularly for patients experiencing silent aspiration. Coughing or signs of choking is commonly used as a clinical marker to diagnose the presence of aspiration, however silent aspiration is common28, 1, 10 particularly in children with neurological impairment29.
This may reduce the reliability of clinical swallow assessment as a diagnostic tool for aspiration29. In contrast, other studies have shown CSA is a reasonable screening tool for children and adults at risk of oropharyngeal aspiration30, particularly aspiration of fluids21. Clinical swallow assessment is an essential diagnostic tool for centres where VFSS facilities are not available.
A preliminary search of JBI Library of Systematic Reviews, JBI COnNECT+, The Cochrane Library, PubMed and CINAHL has been conducted and revealed that no other systematic review either exists on this topic or is under way.
Types of participants
Studies will be considered in this review if they include: infants, children or adults with dysphagia.
The following definitions will be used within this review:
- infant will be defined as < 36 months;
- children will be defined as 36 months - 18 years;
- adults will be defined as 18 years and over;
- dysphagia will be defined as ‘a swallowing disorder’6.
The definition uses 36 months as the cut off between infants and children as this is the stage when the infant transitions from sucking patterns used for bottle feeding and early solids and begins to chew solids, self feed and drink from a range of cups consistent with an older child4.
There will be no exclusion of studies based on age or gender.
Studies will be excluded that include the population of head and neck cancer (HNC) patients, patients with a tracheostomy in situ and patients with craniofacial anomalies. The primary treatment for HNC patients includes surgery and radiotherapy31. Radiotherapy can have a specific impact on swallowing32, not necessarily seen in other populations and surgical resections can result in predictable swallowing difficulties32.
Patients with a tracheostomy in situ and patients with craniofacial anomalies also have known anatomical changes that have a specific impact on swallow function.
The clinician undertaking the clinical swallow assessment has access to any anatomical changes following surgery. This is different to other populations whereby information regarding specific neurological and anatomical information is not necessarily available.
Focus of the review
The focus of this review will be on diagnostic test accuracy (sensitivity and specificity) of Clinical Swallow Assessment (CSA) compared with Video Fluoroscopic Swallow Study (VFSS) in diagnosing oropharyngeal aspiration in children and adults with dysphagia. In this review, VFSS will be the ‘Gold Standard’ or, reference test and CSA will be the index test.
The terms VFSS and Modified Barium Swallow (MBS) refer to the same study31, 14. Although this review will use the term VFSS defined as the radiographic assessment of the oral and pharyngeal swallow, MBS studies will also be included.
The terms Clinical Swallow Assessment and Bedside Swallow Evaluation also refer to the same diagnostic tool33. Clinical Swallow Assessment (CSA) will be used within this review, defined as the non-instrumental, non-radiologic assessment of swallow function by a Speech Pathologist, however studies of Bedside Swallow Evaluation will also be included.
Studies that do not use CSA as the index test or do not use VFSS as the reference test will be excluded. In addition, studies of effectiveness, experience or comparison of treatment interventions will be excluded.
The diagnostic test accuracy of CSA and VFSS will be compared using: sensitivity and specificity. Where possible, positive and negative predictive values will also be analysed and reported.
Sensitivity of a diagnostic test is defined as ‘the ability of the test to identify correctly those who have the disease’34.
Specificity of a diagnostic test is defined as ‘the ability of the test to identify correctly those who do not have the disease’34.
Positive predictive value is defined as the proportion of patients who test positive and actually have the disease in question34.
Negative predictive value is defined as the proportion of patients who test negative and actually do not have the disease in question34.
Types of studies
This review will include studies of diagnostic test accuracy. It is anticipated studies will predominantly be cross-sectional studies.
The search strategy aims to find all relevant available literature, published and unpublished. Initial search terms and databases were considered by researching this area35, 36 and discussion with a research librarian. Databases will be searched from their inception, to April 31st 2012. A three-step search will be used. Initially, a limited search of PubMed and CINAHL will be undertaken in order to identify appropriate keywords. Analysis of the text words and MeSH terms identified by the search to describe relevant articles will then be used to identify additional search terms which will then be used to search across all included databases. The databases to be searched are listed in Appendix I and the initial search terms are listed in Appendix II. Thirdly, the reference list of identified papers will be searched for additional studies. Hand searching of relevant journals will also occur for the following journals - Dysphagia and International Journal of Speech-Language Pathology.
The search will not be limited by year but will be limited to those published in the English language.
The following list of PubMed MeSH terms are also used to describe VFSS and CSA and will be included in the search strategy
- deglutition: The act of taking solids and liquids into the gastrointestinal tract through the mouth and throat.
- pneumonia, aspiration: A type of lung inflammation resulting from the aspiration of food, liquid, or gastric contents into the upper respiratory tract.
- photofluorography: The photography of images produced on a fluorescent screen by X-rays.
- Videofluorography: Motion picture study of successive images appearing on a fluoroscopic screen.
Assessment of methodological quality
Selected studies will be assessed by two independent reviewers for methodological validity prior to inclusion in the review using the QUADAS checklist37(Appendix III). Any disagreements that arise between the reviewers will be resolved through discussion, or with a third reviewer.
Data will be extracted from included studies using the STARD checklist38 consisting of 25 items (Appendix IV). The data extracted will include specific details regarding: populations, data collection and methods used for calculating or comparing measures of diagnostic accuracy. In situations where relevant study features are not provided within the study paper, the reviewer will contact the author to attempt to source additional information.
The sensitivities and specificities from individual studies will be combined to generate a summary estimate of the accuracy of clinical swallow assessment compared with VFSS. Meta-analysis includes graphing the results of individual studies. The sensitivity and specificity are plotted as points on a graph. These plotted points are presented as an ROC (receiver operating curve) space, demonstrating the covariation between sensitivity and specificity39
Revman 5 (Cochrane Collaboration)35 and Open Office.org Calc computer software will be used for data analysis and synthesis.
Subgroup analyses (if appropriate) will include:
- infants, children and adults
- subjects with a neurological condition compared to subjects without a neurological condition
Consensus does not exist for age ranges that define infancy, childhood and adolescence and definitions vary depending on the type of research being conducted40.
Where statistical pooling is not possible the findings will be presented in narrative form including tables and figures to aid in data presentation where appropriate.
Conflicts of interest
As this systematic review forms partial submission for the degree award of Masters of Clinical Sciences, a secondary reviewer will only be used for critical appraisal.
Drs Tim Schultz and Andrew Tai form the supervisory team for this project.
 Lefton Greif MA, Carroll JL, Loughlin GM. Long-term follow-up of oropharyngeal dysphagia
in children without apparent risk factors. Pediatric Pulmonology.2006; 41 (11): 1040-1048.
 Bosma, JF. Development of feeding. Clinical Nutrition.1986; 5 (5): 210-218.
 Dodds WJ. The physiology of swallowing. Dysphagia
.1989; 3 (4): 171-178.
 Arvedson JA, and Brodsky L, Pediatric swallowing and feeding assessment and management: 2nd Edition; Delmar Cenage Learning; 2002.
 Logemann JA. Swallowing physiology and pathophysiology. Otolaryngologic Clinics of North America.1988; 21 (4): 613-623.
 American Speech-Language-Hearing Association,. Roles of speech-language pathologists in swallowing and feeding disorders: technical report. 2001 [cited 2012 15/6/2012]. Available from: http://www.asha.org/docs/html/TR2001-00150.htmlhttp://www.asha.org/docs/html/TR2001-00150.html
 Martin Harris B and Jones B. The videofluorographic swallowing study. Otorinolaringol..2009; 59 (1): 19-29.
 Rosenbek JC, Robbins JA, Roecker EB, Coyle JL, Wood J. A penetration-aspiration scale. Dysphagia
.1996; 11 (2): 93-98.
 Weir K, McMahon S, Barry L, Ware R, Masters B, Chang AB. Oropharyngeal aspiration and pneumonia in children. Pediatric Pulmonology.2007; 42 (11): 1024-1031.
 de Benedictis FM, Carnielli VP, de Benedictis D. Aspiraiton lung disease. Pediatric Clinics of North America.2009; 56 (1): 173-90.
 Boesch RP, Daines C, Willging JP, Kaul A,, Cohen AP, Wood RE, Amin RS. Advances in the diagnosis and management of chronic pulmonary aspiration in children. European Respiratory Journal. 2006; 28 (4): 847-861.
 Rutter MJ. Evaluation and management of upper airway disorders in children. Seminars in Pediatric Surgery.2006; 15 (2): 116-23.
 Ruark JL, McCullough GH, Peters RL, Moore CA. Bolus consistency and swallowing in children and adults. Dysphagia
.2002; 17 (1): 24-33.
 Tutor JD and Gosa MM. Dysphagia
and aspiration in children. Pediatric Pulmonology.2012; 47 (4): 321-337.
 American-Speech-Language-Hearing Association. Swallowing disorders (dysphagia
) in adults.1997-2012 [cited 2012 10/6/2012]; Available from: http://www.asha.org/public/speech/swallowing/SwallowingAdults.htmhttp://www.asha.org/public/speech/swallowing/SwallowingAdults.htm
 American-Speech-Language-Hearing Association. Feeding and swallowing disorders (dysphagia
) in children. 1997-2012 [cited 2012 10/6/2012]; Available from: http://www.asha.org/public/speech/swallowing/Swallowing-Disorders-Causes/http://www.asha.org/public/speech/swallowing/Swallowing-Disorders-Causes/
 Weir K, McMahon S, Barry L, Chang AB. Clinical signs and symptoms of oropharyngeal aspiration and dysphagia
in children. European Respiratory Journal.2009; 33 (3): 604-611.
 Okubo PCMI, Fabio SRC, Domenis DR, Takayanagui OM. Using the national institute of health scale to predict dysphagia
in acute ischaemic stroke. Cerebrovascular Diseases.2012; 33 (6): 501-507.
 Richter GT. Management of oropharyngeal dysphagia
in the neurologically intact and developmentally normal child. Current Opinion Otolaryngology Head Neck Surgery.2010; 18 (6): 554-563.
 DeMatteo C, Matovich D, Hjartarson A. Comparison of clinical and videofluoroscopic evaluation of children with feeding and swallowing difficulties. Developmental Medicine and Child Neurology.2005; 47 (3): 149-157.
 Cichero JAY, Hay GB, Murdoch BE, Halley PJ. Videofluoroscopic fluids versus mealtime fluids: differences in viscosity and density made clear. Journal of Medical Speech-Language Pathology.1997; 5 (3): 203-215.
 Cannon WB, and Moser A. The movements of food in the esophagus. American Journal of Physiology; 1: 435-444.
 Ramsey GH, Watson JS, Gramiak R, Weinberg SA. Cinefluorographic analysis of the mechanisms of swallowing. Radiology.1955; 64 (4): 498-518.
 Palmer JB, Kuhlemeier KV, Tippett DC, Lynch C. A protocol for the videofluorographic swallowing study. Dysphagia
.1993; 8 (3): 209-214.
 Mari F, Matei M, Ceravolo MG, Pisani A, Montesi A, Provinciali L. Predictive value of clinical indices in detecting aspiration in patients with neurological disorders. Journal of Neurology, Neurosurgery and Psychiatry.1997; 63 (4): 456-460.
 Warms T and Richards J. “Wet voice” as a predictor of penetration and aspiration in oropharyngeal dysphagia
.2000; 15 (2): 84-88.
 Garon BR, Sierzant T, Ormiston C. Silent aspiration: results of 2,000 video fluoroscopic evaluations. Journal of Neuroscience Nursing.2009; 41 (4): 178-185.
 Arvedson J, Rogers B, Buck G, Smart P, Msall M. Silent aspiration prominent in children with dysphagia
. International Journal of Pediatric Otorhinolaryngology.1994; 28 (2-3): 173-181.
 Suiter DM, and Leder SB. Clinical utility of the 3-ounce water swallow
.2008; 23 (3): 321-337.
 Raber-Durlacher JE, Brennan MT, Verdonck-de Leeuw IM, Cubson RJ, Eilers JG, Waltino T, Bots CP, Michelet M, Sollecito TP, Rouleau TS, Sewnaik A, Bensadoun RJ, Fliedner MC, Silverman Jr S, Spijkervet FKL, Dysphagia
section, oral care study group, multinational association of support care in cancer/international society of oral oncology. Swallowing dysfunction in cancer patients. Support Care Cancer.2012; 20 (3): 433-443.
 Gaziano JE. Evaluation and management of oropharyngeal dysphagia
in head and neck cancer. Cancer Control.2002; 9 (5): 400-409.
 McCullough GH, Wertz RT, Rosenbek JC. Sensitivity and specificity of clinical/bedside examination signs for detecting aspiration in adults subsequent to stroke. Journal of Communication Disorders.2001; 34 (1-2): 55-72.
 Gordis L. Epidemilogy. Third edition. Elsevier Inc. 2004. p 72-83.
 White S, Schultz T, Enuameh YAK. Conducting a systematic review of diagnostic test accuracy
evidence. Synthesizing evidence of diagnostic accuracy. 2011. JBI-LWW p 49-73.
 The Joanna Briggs Institute. Quantitative Evidence. Joanna Briggs Institue reviewers' manual: 2011 edition. 2011. p 45-76.
 Whiting P, Rutjes AWS, Reitsma JB, Bossuyt PMM, Kleijrien J. The development of QUADAS: a tool for the quality assessment of studies of diagnostic test accuracy
included in systematic reviews. BMC Medical Research Methodology.2003; 3 (25): 1-13.
 Meyer GJ. Guidelines for reporting information in studies of diagnostic test accuracy
: the STARD initiative. Journal of Personality Assessment.2003; 81 (3): 191-193.
 Leeflang MMG, Deeks JJ, Gatsonis C, Bossuyt PMM. Systematic reviews of diagnostic test accuracy
. Ann Intern Med.2008; 149 (12): 889-897.
 Institute of Medicine (US) Committee on Clinical Research Involving Children. The Necessity and Challenges of Clinical Research Involving Children. Ethical conduct of clinical research involving children. 2004.
Appendix I: Electronic Databases
Web of Science
Web of Knowledge
Networked Digital Library of Theses and Dissertations
DART-Europe E-theses portal
Appendix II: Initial search terms
oesophageal motility disorders
Appendix III: Critical appraisal tool - the QUADAS check list37
1. Was the spectrum of patients representative of the patients who will receive the test in practice?
2. Were selection criteria clearly described?
3. Is the reference standard likely to correctly classify the target condition?
4. Is the time period between reference standard and index test short enough to be reasonably sure that the target condition did not change between the two tests?
5. Did the whole sample or a random selection of the sample, receive verification using a reference standard of diagnosis?
6. Did patients receive the same reference standard regardless of the index test result?
7. Was the reference standard independent of the index test (i.e. the index test did not form part of the reference standard)?
8. Was the execution of the index test described in sufficient detail to permit replication of the test?
9. Was the execution of the reference standard described in sufficient detail to permit its replication?
10. Were the index test results interpreted without knowledge of the results of the reference standard?
11. Were the reference standard results interpreted without knowledge of the results of the index test?
12. Were the same clinical data available when test results were interpreted as would be available when the test is used in practice?
13. Were uninterpretable/ intermediate test results reported?
14. Were withdrawals from the study explained?
Appendix IV: Data extraction tool - the STARD check list38
1 Was the study identified as being a diagnostic accuracy study?
2 Were research questions or study aims, such as estimating diagnostic accuracy or comparing accuracy between tests or across participant groups, detailed?
3 Does the study describe the study population, inclusion and exclusion criteria, setting and locations where the data were collected?
4 Does the study describe participant recruitment? Was recruitment based on presenting symptoms, results from previous tests, or the fact that the participants had received the index tests or the reference standard?
5 Describe participant sampling: Was the study population a consecutive series of participants defined by the selection criteria in items 3 and 4? If not, specify how participants were further selected
6 Describe data collection: Was data collection planned before the index test and reference standard were performed (prospective study) or after (retrospective study)?
7 Did the study describe the reference standard and its rationale?
8 Describe technical specifications of material and methods involved including how and when measurements were taken, and/or cite references for index tests and reference standard?
9 Describe definition of and rationale for the units, cut-offs and/or categories of the results of the index tests and the reference standard.
10 Describe the number, training and expertise of the persons executing and reading the index tests and the reference standard
11 Describe whether or not the readers of the index tests and reference standard were blind (masked) to the results of the other test and describe any other clinical information available to the readers.
12 Describe methods for calculating or comparing measures of diagnostic accuracy, and the statistical methods used to quantify uncertainty (e.g. 95% confidence intervals)
13 Describe methods for calculating test reproducibility, if done
14 Report when study was done, including beginning and ending dates of recruitment
15 Does the study report clinical and demographic characteristics of the study population (e.g. age, sex, spectrum of presenting symptoms, co morbidity, current treatments, recruitment centres?
16 Does the study report the number of participants satisfying the criteria for inclusion that did or did not undergo the index tests and/or the reference standard; describe why participants failed to receive either test (a flow diagram is strongly recommended) ?
17 Does the study report time interval from the index tests to the reference standard, and any treatment administered between?
18 Does the study report distribution of severity of disease (define criteria) in those with the target condition; other diagnoses in participants without the target condition?
19 Does the study report a cross tabulation of the results of the index tests (including indeterminate and missing results) by the results of the reference standard; for continuous results, the distribution of the test results by the results of the reference standard?
20 Does the study report any adverse events from performing the index tests or the reference standard?
21 Does the study report estimates of diagnostic accuracy and measures of statistical uncertainty (e.g. 95% confidence intervals)?
22 Does the study report how indeterminate results, missing responses and outliers of the index tests were handled?
23 Does the study report estimates of variability of diagnostic accuracy between subgroups of participants, readers or centres?
24 Does the study report estimates of test reproducibility?
25 Does the study discuss the clinical applicability of the study findings?