Introduction
Patients in need of palliative care (PC) are present in many different care settings. Prevalence varies from 9% to 73% in acute-care settings depending on pathologies, comorbidities, or frailty.1-5 An international prevalence study revealed that the proportion of individuals who died from diseases indicating a need for PC ranged from 38% to 74%.6 According to Murtagh et al.,7 63% to 82% of the population will need PC at some point in their disease trajectory. With the increase of aging populations and the number of patients suffering from chronic diseases and polymorbidity, the number of patients in need of PC is likely to increase further.
When patients do not receive appropriate PC, there is strong evidence of i) excessive hospital mortality (80%) although the majority of people wish to die at home,8-11 ii) suboptimal symptom management,12-14 iii) unplanned hospitalizations with longer hospital stays,15,16 iv) prescription of inappropriate treatments due to a lack of anticipative care plans,17,18 and v) insufficient support for the patient and their relatives.15,19,20 Approximately 75% to 80% of all patients in need of PC can be cared for by non-specialist teams (general PC), and the remaining 20% to 25% may need specialized PC due to the complexity of their conditions.21-23 There is an internationally recognized distinction between patients requiring general versus specialized PC. General PC is provided by non-specialized professionals for non-complex situations. Specialized PC is for patients whose illness is characterized by great clinical instability and/or a high level of psycho-socio-existential suffering, making their situation so complex that non-PC-specialists do not have the necessary competencies to care for them.8,12
Identification of patients in need of general or specialized PC is a prerequisite for appropriate management.24 However, adequate identification is challenging as it relies on an unclear World Health Organization (WHO) definition of PC as “an approach that improves the quality of life of patients and their families facing the problem associated with life-threatening illness, through the prevention and relief of suffering by means of early identification and impeccable assessment and treatment of pain and other problems, physical, psychosocial and spiritual.”25(para.1) This definition does not allow differentiation between general and specialized PC.26-29 Despite this, experts agree that PC depends not only on patient diagnosis or prognosis but also on patient needs, and is not exclusively associated with terminal care. They confirm that PC should be initiated early in a patient's disease trajectory.30 Without a valid tool, the identification of patients in need of PC is suboptimal, because it occurs mostly at the end of life or only for highly complex situations.
A growing number of instruments exist to help professionals identify patients in need of PC. These instruments include criteria concerning the severity of illness, the progression of the disease, and its associated frailty. Some instruments were designed only for a specific setting, such as the emergency department or intensive care.31 It is recommended that PC is integrated early in the stage of a life-threatening event; however, the treatment goal of patients who require intensive care is primarily to sustain vital functions. Instruments designed for PC patients may therefore not be appropriate for this population. Other instruments were designed for a specific pathology; for example, interstitial lung disease,32 or for a specific age group such as older people.33 Others were designed for general use.34-36
To implement such an instrument into daily practice, it is necessary to select the most suitable one based on its psychometric properties, particularly its validity (measures what it is supposed to measure), reliability (consistency), and responsiveness (ability to detect change over time).
In PC, many patients are unable to fill in a patient-reported outcome to evaluate their own PC needs; for example, patients with dementia, any cognitive impairment, or delirium; those who are not well-informed about the stage of their disease; or those who have some cultural barriers to discussing this topic. In this context, it seems important that health care professionals carry out this first screening, which will help them to engage in discussion with patients and their relatives about their needs and disease trajectory, and to give them the most appropriate care. The focus of this review is therefore on clinician-reported outcome measures (ClinROMs) for the identification of patients in need of PC.
A preliminary search of MEDLINE, the Cochrane Database of Systematic Reviews, the COSMIN database, and the JBI Database of Systematic Reviews and Implementation Reports was conducted, and two systematic reviews on the topic were identified.37,38 Both reviews are focused on the inventory and the description of the available instruments.
In Maas et al.37 a survey was conducted with European general practitioners to identify non-published instruments. Maas et al. carried out their review in only two databases (PubMed/MEDLINE and Embase) and focused their searching on articles describing identification instruments suitable for use in primary care. A narrative approach was used to synthesize data. They then compared the content of the different instruments and the identification criteria. Little information is available about the data collection process, the quality assessment of the studies, the studies’ characteristics, and whether authors were contacted to obtain complementary data. They found five articles, which described four different instruments: the RADboud indicators for Palliative Care needs (RADPAC),39 the Residential home palliative care tool,40 the Supportive and Palliative Care Indicators Tool (SPICT),35 and the Early identification tool for palliative care patients.41 The survey also included three instruments that had not been identified in the literature (despite two of these instruments being widely known), which raises questions about the quality of the review. These three instruments are the Prognostic Indicator Guidance of the UK-based Gold Standard Framework (GSF-PIG),34 the NECPAL CCOMS-ICO,36 and the NHS Quick guide to identifying patients for supportive and palliative care (Quick guide).42 The instruments were compared. Where sensitivity and specificity were reported in the original studies, Maas et al.37 mentioned these, but did not mention any psychometric assessment. None of the presented instruments distinguishes patients in need of general versus specialized PC.
Walsh et al.38 conducted their systematic review by following the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA).43 Most of the PRISMA criteria were presented, but Walsh et al.38 did not perform any systematic evaluation of the psychometric properties and risks of bias. Instead, they used a deductive approach to analyze the content of four instruments to determine the usability and acceptability in clinical practice. These four instruments included the GSF-PIG,34 the NECPAL CCOMS-ICO,36 the SPICT,35 and the RADPAC.39 They concluded that there is limited evidence that these instruments appropriately identify patients in need of PC early in their illness trajectory. None of the presented instruments distinguishes patients in need of general versus specialized PC.
In conclusion, neither of the two identified reviews included a systematic appraisal of the psychometric properties of the selected instruments. This justifies the need to perform a new systematic review with the following aim: to provide a comprehensive overview of the psychometric properties of available clinician-reported instruments developed to identify patients in need of PC regardless of the care setting — except for emergency departments and intensive care units, and nursing homes.
Review question
What are the measurement properties of instruments that allow the identification of adult patients in need of PC and that can be used in all acute care settings except for emergency departments, and intensive care units?
Inclusion criteria
Participants
The COSMIN guidelines for systematic reviews of measurement properties recommend the following inclusion criteria: i) the instrument should aim to measure the construct of interest (identification of patients in need of PC), ii) the study should concern the target population of interest (adult patients in need of PC), iii) the study should concern the type of measurement instrument of interest (ClinROMs), and iv) the aim of the study should be the development of a measurement instrument or the evaluation of one or more of its measurement properties.44
This review will consider studies that include adult patients with life-limiting cancer or non-cancer illness (e.g. chronic, progressive, incurable illness likely to cause death), in PC or in end-of-life care situations. Patients will need to be hospitalized in any acute-care settings, except emergency departments, or intensive care units. This review will consider studies involving palliative care, adult patients (>18 years), all pathologies (cancer/non-cancer), acute-care settings, instruments/tools for identification, and that report on the development of instruments or one or more measurement properties of instruments: content validity, reliability, responsiveness.
This review will exclude studies involving disabled persons, intensive care units, emergency departments, nursing homes, caregivers, pediatric patients, neonates, prognosis instruments, the surprise question only, biomarkers, and instruments that are used as an outcome measurement with no measurement properties reported.
Instrument and construct
This review will consider studies presenting any measurement property of ClinROMs, allowing the identification of patients in need of generalized or specialized PC (construct of interest). As those instruments have to be used by professionals other than physicians, we will include instruments comprising general indicators of decline and frailty, but exclude biological markers that require specific investigation or examination (e.g. serum albumin). Due to the need to identify patients in need of PC as early as possible, prognostic instruments will be excluded.
Outcomes
The outcomes will include the measurement properties of the assessed instruments and refer to the psychometric properties of the identification instruments.
The COSMIN proposes a guideline for systematic reviews of measurement properties for patient-reported outcome measures (PROMs).44 As this guideline includes criteria that are not relevant for ClinROMs (e.g. box 2a-2c content validity), only the relevant criteria will be used in this review. The following 10-step recommendations and instruments will be used. Steps 1 to 4 are standard procedures when performing systematic reviews. Steps 5 to 7 concern the evaluation of the measurement properties of the included instrument. Steps 8 to 10 concern the evaluation of the interpretability and feasibility of the instruments and the reporting of the instruments included in the systematic review. The COSMIN risk of bias checklist will be followed.45 This review will focus on the development of instruments and mainly on their measures of validity and reliability.
The development of instruments is evaluated through questions about the construct, target population, context of use, item generation, data analyzed, comprehensibility and comprehensiveness, and how the instrument was tested.46
Psychometric properties of the instruments are assessed through several types of validity. For this review, the COSMIN group taxonomy measurement will be used.
Measures of validity refer to the degree to which an instrument measures the construct that it has to measure. They include:
- Content validity: the degree to which the content of an instrument is an adequate reflection of the construct to be measured, including face validity. Content validity is evaluated by relevant items for the construct, target population, comprehensiveness of the instrument, and by the Content Validity Index.
- Construct validity: the degree to which the scores of an instrument are consistent with hypotheses based on the assumption that the instrument measures the construct to be measured. It also includes the structural validity (the degree to which an instrument adequately captures the dimensionality of the construct that must be measured). Construct validity is evaluated by factor analysis.
- Criterion validity: the degree to which the scores of an instrument are an adequate reflection of a gold standard criterion.45 Criterion validity is evaluated by the correlation with the gold standard and will only be considered if the gold standard is in accordance with COSMIN guidelines.44
Measures of reliability refer to the extent to which scores for patients who have not changed are the same for repeated measurement under several conditions. They include:
- Internal consistency: the degree of the interrelatedness among the items. It is measured by Cronbach's alpha or KR-20.
- Reliability: the proportion of the total variance in the measurements that is due to true differences between patients. It is assessed by intra-class correlation coefficients, weighted or un-weighted Kappa statistics, and standard error of measurement.
- Measurement error: the systematic and random error of a patient's score that is not attributable to true changes. It is measured by the standard error of measurement, smallest detectable change, limits of agreement, or a percentage of agreement.45
When applicable, responsiveness — the ability to detect change over time in the construct to be measured — will be assessed using the following criteria: absolute or relative correlations or differences of the change scores, area under the receiver operating curve, or sensitivity and specificity.
Types of studies
Any quantitative study design will be considered for inclusion in this systematic review of measurement properties; however, the included studies will be those that report on the development and/or validation of measurement instruments as described above. Studies published in English and French will be included.
Methods
The proposed systematic review will be conducted in accordance with the COSMIN methodology for systematic reviews of measurement properties for PROMs.47 The title has been registered in PROSPERO (CRD42020150074).
Search strategy
The search strategy will aim to locate published and unpublished studies. It will be based on the COSMIN recommendations and use the precise search filter for measurement properties to capture relevant measurement properties.48 The systematic literature search will be performed in collaboration with a librarian. An initial limited search of MEDLINE and CINAHL will be conducted, followed by an analysis of text words contained in the title and abstract, and of the index terms used to describe the article. A draft search strategy for MEDLINE Ovid SP is provided in Appendix I.
The search will be conducted in four databases, and the search strategy will be adapted to the syntax and subject headings of each database. No date restrictions will apply. Finally, the reference lists of all studies selected for critical appraisal will be screened for additional studies.
Information sources
The bibliographic databases to be searched will include Embase, MEDLINE (Ovid SP), PubMed (NOT medline[sb]), and CINAHL (EBSCO). The search for unpublished studies will include Google Scholar, government websites (e.g. National Institute of Nursing Research), hospice websites, the Library Network of Western Switzerland, and WorldCat. We will also contact researchers who have recently published a paper on this topic, to ensure that we are not missing on some unpublished work that could be included in this review.
Study selection
As recommended by PRISMA,43 following the search, all identified citations will be collated and uploaded into EndNote X8 (Clarivate Analytics, PA, USA) and duplicates removed. Titles and abstracts will then be screened by two independent reviewers (FTL and ASR) for assessment against the inclusion criteria for the review, using Rayyan software (Qatar Computing Research Institute, Doha, Qatar), which allows blinded assessment.49 The full text of selected citations will be assessed in detail against the inclusion criteria by two independent reviewers (FTL and ASR). Reasons for exclusion of full-text studies that do not meet the inclusion criteria will be recorded and reported in the systematic review. Any disagreements that arise between the reviewers at each stage of the study selection process will be resolved with a third reviewer (CMA) and through discussion. The results of the search will be reported in full in the final systematic review and presented in a PRISMA flow diagram.50
Assessment of methodological quality
Eligible studies will be critically appraised by two independent reviewers for methodological quality using the standardized critical appraisal instrument from COSMIN methodology and instruments.46 According to the COSMIN Methodology for Systematic Reviews of Patient-Reported Outcome Measures (PROMs) user manual, “The result of each study on a measurement property should be rated against the updated criteria for good measurement properties. Each result is rated as either sufficient (+), insufficient (–), or indeterminate.”47(p.28) Any disagreements that arise will be resolved with a third reviewer and through discussion. Following critical appraisal, the decision to exclude MEDLINE studies will be based on the COSMIN recommendations.
Data extraction
Two independent reviewers (FTL and ASR) will extract data from studies included in the review, using the modified pilot tested data extraction form (Appendix II). The data extracted will include specific details about the tests, populations, study methods, and outcomes of significance to the review question and specific objectives. Any disagreements that arise between the reviewers will be resolved through discussion, and if no consensus can be found, a third reviewer (CMA) will be involved. Authors of papers will be contacted to request missing or additional data where required.
Data synthesis
The results will be quantitatively pooled or qualitatively summarized. They will be reported in a table with the rating of the pooled or summarized results and the grading of the quality of evidence (high, moderate, low, very low): “If possible, the results from different studies on one measurement property should be statistically pooled in a meta-analysis. Pooled estimates of measurement properties can be obtained by calculating weighted means (based on the number of participants included per study) and 95% confidence intervals.”47(p.31) This meta-analysis will be performed by a statistician. The strategy for meta-analysis will be based on the COSMIN guide.46 Where statistical pooling is not possible the findings will be presented in narrative form including tables and figures to aid in data presentation, where appropriate.
Assessing certainty in the findings
Grading of the quality will be based on a modified GRADE approach, where the quality of the evidence is graded as high, moderate, low, or very low evidence.45 “For evaluating measurement properties of systematic reviews of PROMs, the following four factors should be taken into account: (1) risk of bias (i.e. the methodological quality of the studies), (2) inconsistency (i.e. unexplained inconsistency of results across studies), (3) imprecision (i.e. total sample size of the available studies), and (4) indirectness (i.e. evidence from different populations than the population of interest in the review).”47(p.32) A Summary of Findings will be created using GRADEpro GDT software (McMaster University, ON, Canada). Information about the general characteristics of the included studies will be presented as well as the psychometric properties of each instrument.
Funding
This systematic review is part of a PhD thesis by the first author (FTL). The PhD candidate received a grant from the Swiss Academy of the Medical Sciences (SAMS) and is supported by the Public Health Service of the Canton de Vaud of Switzerland.
Appendix I: Search strategy for MEDLINE (Ovid)
Search conducted on 28 March 2019.
1427 records retrieved.
(exp palliative care/ OR terminal care/ OR exp terminally ill/ OR exp palliative medicine/ OR exp hospice and palliative care nursing/ OR (palliative OR “terminally ill” OR (terminal ADJ1 (care OR disease∗ OR patient∗))).ab,ti,kf.) AND (exp patient selection/ OR exp transitional care/ OR referral and consultation/ OR needs assessment/ OR (identification OR identify OR identifying OR referral OR (patient∗ ADJ3 selection∗) OR (transition∗ ADJ3 care) OR (assessment∗ ADJ6 (need OR needs))).ab,ti,kf.) AND (surveys and questionnaires/ OR (tool OR tools OR questionnaire∗ OR instrument OR instruments OR scale OR scales OR (surprise∗ ADJ3 question∗) OR “gold standards framework” OR “gold standard framework” OR NECPAL OR SPICT OR RADPAC OR HR-PRO OR NAT OR PC-NAT OR GSF OR CriSTAL OR SPICT OR DanPaCT OR P-caREs).ab,ti,kf.)
Appendix II: Data extraction instrument
References
1. To TH, Greene AG, Agar MR, Currow DC. A point prevalence survey of hospital inpatients to define the proportion with palliation as the primary goal of care and the need for specialist palliative care.
Intern Med J 2011; 41 (5):430–433.
2. van Niekerk L, Raubenheimer PJ. A point-prevalence survey of public hospital inpatients with palliative care needs in Cape Town, South Africa.
S Afr Med J 2014; 104 (2):138–141.
3. Gott M, Frey R, Raphael D, O’Callaghan A, Robinson J, Boyd M. Palliative care need and management in the acute hospital setting: a census of one New Zealand Hospital.
BMC Palliat Care 2013; 12:15.
4. Zertuche-Maldonado T, Tellez-Villarreal R, Pascual A, Valdovinos-Chavez SB, Barragan-Berlanga AJ, Sanchez-Avila MT, et al. Palliative care needs in an acute internal medicine ward in Mexico.
J Palliat Med 2018; 21 (2):163–168.
5. Meffert C, Rucker G, Hatami I, Becker G. Identification of hospital patients in need of palliative care — a predictive score.
BMC Palliat Care 2016; 15:21.
6. Morin L, Aubry R, Frova L, MacLeod R, Wilson DM, Loucka M, et al. Estimating the need for palliative care at the population level: a cross-national study in 12 countries.
Palliat Med 2016; 31 (6):526–536.
7. Murtagh FE, Bausewein C, Verne J, Groeneveld EI, Kaloki YE, Higginson IJ. How many people need palliative care? A study developing and comparing methods for population-based estimates.
Palliat Med 2014; 28 (1):49–58.
8. Office fédéral de la santé publique, Conférence suisse des directrices et directeurs cantonaux de la santé [National strategy of palliative care 2013-2015]. Berne: OFSP et CDS; 2012 [cited 2019 Apr 4]. Available from:
https://www.bag.admin.ch/bag/fr/home/das-bag/publikationen/broschueren/publikationen-im-bereich-palliative-care/strategie-nat-soins-palliatifs.html. French.
9. Pollock K. Is home always the best and preferred place of death?
BMJ 2015; 351:h4855.
10. Burge F, Lawson B, Johnston G, Asada Y, McIntyre PF, Flowerdew G. Preferred and actual location of death: what factors enable a preferred home death?
J Palliat Med 2015; 18 (12):1054–1059.
11. Arnold E, Finucane AM, Oxenham D. Preferred place of death for patients referred to a specialist palliative care service.
BMJ Support Palliat Care 2015; 5 (3):294–296.
12. Murtagh FE, Addington-Hall J, Edmonds P, Donohoe P, Carey I, Jenkins K, et al. Symptoms in the month before death for stage 5 chronic kidney disease patients managed without dialysis.
J Pain Symptom Manage 2010; 40 (3):342–352.
13. Manitta V, Zordan R, Cole-Sinclair M, Nandurkar H, Philip J. The symptom burden of patients with hematological malignancy: a cross-sectional observational study.
J Pain Symptom Manage 2011; 42 (3):432–442.
14. Penders YW, Onwuteaka-Philipsen B, Moreels S, Donker GA, Miccinesi G, Alonso TV, et al. Differences in primary palliative care between people with organ failure and people with cancer: an international mortality follow-back study using quality indicators.
Palliat Med 2018; 32 (9):1498–1508.
15. Rosenwax L, Spilsbury K, Arendts G, McNamara B, Semmens J. Community-based palliative care is associated with reduced emergency department use by people with dementia in their last year of life: a retrospective cohort study.
Palliat Med 2015; 29 (8):727–736.
16. Seow H, Brazil K, Sussman J, Pereira J, Marshall D, Austin PC, et al. Impact of community based, specialist palliative care teams on hospitalisations and emergency department visits late in life and hospital deaths: a pooled analysis.
BMJ 2014; 348:g3496.
17. Dalgaard KM, Bergenholtz H, Nielsen ME, Timm H. Early integration of palliative care in hospitals: a systematic review on methods, barriers, and outcome.
Palliat Support Care 2014; 12 (6):495–513.
18. Lindsay J, Dooley M, Martin J, Fay M, Kearney A, Barras M. Reducing potentially inappropriate medications in palliative cancer patients: evidence to support deprescribing approaches.
Support Care Cancer 2014; 22 (4):1113–1119.
19. Greer JA, Jackson VA, Meier DE, Temel JS. Early integration of palliative care services with standard oncology care for patients with advanced cancer.
CA Cancer J Clin 2013; 63 (5):349–363.
20. Temel JS, Greer JA, El-Jawahri A, Pirl WF, Park ER, Jackson VA, et al. Effects of early integrated palliative care in patients with lung and GI cancer: a randomized clinical trial.
J Clin Oncol 2017; 35 (8):834–841.
21. Afshar K, Geiger K, Muller-Mundt G, Bleidorn J, Schneider N. Generalist palliative care for non-cancer patients: a review article.
Schmerz 2016.
22. Brown CR, Hsu AT, Kendall C, Marshall D, Pereira J, Prentice M, et al. How are physicians delivering palliative care? A population-based retrospective cohort study describing the mix of generalist and specialist palliative care models in the last year of life.
Palliat Med 2018; 32 (8):1334–1343.
23. von Wartburg l. Soins palliatifs généraux [General palliative care]. Recommandations et instruments de mise en œuvre. Berne: Office fédéral de la santé publique OFSP, Conférence suisse des directrices et directeurs cantonaux de la santé CDS & General palliative care 2015; [cited 2019 Apr 4]. Available from:
https://www.bag.admin.ch/bag/fr/home/strategie-und-politik/nationale-gesundheitsstrategien/strategie-palliative-care/grundlagen-zur-strategie-palliative-care/allgemeine-palliative-care.html. French.
24. Gomez-Batiste X, Caja C, Espinosa J, Bullich I, Martinez-Munoz M, Porta-Sales J, et al. The Catalonia World Health Organization demonstration project for palliative care implementation: quantitative and qualitative results at 20 years.
J Pain Symptom Manage 2012; 43 (4):783–794.
25. World Health Organization. WHO definition of palliative care [Internet]. 2002 [cited 2019 Apr 10]. Available from:
https://www.who.int/cancer/palliative/definition/en/.
26. Boyd K, Murray SA. Recognising and managing key transitions in end of life care.
BMJ 2010; 341:c4863.
27. Mitchell H, Noble S, Finlay I, Nelson A. Defining the palliative care patient: its challenges and implications for service delivery.
BMJ Support Palliat Care 2013; 3 (1):46–52.
28. Van Mechelen W, Aertgeerts B, De Ceulaer K, Thoonsen B, Vermandere M, Warmenhoven F, et al. Defining the palliative care patient: a systematic review.
Palliat Med 2013; 27 (3):197–208.
29. Glare P, Plakovic K, Schloms A, Egan B, Epstein AS, Kelsen D, et al. Study using the NCCN guidelines for palliative care to screen patients for palliative care needs and referral to palliative care specialists.
J Natl Compr Canc Netw 2013; 11 (9):1087–1096.
30. Lamba S. Early goal-directed palliative therapy in the emergency department: a step to move palliative care upstream.
J Palliat Med 2009; 12 (9):767.
31. Richardson P, Greenslade J, Shanmugathasan S, Doucet K, Widdicombe N, Chu K, et al. PREDICT: a diagnostic accuracy study of a tool for predicting mortality within one year: who should have an advance healthcare directive?
Palliat Med 2015; 29 (1):31–37.
32. Boland JW, Reigada C, Yorke J, Hart SP, Bajwah S, Ross J, et al. The adaptation, face, and content validation of a needs assessment tool: progressive disease for people with interstitial lung disease.
J Palliat Med 2016; 19 (5):549–555.
33. Cardona-Morrell M, Hillman K. Development of a tool for defining and identifying the dying patient in hospital: Criteria for Screening and Triaging to Appropriate aLternative care (CriSTAL).
BMJ Support Palliat Care 2015; 5 (1):78–90.
34. Thomas K, Noble B. Improving the delivery of palliative care in general practice: an evaluation of the first phase of the Gold Standards Framework.
Palliat Med 2007; 21 (1):49–53.
35. Highet G, Crawford D, Murray SA, Boyd K. Development and evaluation of the Supportive and Palliative Care Indicators Tool (SPICT): a mixed-methods study.
BMJ Support Palliat Care 2014; 4 (3):285–290.
36. Gomez-Batiste X, Martinez-Munoz M, Blay C, Amblas J, Vila L, Costa X, et al. Identifying patients with chronic conditions in need of palliative care in the general population: development of the NECPAL tool and preliminary prevalence rates in Catalonia.
BMJ Support Palliat Care 2013; 3 (3):300–308.
37. Maas EA, Murray SA, Engels Y, Campbell C. What tools are available to identify patients with palliative care needs in primary care: a systematic literature review and survey of European practice.
BMJ Support Palliat Care 2013; 3 (4):444–451.
38. Walsh R, Mitchell G, Francis L, van Driel L. What diagnostic tools exist for the early identification of palliative care patients in general practice: a systematic review.
J Palliat Care 2015; 31 (2):118–123.
39. Thoonsen B, Engels Y, van Rijswijk E, Verhagen S, van Weel C, Groot M, et al. Early identification of palliative care patients in general practice: development of RADboud indicators for PAlliative Care Needs (RADPAC).
Br J Gen Pract 2012; 62 (602):e625–e631.
40. Grbich C, Maddocks I, Parker D, Brown M, Willis E, Piller N, et al. Identification of patients with noncancer diseases for palliative care services.
Palliat Support Care 2005; 3 (1):5–14.
41. Rainone F, Blank A, Selwyn PA. The early identification of palliative care patients: preliminary processes and estimates from urban, family medicine practices.
Am J Hosp Palliat Care 2007; 24 (2):137–140.
42. McDaid P. A quick guide to identifying patients for supportive and palliative care [Internet]. NHS; 2011 [cited 2019 Apr 20]. Available from:
http://www.cheshire-epaige.nhs.uk/wp-content/uploads/2018/11/A-Quick-Guide-to-Identifying-Patients-for-Supportive-and-Palliative-Care.pdf.
43. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration.
PLoS Med 2009; 6 (7):e1000100.
44. Prinsen CAC, Mokkink LB, Bouter LM, Alonso J, Patrick DL, de Vet HCW, et al. COSMIN guideline for systematic reviews of patient-reported outcome measures.
Qual Life Res 2018; 27 (5):1147–1157.
45. Mokkink LB, de Vet HCW, Prinsen CAC, Patrick DL, Alonso J, Bouter LM, et al. COSMIN Risk of Bias checklist for systematic reviews of Patient-Reported Outcome Measures.
Qual Life Res 2018; 27 (5):1171–1179.
46. Terwee CB, Prinsen CAC, Chiarotto A, de Vet HC, Bouter LM, Alonso J, et al. COSMIN methodology for assessing the content validity of PROMs. Amsterdam; 2018 [cited 2019 Apr 20]. Available from:
https://www.cosmin.nl/wp-content/uploads/COSMIN-methodology-for-content-validity-user-manual-v1.pdf
47. Mokkink LB, Prinsen CAC, Patrick DL, J. A, Bouter LM, De Vet HC,
et al. COSMIN methodology for systematic reviews of Patient-Reported Outcome Measures (PROMs). Amsterdam; 2018 [cited 2019 Apr 4]. Available from:
https://www.cosmin.nl/wp-content/uploads/COSMIN-syst-review-for-PROMs-manual_version-1_feb-2018.pdf
48. Terwee CB, Jansma EP, Riphagen II, de Vet HC. Development of a methodological PubMed search filter for finding studies on measurement properties of measurement instruments.
Qual Life Res 2009; 18 (8):1115–1123.
49. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews.
Syst Rev 2016; 5 (1):210.
50. Moher D, Liberati A, Tetzlaff J, Altman DG. the PRISMA Group. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement.
PLoS Med 2009; 6 (7):e1000097.
