Secondary Logo

Journal Logo

ORIGINAL RESEARCH

Unfolding and characterizing the barriers and facilitators of scaling-up evidence-based interventions from the stakeholders’ perspective: a concept mapping approach

Zhang, Xiaoyan MMed1; Zhao, Junqiang MMed2; Li, XueJing MMed3; Yan, Lijiao MMed3; Hao, Yufang PhD4,∗; Liu, Jianping PhD5,∗

Author Information
JBI Evidence Implementation: June 2022 - Volume 20 - Issue 2 - p 117-127
doi: 10.1097/XEB.0000000000000305

Abstract

What is known about the topic?

  • Stroke patients need continuous community-based care for dysphagia.
  • Nurse-led dysphagia identification and management can increase screening frequency and accuracy, thereby effectively lowering the risk of patients’ chest infections.
  • Successful scaling-up of evidence-based interventions in a new health care delivery system is related to innovation attributes, implementer attributes, and community attributes.

What does this article add?

  • There are many barriers and facilitators affecting the EBP-PSDIM program scaling-up from the stakeholders’ perspective, including community, resource team, evidence-based practice program, and scaling-up strategy-related factors.
  • Factors involved in scaling-up the EBP-PSDIM program were initially validated as being multidimensional and conceptually distinct.
  • The ‘perceived needs of the community’ was rated as the most important and feasible factor to address, while ‘costs/resource mobilization’ was rated as the least important and feasible factor in the EBP-PSDIM program.

Introduction

Although the movement towards evidence-based interventions (EBIs) has led to variable success, uneven uptake of research findings and the resulting inappropriate care occur across clinical settings. Implementation science promotes the systematic uptake of evidence-based practices into routine practice, which focuses on the challenges of scaling-up, sustainability, replication, program integration, equitability, and real-world effectiveness.1,2 Among these challenges, EBI scaling-up has recently attracted much attention in literature. Due to the lack of practice and theories, EBI scalingup has not been highly implemented, and therefore, remains one of the challenges most difficult to solve.3 The term scaling-up is now frequently used in implementation science literature but there is need for a formal and well-accepted definition. In this study, scaling-up was defined as expanding EBIs manifested to be efficacious in a specific context to real-world conditions so as to benefit a more eligible population while maintaining effectiveness.4 Existing theories, frameworks, or guides suggest that successful scaling-up of EBIs involves innovation attributes, implementer attributes, community attributes, strategies, the social-political environment, and research background.5–7 Nevertheless, mechanisms and factors for scaling-up EBIs remain unclear, and relatively little is known as to how stakeholders perceive barriers and facilitators to the scalingup of EBIs in empirical research. A stakeholder is an individual, group, and/or organization with a vested interest in deciding to implement best practices. The stakeholder-engaged analysis could also maximize the alignment of stakeholders’ interests with the objectives of guideline practices and could address the risks associated with unsupportive stakeholders.8 In addition, implementation science attends to the effectiveness of implementation strategies. To lay the foundation for research on the effectiveness of implementation strategies, it is important to understand the barriers and facilitators impacting the scaling-up of EBIs.

Aims

This study aimed to identify the barriers and facilitators of scaling-up the EBP-PSDIM program from the stakeholders’ perspective, and to assess their importance and feasibility.

Background

Dysphagia is a common and serious complication after stroke, which affects up to 78% of stroke survivors.9 Dysphagia may result in poor outcomes in stroke patients. The dysphagia patients are almost five times more likely to develop aspiration pneumonia than nondysphagia patients.10 Dysphagia is also considered an independent risk factor for prolonged hospital stay and hospitalization.11 However, early nurse-led identification and management of dysphagia should be prioritized as this could significantly reduce complications and improve patients’ quality of life.12 Even though multiple authoritative evidence-based guidelines are available and clearly recommend standard screening tools, treatment, and nursing management for patients with dysphagia, which could be adopted as a basis for clinical decision-making, clinical audits have demonstrated a huge gap between evidence and clinical practice because of health providers’ lack of awareness, knowledge, and skills.13

To address this gap, we conducted an EBP-PSDIM program development study and a sustained application study in a hospital context, based on best practice guidelines.13,14 The topics of interventions included dysphagia screening, diet management, psychological nursing, rehabilitation exercise, traditional Chinese medicine nursing techniques, nutrition screening, and quality control. As a result, we improved patients’ swallowing function, quality of life, and hospitalization satisfaction. However, these two evidence-based studies were implemented in a hospital context without regard for communities or families.

Owing to the shortage of medical resources and financial burden, about 80% of post-stroke patients return to communities after the condition has stabilized.15 About 54% of stroke patients still experience dysphagia after discharge.16 Patients need continuous community-based care for dysphagia.17 Moreover, as the concept of high-quality nursing deepens, geriatric medical care institutions (including communities) have progressively incorporated dysphagia assessment and care into the stroke quality evaluation system in mainland China.18 Hence, scaling-up the EBP-PSDIM program to the community level should be of high priority to improve the nursing quality and reduce the risks of comorbidity, malnutrition, and mortality. However, relatively little is known as to how stakeholders perceive barriers and facilitators in regards to scaling-up the EBP-PSDIM program. The present study is part of a multiphase program guided by the ExpandNet/WHO framework for scaling-up,5 with the overarching goal to improve the outcome of dysphagia, nursing service quality, and organizational evidence-based culture construction in communities. This study adopted a concept mapping approach to examine the stakeholders’ perspectives on barriers and facilitators of scaling-up the EBP-PSDIM program, and to assess their importance and feasibility.19

Methods

Design

A concept mapping approach was used. Concept mapping is a normalized, structured, stakeholder-engaged mixed-methods approach, which has been widely used for organizing stakeholders’ ideas,20–22 following classical six steps of preparation, brainstorming, sorting and rating, analysis, interpretation, and utilization. It is a hybrid research design (qualitative procedures to produce data and quantitative methods to analyze data), where opinions on a topic of interest are integrated and both visual and graphic statements expressed by the participants are created through modern multivariate statistical analytical techniques, such as hierarchical cluster analysis (HCA) and multidimensional scaling (MDS). This approach enables intuitively representing the relationships among a set of related concepts, empirically clustering these concepts into conceptually distinct categories, and rating them on multiple dimensions.

Participants and setting

A purposive sampling procedure was used so as to achieve a heterogeneous sample that represented different positions and views related to the EBP-PSDIM program in March 2019. In principle, there is no strict limit on the number of individuals participating in the concept mapping approach, and there is no specific statistical method for strictly calculating the sample size. Most studies considered that at least 15 participants should be involved in concept mapping.19 In line with the sample sizes reported in similar studies, we recruited a total of 18 stakeholders for brainstorming. The stakeholders in this study were from three institutions. The roles of the institutions and the inclusion criteria of the stakeholders are shown in Table 1. Among the recruited stakeholders, seven were from the program research institution (Evidence-based Nursing Research Center, Beijing University of Chinese Medicine), involving three experts in evidence-based methodologies, one nursing education expert, two evidence-based researchers, and one psychologist; five were from the previous dysphagia program implementation department (Dongzhimen Hospital affiliated to Beijing University of Chinese Medicine), involving one nursing leader, one nursing manager, and three clinical nurses); and eleven were from the community hospital (Beijing Longfu Hospital), including one doctor, one dietitian, two nursing managers, one nursing educator, three clinical nurses, and three family members of patients. Most stakeholders participating in the brainstorming session were women (77.8%), with an average age of 38.83 years (SD = 11.74). Most of them had a master's degree or higher. Nearly 64% of nonpatient caregiver stakeholders had direct experience in implementing an evidence-based practice program.

Table 1 - Composition, roles, and inclusion criteria for stakeholders
Stakeholder institutions Roles Inclusion criteria
The program research institutionThe previous dysphagia program implementation departmentThe community Co-responsible for the EBP-PSDIM program Methodology, resources, and information supportCo-participate in EBP-PSDIM program Implementary experience, training supportCo-responsible for the EBP-PSDIM program User organization Associate professor or above; master's degree or above Participated in at least one evidence-based program Great experience in their fieldOver 3 years of clinical experience (working full-time) Participated in the previous dysphagia programCommunity medical staff Over 3 years of clinical experience (working full-time) ExperiencedFamily members of patients Bachelor's degree or above Primary caregiver Over 3 months of care experience

Data collection

Brainstorming

The focus statement constituted the prompt for the brainstorming session to generate statements in the present study. The brainstorming session involved three roles: a leader, a recorder, and group members (stakeholders). The leader introduced the EBP-PSDIM program, presided over the ground rules, and utilized the following focus prompt to stimulate discussion about the factors influencing the scalability of the EBP-PSDIM program: ‘What are the factors (barriers or facilitators) that influence the acceptance and adoption of interventions of the EBP-PSDIM program in the community?’. The group members offered their opinions on the topic of the meeting, followed by discussion. The relevant statements had to be concise and specific, without being biased in favor of a certain solution or excluding creative ideas. The recorder noted each of the barriers and facilitators. The brainstorming steps generated 94 statements. The research team eliminated duplicate statements, edited statements for clarity, and synthesized similar statements. The final list of statements included 61 unique barriers and facilitators that reflected stakeholders’ perceptions of the factors influencing the scaling-up.

Sorting and rating

An unstructured card sorting process was used to ascertain the association between the statements. The stakeholders completed sorting and rating independently, and the statements were placed on separate cards and reordered at random. Subsequently, the stakeholders organized the cards into themes that made sense to them. In this process, each card could notably be placed in only one category, not two categories at the same time, and all the cards could not be placed in a single category. Further, the stakeholders were requested to rank each of the statements on a five-point scale in terms of their importance to research questions and feasibility to address. Importance ratings ranged from 1 (not at all important) to 5 (extremely important) and feasibility ratings ranged from 1 (not at all feasible) to 5 (not at all feasible). Fifteen stakeholders (83.3%) completed the sorting and rating stage of the present study, while three family members of patients (16.7%) were unable to finish the sorting and rating stage because of poor understanding of the EBP-PSDIM program.

Data analysis

To group the statements and generate a visual display of how the items clustered across all stakeholders, the data from the aforementioned card sorting of each participant were entered into the row column matrix. In this square symmetric similarity matrix, the value was either zero or one. A’1’indicated that a stakeholder had put the row column statements in one category, and A ‘0’ signified that the row column statements were not placed in a category, thus forming a binary symmetric similarity matrix. Next, the matrices of all the stakeholders were summed to obtain the overall combined classification matrix. All the cells in this matrix were integer values ranging from 0 to the total number of stakeholders. A high value in the matrix demonstrated that more stakeholders put the pair of statement sentences in one category, implying to a certain degree that two statements had similarities in concepts. This matrix was also regarded as the relational structure of the concept domain.19 As this combined classification matrix had too many ‘0’ values, to reduce the occurrence of statistical error, the matrix was converted into a correlation matrix by employing the Ochiai coefficient.23 Then, the value in the correlation matrix was subtracted from 1 to obtain the dissimilar matrix. Thereafter, SPSS 22.0 for HCA and MDS was utilized for statistical analysis of dissimilar matrices. The average ratings of importance and feasibility were also calculated and reported for each barrier and facilitator and each cluster to provide priority factors for future implementation strategies.

Interpretation

In the interpretation phase, owing to the differences in specific topics, knowledge bases, desired levels of specificity, and context, there was no available mathematical criterion for identifying the final number of clusters. Hence, three experts in evidence-based program planning were asked to interpret the final number of clusters based on the statistical analysis results and their professional knowledge; they were asked to identify an appropriate name for each final cluster. This approach of determining the number of clusters to use in the final representation is recommended by concept mapping.8,24 Additionally, Go-zone was performed to demonstrate the priority of each factor in a matrix.25 On the basis of the respective average rating, the Go-zone matrix assigned x-axis and y-axis to stakeholders’ ratings of importance and feasibility. The lines divided the graph into four quadrants in the matrix, which corresponded to the mean rating of each axis (low-low, low-high, high-low, high-high). Of the four quadrants, the statements in the top right quadrant were considered highly important and feasible while the bottom left quadrant statements were regarded as relatively less important and not feasible to the scaling-up of the EBP-PSDIM program. Following the assumption that high ratings were ideal, the top right quadrant (high-high quadrant) was considered as the Go-zone as it contained the statements rated highest in importance and feasibility.

Ethics

The study conformed with the principles outlined in the Declaration of Helsinki and was approved by the Ethics Committee of Dongzhimen Hospital affiliated to Beijing University of Chinese Medicine (DZMEC-KY-2019-175). All stakeholders provided informed consent before participating and were informed that refusing to participate or withdrawal from the study would not cause any consequences.

Results

Cluster descriptions

On the basis of the results of HCA and MDS, the ExpandNet/WHO framework for scaling-up, and specific connotations of clusters, three experts involved in evidence-based program planning first classified and named 61 factors into 10 second-level clusters. Subsequently, a total of 10 second-level clusters were further integrated into four primary clusters that together unfolded and characterized the way stakeholders understood the barriers and facilitators likely to influence the scaling-up of the EBP-PSDIM program (Additional file 1, https://links.lww.com/IJEBH/A72). The concept map with the cluster labels is shown in Figs. 1 and 2, which illustrate the relationships among the clusters. Each statement is presented with a corresponding number on the maps. The stress value of MDS is 0.347, highlighting that the final map was acceptable.19

F1
Figure 1:
Hierarchical cluster analysis showing 10 second-level clusters of 61 statements.
F2
Figure 2:
Cluster map from multidimensional scaling and interpretation (four primary clusters).

Cluster ratings

Additional file 1, https://links.lww.com/IJEBH/A72 presents the mean ratings ofeach statement and cluster, and Fig. 3 provides the Go-zone matrix for all the statements influencing the scaling-up implementation of the evidence-based practice of identifying and managing post-stroke dysphagia. Although the stakeholders generally agreed on the importance and feasibility of the generated statements, among all second-level clusters, cluster 2, that is, ‘perceived needs of the community’, was rated the most important and most feasible to address, whereas cluster 9, that is, ‘costs/resource mobilization’, was rated the least important and feasible one. The Go-zone matrix created four different zones depending on whether the statements were rated above or below the mean for importance and feasibility: low importance-low feasibility, low importance-high feasibility, high importance-low feasibility, and high importance-high feasibility. Of all the statements, factors rated with the highest importance included factor 45 (‘shortage of human resource and heavy workload in nursing’) and factor 23 (‘the community nurses lack knowledge and skills to identify and manage dysphagia’). Factor 37 (‘the community will have no significant personnel changes or institutional reforms in the next three years’) was the least important and the least feasible to address.

F3
Figure 3:
Go-zone plot for all 61 statements based on ratings.

Discussion

Translation of EBIs into daily clinical practice has historically been challenging, which has led to a consistently large gap between research and practice. Implementation research addresses the effective implementation strategies to improve distribution and receipt of EBIs, incorporate EBIs into care delivery, and de-implement practices that are no longer effective or were never effective. Successful implementation strategies are tailored to perceived local barriers or facilitators.26,27 To the best of our knowledge, the present study is the first to explore the barriers and facilitators likely to influence EBI scaling-up from the stakeholders’ perspective in mainland China. By adopting a stakeholder-engaged, multistep, mixed-methods approach, a total of 61 statements were generated and grouped into four primary clusters. These factors were conceptually distinct and included community-related factors, resource team-related factors, evidence-based practice program-related factors, and scaling-up strategy-related factors. Our findings offer practical experience for EBIs scaling-up and may further facilitate the development of the situation-specific factor framework and intervention scalability assessment tool for scaling-up in implementation science. Additionally, the Go-zone quadrants premised on the importance and feasibility ratings of factors could be employed to help decision-makers to prioritize factors to be considered when developing targeted implementation strategies.

Community-related factors: salient factors from the community (that is, user organization) cluster included implementation capacities and the perceived needs of the community. EBI scaling-up must take into account the reality and must address persistent or sharply felt problems of the user organization.5 The stakeholders cited satisfying needs as one of the major factors for the successful implementation of EBIs and the cluster rating results indicated that the needs of the community cluster were rated as the most important and most feasible to address. This suggests that the needs should be the source and motivation for the introduction of the program. In particular, EBIs will often have a certain impact on existing work processes and systems, and will also effectuate changes in thinking and ideas. Successful scaling-up requires realistic needs and implementation capabilities.28 This directly affects the extent to which nurses accept and adopt the EBIs. Studies have verified that successful scaling-up can be facilitated when members of the user organization realize the necessity of EBIs and are motivated to implement them.6 We suggest that researchers and implementers should use reliable and effective tools to assess the clinical needs before scalingup EBIs. Further, the user organization must have the appropriate implementation capabilities, decision-making authority, leadership, and appropriate timing, and circumstances. Hence, through advocacy, ways should be found to strengthen the perceived needs/motivations using both formal and informal channels if necessary. Here, future research regarding scaling-up EBIs should begin in areas where capacity (such as physical facilities and equipment, skills, or leadership) is stronger. This process can maximize opportunities and minimize constraints arising from impending changes, such as major personnel changes and institutional reforms, or improve the attributes of the user organization to enhance the chances of scaling-up success. Moreover, we encourage skill training for organizations with weak implementation capabilities.

Resource team-related factors: this primary cluster included the scale, stability, and necessary skills of the resource team. The resource team that could assume this role formally or informally often refers to individuals and organizations that seek to promote EBIs. In our study, the resource team included researchers, clinical experts, technical experts, pedagogical experts, and other individuals who could be employed. Evidence-based practice is a dynamic, continuous, and complex process; it is influenced by evidence, patient preference, clinical judgment, and context. When the context changes, multiple resources are required to work together and cooperate to promote successful implementation. A major factor in ensuring the success of scaling-up is a strong resource team with stability, credibility, and appropriate skills to support strategic management, training, research, monitoring, and evaluation. Similar results have been found in previous studies.7,28,29

Evidence-based practice program-related factors: noteworthy factors in this primary cluster included the credibility of evidence, the relative advantage of evidence, and ease of transfer/installation. This is consistent with the research results of the study by Zamboni and the ExpandNet guideline.5,29 First, the credibility of the evidence must be verified, for instance, by determining what kind of team developed the evidence and whether the development process was rigorous and cross-disciplinary; whether the evidence has obtained the cooperation or support of an authoritative evidence-based institution; whether it has passed the external validity test and made a tangible change to the organization during the test. The credibility of the evidence is the most crucial factor that determines whether the evidence can be trusted and considered, and it is integral in the scaling-up process. Next, before implementing the scaling-up, clarification is necessary on whether EBIs are more advantageous than the existing methods or other innovative ideas; this is essential to convince potential user organizations that the costs of implementation are beneficial. Moreover, ease of transfer/installation must be confirmed, namely, by determining the knowledge and technical level needed to introduce EBIs; moreover, it should be verified whether it is possible to transform components into a form more compatible with the system without changing the core function. The possibility to effectively introduce the evidence into the new context is closely linked to the ease of transfer/installation.30

Scaling-up strategy-related factors: the final primary cluster included the organizational process, costs/resource mobilization, monitoring, and evaluation. This involved determining the form of the scaling-up, the division of labor among the various stakeholders in the scaling-up, and questions as to who will organize this process and how.31 A good organizational process has been shown to be able to clarify the purpose, avoid blindness, and render the process orderly and incrementally. When multiple user organizations exist, their roles, responsibilities, and ownership must be clearly defined. Further, if the scaling-up of EBIs does not involve the mobilization of numerous resources and decision-makers, the cost of implementation and maintenance will be low. Milat and Kumaranayake reported similar results.4,32 In addition, monitoring and evaluation are the key steps to promote and measure the effectiveness of the scaling-up strategies, including time requirements, full supervision, continuous monitoring, and evaluation measures with appropriate outcomes, such as implementation outcomes, service outcomes, and patient outcomes.33

Moreover, in contrast with previous studies,5,34 other factors that influence EBI scaling-up were not obvious in the present study. These factors include external context (whether the user organization has affiliated organizations that disagree with the implementation of the program). In the present study, as the EBP-PSDIM program involved fewer environments and decision-makers, with only the department director and the head nurse having the right to make decisions, no obvious external environment-related factors were involved. However, if researchers or implementers expand other evidence-based programs, they should comprehensively discuss factors with specific programs, including consideration of factors related to the external environment. In addition, many evidence-based practice studies are one-shot trials with little or no long-term plan. The effectiveness and sustainability of the nurse-led EBP-PSDIM program has already been addressed in previous studies,13,14 while here we discussed the aspect of scaling-up. We suggest that researchers and practitioners pay attention to the continuity of evidence-based practice research to identify long-term effects.35–37

Limitations

The present study had several notable limitations. Although representatives of stakeholders at different organizations and levels were invited, the concept-mapping processing was intensive and required several hours of participation, which may have deterred some stakeholders from participating. In addition, this study focused on a single point of scaling-up, and it may not be applicable to cases with multiple user organizations. The present survey results may be restricted in different countries/regions, different medical institutions, and different scaling-up programs owing to the specific influential factors, with their importance and feasibility being affected by the practice programs, the number of user organizations, and health policies. However, this study provides an example of a participatory approach and process that could be replicated in other settings so as increase understanding of the factors likely to influence the scaling-up of the EBP-PSDIM program.

Implications for future research

Despite evidence of the increasing need to address scaling-up EBIs in implementation science, few studies have explored the factors influencing the scaling-up of EBIs in empirical research and even fewer have focused on stakeholders’ perspectives. This study thus provides novel information and implications that are crucial for exploring scaling-up frameworks, developing scalability tools, and developing implementation strategies. From these results, we can see that the factors affecting EBI scaling-up are multidimensional. On the one hand, factors differ in importance and feasibility attributions because of the constraints of specific evidence-based programs. The practical significance lies in that when there are numerous influential factors and limited research resources, time, and costs, researchers could determine priority levels based on factors’ attributions. Future research should identify effective strategies for scaling-up based on these priority barriers and facilitators. On the other hand, factors involve multiple levels, including the community, resource team, evidence-based practice program, and scaling-up strategy-related factors. Most of the available scaling-up frameworks are process frameworks, and the results of this study are beneficial to develop factor frameworks. Future studies need to further explore the interrelationship between different factors, as there may be synergistic effects among factors, and different factors have different effects on scaling-up results. For example, strong leadership and urgent unmet clinical need may improve resource mobilization and investment of funding in scaling-up EBIs, and unreliable EBIs may diminish the advantages.

Conclusion

From the stakeholders’ perspective, factors involved in the EBP-PSDIM program scaling-up were initially validated as being multidimensional and conceptually distinct, including community, resource team, evidence-based practice program, and scaling-up strategy-related factors. The ‘perceived needs of the community’ was rated as the most important and feasible factor to address, while ‘costs/resource mobilization’ was rated as the least important and feasible one in the EBP-PSDIM program. The importance and feasibility ratings of the barriers and facilitators could be used to help decision-makers to prioritize the most appropriate factors to be considered when developing implementation strategies. These findings contribute to recognized gaps in the literature, including offering practical experience for scaling-up EBIs and further supporting the situation-specific theory of the framework for scaling-up in implementation science.

Acknowledgements

We wish to acknowledge Chengwei Shi for statistical analysis, Yuanhong Wang and Jingjing Chai for facilitating access to documents, and Ziyu Tian for paper proofreading. We also thank all stakeholders for their contributions.

Ethics approval and consent to participate: The study conformed with the principles outlined in the Declaration of Helsinki and was approved by the Ethics

Committee of Dongzhimen Hospital affiliated to Beijing University of Chinese Medicine (DZMEC-KY-2019-175). All stakeholders provided informed consent before participating and were informed that refusing to participate or withdrawal from the study would not cause any consequences.

Consent for publication: written informed consent for publication was obtained from all participants.

Availability of data and materials: the datasets used and/or analyzed during the current study are available from the corresponding author upon reasonable request.

Competing interests: the authors declare that they have no competing interests.

Funding: this research was supported by grants from the key program of the National Natural Science Foundation of China (NSFC), 81830115.

Authors’ contributions: Under the supervision of Y.F.H. and J.P.L., X.Y.Z. analyzed and interpreted all the data for this study and wrote the manuscript. Y.F.H., J.P.L., X.Y.Z., J.Q.Z., X.J.L., and L.J.Y. provided information related to the primary study design, implementation of the intervention, and data collection. All authors read and approved the final manuscript.

Conflicts of interest

There are no conflicts of interest.

References

1. Khalil H. Knowledge translation and implementation science: what is the difference? Int J Evid Based Healthc 2016; 14:39–40.
2. MEASURE Evaluation Implementation Research Technical Working Group. Fundamentals of implementation research [PDF file]. 2012. Available at: https://www.measureevalua-tion.org/resources/publications/ms-12-55.html. [Accessed 15 November 2021].
3. Fagan AA, Bumbarger BK, Barth RP, et al. Scaling up evidence-based interventions in US public systems to prevent behavioral health problems: challenges and opportunities. Prev Sci 2019; 20:1147–1168.
4. Milat AJ, King L, Bauman AE, Redman S. The concept of scalability: increasing the scale and potential adoption of health promotion interventions into policy and practice. Health Promot Int 2013; 28:285–298.
5. ExpandNet. World Health Organization. Nine steps for developing a scaling-up strategy [PDF file]. 2010. Available at: http://www.who.int/immunization/hpv/deliver/nine_-steps_for_developing_a_scalingup_strategy_-who_2010.pdf. [Accessed 12 February 2021].
6. Yamey G. Scaling up global health interventions: a proposed framework for success. PLoS Med 2011; 8:e1001049.
7. Milat AJ, Bauman A, Redman S. Narrative review of models and success factors for scaling up public health interventions. Implement Sci 2015; 10:113.
8. Chan CWH, Choi KC, Chan HYL, et al. Unfolding and displaying the influencing factors of advance directives from the stakeholder's perspective: a concept mapping approach. J Adv Nurs 2019; 75:1549–1562.
9. Arnold M, Liesirova K, Broeg-Morvay A, et al. Dysphagia in acute stroke: incidence, burden and impact on clinical outcome. PLoS One 2016; 11:e0148424.
10. Feng MC, Lin YC, Chang YH, et al. The mortality and the risk of aspiration pneumonia related with dysphagia in stroke patients. J Stroke Cerebrovasc Dis 2019; 28:1381–1387.
11. Rofes L, Muriana D, Palomeras E, et al. Prevalence, risk factors and complications of oropharyngeal dysphagia in stroke patients: a cohort study. Neurogastroenterol Motil 2018; e13338[Epub ahead of print].
12. Hines S, Kynoch K, Munday J. Nursing interventions for identifying and managing acute dysphagia are effective for improving patient outcomes: a systematic review update. J Neurosci Nurs 2016; 48:215–223.
13. Gao SQ, Zhao JQ, Guo HL, Tian RX, Hao YF. Dysphagia identification and management in post-stroke patients: an evidence-based nursing practice research. Int J Evid Based Healthcare 2016; 14:189.
14. Zhao JQ, Wang JF, Li XJ, et al. A qualitative study on factors influencing the sustainability of an evidence-based nursing innovation program-dysphagia identification and managementin post-stroke patients. J Nursing Admin 2018; 7:481–485.
15. Li S, Tian Q, Fan J, et al. Hospital use in survivors of transient ischaemic attack compared with survivors of stroke in central China: a nested case-control study. BMJ Open 2019; 9:e024052.
16. Ho YH, Liu HY, Huang ST. The prevalence and signs of dysphagia among stroke patients in rehabilitation units. J Nurs 2014; 61:54–62.
17. Minshall S, Pownall S. Management of swallowing problems in community settings. Br J Community Nurs 2019; 24:323–327.
18. Hou S, Li B, Luo Y, Shang S. Creation and location-specific revision of a core nursing outcomes evaluation system based on nursing outcomes classification for stroke inpatients in China. Int J Nurs Knowl 2019; 30:154–161.
19. Trochim W. An introduction to concept mapping for planning and evaluation. Eval Program Plann 1989; 12:1–16.
20. Lobb R, Pinto AD, Lofters A. Using concept mapping in the knowledge-to-action process to compare stakeholder opinions on barriers to use of cancer screening among South Asians. Implement Sci 2013; 8:37.
21. Green AE, Aarons GA. A comparison of policy and direct practice stakeholder perceptions of factors affecting evidence-based practice implementation using concept mapping. Implement Sci 2011; 6:104.
22. Leyns CC, De Maeseneer J, Willems S. Using concept mapping to identify policy options and interventions towards people-centred health care services: a multi stakeholders perspective. Int J Equity Health 2018; 17:177.
23. Zhou Q, Leydesdorff L. The normalization of occurrence and Co-occurrence matrices in bibliometrics using Cosine similarities and Ochiai coefficients. J Assoc Information Sci Technol 2016; 67:2805–2814.
24. Trochim WM, Milstein B, Wood BJ, Jackson S, Pressler V. Setting objectives for community and systems change: an application of concept mapping for planning a statewide health improvement initiative. Health Promot Pract 2004; 5:8–19.
25. Green AE, Fettes DL, Aarons GA. A concept mapping approach to guide and understand dissemination and implementation. J Behav Health Serv Res 2012; 39:362–373.
26. Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci 2013; 8:139.
27. Reynolds SS, Sova C, McNalty B, Lambert S, Granger B. Implementation strategies to improve evidence-based bathing practices in a neuro ICU. J Nurs Care Qual 2019; 34:133–138.
28. Troup J, Fuhr DC, Woodward A, Sondorp E, Roberts B. Barriers and facilitators for scaling up mental health and psychosocial support interventions in low- and middleincome countries for populations affected by humanitarian crises: a systematic review. Int J Ment Health Syst 2021; 15:5.
29. Zamboni K, Schellenberg J, Hanson C, Betran AP, Dumont A. Assessing scalability of an intervention: why, how and who? Health Policy Plan 2019; 34:544–552.
30. Aarons GA, Sklar M, Mustanski B, Benbow N, Brown CH. ‘Scaling-out’ evidence-based interventions to new populations or new health care delivery systems. Implement Sci 2017; 12:111.
31. L’Engle K, Plourde KF, Zan T. Evidence-based adaptation and scale-up of a mobile phone health information service. Mhealth 2017; 3:11.
32. Kumaranayake L. The economics of scaling up: cost estimation for HIV/AIDS interventions. AIDS 2008; 22:S23–S33.
33. Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health 2011; 38:65–76.
34. Hategeka C, Tuyisenge G, Bayingana C, Tuyisenge L. Effects of scaling up various community-level interventions on child mortality in Burundi, Kenya, Rwanda, Uganda and Tanzania: a modeling study. Glob Health Res Policy 2019; 4:1.
35. Ilott I, Gerrish K, Pownall S, Eltringham S, Booth A. Exploring scale-up, spread, and sustainability: an instrumental case study tracing an innovation to enhance dysphagia care. Implement Sci 2013; 8:128.
36. Blackstone S, Iwelunmor J, Plange-Rhule J, et al. Sustaining nurse-led task-shifting strategies for hypertension control: a concept mapping study to inform evidence-based practice. Worldviews Evid Based Nurs 2017; 14:350–357.
37. Khalil H, Kynoch K. Implementation of sustainable complex interventions in health care services: the triple C model. BMC Health Serv Res 2021; 21:143.

Yufang Hao and Jianping Liu should be considered as joint corresponding authors.

Keywords:

dysphagia; evidence-based intervention; factor; scaling-up; stakeholder; stroke

Supplemental Digital Content

© 2022 JBI. Unauthorized reproduction of this article is prohibited.

A video commentary on implementation project titled: How do health professionals prioritise clinical areas for implementation of evidence into practice? The commentary is provided by Andrea Rochon RN, MNSc, Research Assistant, Queen's University, Ontario, Canada