Secondary Logo

Journal Logo

Feature Articles

The Evaluation of a Palliative Care Advanced Practice Nursing Externship

Gentry, Jennifer H. DNP, ANP-BC, GNP, ACHP, FPCN; Dahlin, Constance MSN, ANP-BC, ACHPN, FPCN, FAAN

Author Information
Journal of Hospice & Palliative Nursing: June 2020 - Volume 22 - Issue 3 - p 172-179
doi: 10.1097/NJH.0000000000000637
  • Free


In 2014, more than 2.6 million Americans died; most of these deaths were attributable to serious illnesses, including heart disease, cancer, lung disease, and dementia.1 With more seriously ill persons, there is a great need for palliative care in a health care system not prepared to provide it.2 Because of a lack of trained palliative care providers, patients may be unable to access expert assistance in symptom management, skilled communication, and advance care planning. This deficiency may lead to reduced quality of care, poorer quality of life, and care that is less congruent with patient goals.2 In general, health care professionals lack the training to provide basic or primary palliative care when specialty palliative care is not needed or available.2 The need for additional palliative care training and skills for health care professionals is further underscored by anticipated shortages of palliative care specialists.3 An estimated shortage of 10 000 hospice and palliative care (HPC) specialists is projected in the coming years.3

One of the proposed solutions to palliative care workforce shortages is to increase the frequency and quality of palliative care education for advanced practice registered nurses (APRNs). In the call for action “Nurses Lead and Transform Palliative Care,”4 recommendations for the provision of quality palliative care to all seriously ill persons, families, and populations can only be achieved by the delivery of primary palliative care from every professional nurse. The authors of the Hospice and Palliative Nurses Association (HPNA) position paper on APRN practice argue that APRNs possess the skills needed and are situated to address the complex needs of persons with life-threatening, progressive illness.5 Current palliative care training is deficient in many health care disciplines, including APRNs.6,7 In addition, APRN students report little exposure to palliative care and express discomfort with discussing or providing palliative care.8 Pawlow et al9 described the current HPC APRN workforce and noted that more than 60% of HPC APRNs had no palliative care content in their graduate programs. Because many of the specialty HPC APRNs received palliative care training through continuing education programs,9 expanded use of intense, short-duration continuing education may be a practical way to improve APRN palliative care knowledge and skill.

Further supporting the need for palliative nursing education, leaders from the HPNA and the American Association of Colleges of Nursing publicized the need to improve palliative care training for undergraduate nurses.10 The authors of the American Nurses Association/HPNA 2017 call for action, “Nurses Lead and Transform Palliative Care,”4 recommend that all nurses, including APRNs, receive the End of Life Nurses Education Consortium (ELNEC) training to close the educational gap. End of Life Nurses Education Consortium is a well-established curriculum that has been used since 2001 to train nurses and nurse educators.11 The ELNEC curriculum has been adapted to address unique APRN needs within that scope of practice.11 As noted by Dahlin et al,3 there are only a handful of palliative care specialty training programs for APRNs, and these programs may not be practical for expanded use in APRN education. The Palliative Care APRN Externship represents a novel way to use established curricula, take up the call to action, and potentially improve access to palliative care. The purpose of this article is to prove a brief highlight of educational strategies/learner needs to consider in palliative care APRN education, describe outcomes of the Palliative Care APRN Externship in 1 site, and discuss the application of such training in addressing palliative care workforce needs.

Although the current state of palliative care nursing education is not ideal, much work has been done to identify palliative care educational needs and measure the impact of programs designed to meet these needs. In a survey of nurse practitioner students and critical care nurses who had a variety of experience and educational preparation, respondents placed a high value on palliative care; however, less than half of those surveyed had received palliative care education during the 2 years before the survey.12 The researchers identified differences in beliefs and attitudes toward palliative care competencies by experience level, certification, and level of training.12 Nurses with more experience ranked talking with the dying, pain, and symptom management higher than the less experienced nurses.12 In turn, less experienced nurses were less comfortable with ethical issues, personal feelings, and knowledge of hospice.12 The authors concluded that, in an educational experience, such as the Palliative Care APRN Externship, strategies that account for generational and other demographic characteristics of participants should be used.12

In a seminal article addressing multigenerational challenges in education, Johnson and Romanello13 point out that nursing educators must recognize their own generational biases and incorporate differences between multigenerational learners in educational program design. For example, those born into the Baby Boom generation (between 1943 and 1960) prefer lectures, are less comfortable doing group work, and are less adept at technology.13 However, Generation X learners (born between 1961 and 1981) believe that education must be practical and of direct benefit while being more comfortable with technology and distance learning.13 The Millennial generation, that is, Generation Y, born between 1982 and 2002, are savvy with technology, prefer group projects, and appreciate the use of simulation and games in learning activities.13 Nursing educators may be teaching multiple generations simultaneously and include students who prefer both passive and active learning styles.14 Consequently, educators should use a variety of methods to address generational differences in learning styles.14

Adapting educational content and methods to the needs of the learner is further supported by a study by Jors et al.6 Using an open-ended survey, 675 experienced physicians and nurses responded to a question about which changes in education and training they would like to see to provide better care for the dying.6 The authors identified 2 main categories of responses: the format of education and content.6 Respondents emphasized the importance of practical clinical experiences with oversight from experienced clinicians, enhanced classroom lectures/seminars in training programs, lifelong learning to support clinical competence, and creative teaching methods, such as role play or case studies.6 Participants revealed several valuable content areas including communication skills training, how to care for the patient's family and friends, and how to manage pain and symptoms appropriately.6 Communication skills training in palliative care is often accomplished through role play. Role play is a form of simulation in a controlled environment, allowing for mistakes and learning without consequences from real patients and families.15 Simulation-based communication training has been shown to increase confidence in palliative care communication.15 In addition, the use of interprofessional learners in communication simulation can promote improved teamwork and interprofessional communication.15 The use of role play, as an active learning strategy, was an important component of the Palliative Care APRN Externship communication training.

Although classroom education is essential, the application of knowledge in a clinical immersion experience can have a significant impact. In planning an educational experience for practicing APRNs, it is critical to include practical content, use learning experiences at the bedside, and support the best possible learning environment. Aspects of a positive, palliative care learning environment include engaging interaction, acceptance, and belonging.16 Engaging and interacting with learners can be promoted through active facilitation by a mentor/preceptor when providing individualized teaching, case discussion with problem solving, and formative feedback.16 Engaging interaction involves reciprocal learner motivated strategies such as preparatory reading, developing learning objectives, and seeking out experiences within the host site.16 Advanced practice registered nurse externship participants were asked to do preparatory reading in advance and to select clinical experiences based on self-identified learning needs.

Validation and reciprocation, another concept in engaging interaction, is a method of mutual exchange between the learner and the host. A learning experience can be enhanced for all when there is a mutual exchange of expertise and validation of workplace practices.16 Creating a positive learning environment must also take how learners “fit in” within the host site culture and social context into account.16 Moreover, learners who feel welcomed and supported at a host site may have a better overall experience.16 Externship learning experiences were designed to facilitate exposure to the palliative care interdisciplinary team informally to promote discussion and support a positive learning environment.

Dahlin et al3 developed an intense educational experience aimed at midcareer APRNs as a potential solution to both palliative care workforce shortages and the lack of palliative care training for APRNs. The authors developed and tested a palliative care externship that included not only classroom learning but also clinical training and participant-driven experiences. The authors believed that it was too difficult for practicing APRNs to partake in a formalized nursing fellowship program because of a lack of availability as well as the potential loss of salary and benefits.3 The authors theorized that participation in an externship program might inspire participants to support or initiate palliative care programs in the future.3 The externship curriculum included content aimed at the level of APRNs around pain and symptom management, communication, business development, and quality metrics.3 Participants spent time with palliative care staff on the palliative care unit and doing palliative care consultation. These experiences provided participants with the most essential and meaningful component of the training. Participants selected elective clinical experiences based on self-identified learning needs and goals. Results were uniformly positive, and most of the participants met personal goals and noted increased self-confidence in palliative care competencies.3


The Palliative Care APRN Externship was an education program implemented with the intent of improving APRN knowledge and skill in palliative care. Building upon the program described by Dahlin et al,3 the Palliative Care APRN Externship was expanded to additional sites. This article describes the evaluation of the effectiveness of the program in 1 such site. The APRN externship took place at a teaching hospital, and APRNs were recruited through email. Participation was voluntary and associated with a fee. All participants were nurse practitioners.

Program Description

Two cohorts totaling 10 participants each received 1 week of immersive training. Clinical experiences for participants included observation of palliative care clinicians on an inpatient consult service and the participant's choice of specialty electives. The externship included both a didactic experience and a total of 12 hours of clinical observation. Before arrival, participants received preparatory reading materials. In addition, learners were given a thumb drive containing assessment tools, a copy of slides from classroom presentations, and evidence-based readings to support classroom content. The classroom training was provided by the principal investigator (PI) and additional expert palliative care clinicians and interdisciplinary team members. Educational content consisted of an overview of palliative care, pain management, symptom management, communication, resilience, leadership, care at the end of life, and business/program development. The curriculum included essential elements of quality palliative care from the National Consensus Project for Quality Palliative Care and was based on the established ELNEC curriculum.11,17,18 The clinical experiences included observation of palliative care clinicians in an inpatient consult service and individually selected sites such as an inpatient hospice unit and palliative care clinic. Interactive learning continued during lunchtime seminar sessions featuring team members, including social work, chaplaincy, community care providers, and physical/occupational/speech therapy.

Program Evaluation

The program was evaluated in both formative and summative manner using assessment of electronic survey data and qualitative data obtained from debriefings and electronic survey comments. Participants were informed in advance that an electronic evaluation survey would be provided after completion of the course and that the debriefing sessions were required for the externship. Participants were informed that anonymous notes would be taken during debriefing sessions and that every effort to maintain confidentiality was taken. Consent was implied through participation in the debriefing and electronic survey. The institutional review board at the University of North Carolina at Greensboro determined that this project was exempt.


The HPNA, the continuing education provider, collected evaluation data from participants through an electronic survey. These data were provided in aggregate, deidentified form for analysis. An electronic survey developed by the continuing education provider was used to survey participants after course completion. Questions/responses included in the survey are listed in the Table. The following educational outcomes were evaluated by individual participants using the electronic survey:

Electronic Survey Instrument and Results
  1. Preparation of the individual APRN in his or her role to provide primary palliative care in his or her institutions and communities, with an emphasis on using evidence-based practice and research.
  2. The opinion of individual participants regarding whether the program was worthwhile for their APRN professional practice development.
  3. The opinion of individual participants regarding whether the program enhanced knowledge/skill/practice as a palliative care APRN.
  4. Evaluation of the PI and additional faculty's knowledge of the topic and ability to engage participants in active learning.

In addition, qualitative data were collected daily during 30- to 60-minute group debriefing sessions with participants. The debriefing notes, collected by the PI, were anonymous and not attributable to individual participants. Participants were asked not to discuss their responses outside the debriefing sessions. At the end of the week, a final debriefing session was held. The PI led the debriefing discussions and took handwritten notes.


Ten nurse practitioners completed the externship course, participated in debriefings, and completed electronic survey evaluations. Collated, deidentified, electronic evaluation survey results were provided to the PI by the continuing education provider. Participants were asked to respond to 15 questions based on a 5-point scale: strongly agree, agree, neutral, disagree, and strongly disagree. The electronic survey results included information on the current level of nursing practice but no additional participant characteristics. Daily debriefing discussions included feedback about clinical and classroom experiences. Notes taken from debriefings were transcribed verbatim and categorized into themes by the PI. Themes were identified informally by pulling out repeated words, phrases, and synonyms from transcribed notes. In a similar fashion, narrative responses included with the electronic survey results were analyzed for repeated words, phrases, and synonyms.


Electronic Survey

Collated evaluation survey results received from the continuing education provider were reviewed. The evaluation's summaries from each externship cohort revealed high levels of satisfaction with the program. All responses to questions were agreed or strongly agreed that the education outcomes were met (Table). There were no responses listed as neutral, disagree, or strongly disagree. Narrative comments on the program quality indicated that participants were very satisfied with the program quality. Overall, the results of this evaluation highlight the success of the Palliative Care APRN Externship in supporting the preparation of the individual APRN in his or her role to provide primary palliative care in his or her institutions and communities. Individual participants believed the program to be worthwhile for their APRN professional development, enhanced knowledge/skill, and validated the faculty's knowledge of the topic and engagement with learners.

Debriefing/Narrative Comments

Three major themes were identified from debriefing notes and narrative survey responses: the learning environment, program content/skill development, and validation.

Learning Environment

Comments related to the learning environment included feedback about the clinical and classroom experiences as well as educational strategies. A positive learning environment is essential for the success of a palliative care educational program.16 The first day of the externship included 6 hours of classroom content on pain and symptom management to prepare participants for clinical experiences but also led to fatigue: “The first day was good but so intense that I did not know my name by the end of the day.” The most common feedback was related to clinical experiences and the desire for more clinical opportunities. Most participants reported time spent in clinical areas to be the highlight of the week; however, 1 participant commented about an unwelcoming preceptor and the negative impact this had on the learning environment. In addition, participants spoke of staff enthusiasm and role-modeling teamwork contributing to the experience: “Appreciated seeing how the team worked together to coordinate things,” and “Everyone was enthusiastic about what they do.”

Program Content and Skill Development

Participants wanted more practical content about pain and symptom management. This need is supported in the literature as practicing nurses have sensory and visual learning styles and desire practical, fact-oriented content using visual representations.19 An example of active, sensing learning strategies was the use of case studies and working through opioid conversions as a pedagogical strategy. Participants commented on the value of addressing specific symptoms, namely, pain, dyspnea, fatigue, depression, anxiety, and diarrhea, using assessment tools, evidence-based websites, and medications. Most participants felt the classroom content was useful, whereas others indicated the material was too basic for their needs.

Content devoted to communication was the focus of an entire day of learning and included a seminar-style format with didactic material and role play, including simulating family meetings, delivering difficult news, and eliciting goals of care. Participants found active learning strategies helpful, as 1 participant noted: “Role play more useful than anticipated and identified areas for improvement in communication skills.”

Several participants self-identified as new to roles within palliative care working with newly developed programs. Comments reflected the need for growth in individual roles and the business aspects of their palliative care program: “I have a much clearer understanding of how to have true palliative care conversations.” Information to support program development was valued and included a review of quality improvement metrics, discussion of consult etiquette, and information on billing and coding. Other comments in this theme reflected the need for added content on team building, conflict management, resilience, and addressing ethics topics.


Whereas themes that included educational environment and content were not surprising, a third theme, validation, offers insight into a crucial role that the APRN palliative care externship can play in the development of new palliative care APRNs. Validation is described as acknowledging, identifying with, and accepting another person's feelings and behaviors as understood.20 The need for validation and support from others who have shared understanding was reflected in many comments. Phrases such as “like-minded people,” “affirmation of current practice,” “good to be with like-minded people who understand what I do and get it,” and “I knew more than I thought I did” are illustrative of the importance of validation.

In addition, many APRNs practicing in HPC have less than 10 years in the field and report little palliative care content in the graduate program as well as multiple barriers to practice.9 One of the most frequently reported practice challenges reported by HPC APRNs is a lack of understanding of their role.9 During debriefings, several of the externship participants reported practicing in palliative care as the sole APRN and having done so for less than 5 years. Therefore, being with other APRNs who have shared experiences and who understand the APRN role in palliative care may be critical to success and longevity in the field.

Application to practice consisted of responses about specific changes to future clinical practice and personal application of the externship experience. Some participants indicated improved communication and symptom management skills, whereas others spoke of program development ideas. On a more personal note, participants spoke of personal growth and development: “I was able to process so many things from a provider/career level even down to a personal level,” and “I feel as if everything makes sense now, why I am doing what I am doing.”


The palliative care advanced practice nursing externship presents an exciting opportunity to expand the palliative care workforce and educate more APRNs in primary palliative care skills. The externship was developed after a successful pilot program that trained 48 APRNs.4 The positive results of the pilot program led the researchers to replicate the program to expand access to more APRNs in 3 geographically diverse locations, including the one described in this article.3 The positive evaluations noted in this iteration of the program support continued use of this educational format. Results of the evaluation reinforce the use of clinical experiences along with didactic learning when teaching palliative care learners.

The translation of the externship program may be limited by the volume of learners that can be supported. Many palliative care programs, particularly in academic settings, are faced with requests to provide learning experiences for physician residents, fellows, medical and nursing students, and others. Clinicians may be overwhelmed by the demands of teaching while caring for patients. Palliative care educators may benefit from strategies developed and experiences from programs such as the palliative care advanced practice nursing externship.

The ability to provide palliative care education to APRNs in a 1-week format is an exciting way to expand access to those in need of palliative care. The cost of attendance and the ability to take time away from the job may limit future participation in the externship program. Lack of publicity and advertising costs may restrict the program leaders' ability to recruit participants. As an incentive to submitting evaluation data, attendees will be required to complete program evaluations before continuing education credit is issued. The time lag between program completion and evaluation submission may alter the accuracy of participant responses.

Barriers to obtaining qualitative data include participant reluctance to share experiences in an open debriefing session. Participants sharing answers and content of the discussion outside the debriefing may impact results. Individual participants had unique clinical experiences with a variety of preceptors and a variety of clinical backgrounds and past experiences.

Each of the externship experiences was attended by APRNs from different generational groups. The feedback offered from participants may have varied based on generational differences in learning styles and preferences. Multigenerational audiences demand that content be delivered in ways that are relevant to them. Future externship programs might include small group work with case studies, reflective writing, and opportunities to develop complex medication regimens for symptoms.

Clinical experiences will remain the hallmark of future externship experiences. The primary challenge for upcoming externship programs will be keeping clinical preceptors engaged and willing to host learners while performing patient care. Individually, preceptors can make or break the experience for the learner. One encounter with a preceptor that is not welcoming could adversely impact the overall evaluations of the week. Creating an environment where learners feel valued and part of the team rather than a burden will be essential for the success of future externship programs.

The application of this project would be best done in locations where an established palliative care program exists with experienced APRN faculty, administrative support, and endorsement from program leadership. The educational site must be able to support clinical experiences, including palliative care consults, outpatient clinics, and a variety of specialty electives from which participants can choose. Smaller and less established palliative care programs that lack these essential components may not be able to provide the necessary weeklong educational program, including immersive clinical experiences.

Meeting the needs of learners with varying levels of palliative nursing experience, diverse backgrounds, and learning styles will require creativity. The use of pedagogical methods that include complex case studies, technology resources, and small group work, as well as traditional classroom education, should be implemented. Innovation involving smartphone applications for symptom management and video communication simulation could be incorporated into the curriculum and used by learners when they return to their clinical settings.

Although the curriculum used in the Palliative Care APRN Externship was focused on developing primary palliative care skills, it could be expanded in the future to include more advanced content. Advanced practice registered nurses who have been practicing primary or specialty palliative care for some time but do so in isolation may value an immersive educational experience to validate their skills and competence. Moving the focus from novice to expert could broaden the reach of future externship programs. As an affirmation of the interdisciplinary nature of palliative care and in response to the provider shortage, future externships should include other advanced practice providers such as physician assistants.

The Palliative Care APRN Externship has been shown to positively impact the knowledge and skills of participants and help them feel validated in their current practice. The Palliative Care APRN Externship is a practical way to expand the palliative care workforce and can be adapted to meet many different learning styles and needs.


The authors thank Patrick Coyne, Elizabeth Zechinati, and Susan Denman for their assistance with this project. They acknowledge the Cambia Health Foundation for their support of Constance Dahlin as a Soujourns Leadership Scholar to develop the national externship model.


1. Kochanek K, Murphy S, Xu J, Arias E. Deaths: final data for 2017. National Vital Stat. 2019;68(9):1–77. Accessed September 2, 2019.
2. The Institute of Medicine. Dying in America: Improving Quality and Honoring Individual Preferences Near the End of Life. Washington, DC: National Academies Press; 2014: Accessed September 2, 2019.
3. Dahlin C, Coyne PJ, Cassel JB. The advanced practice registered nurses palliative care externship: a model for primary palliative care education. J Palliat Med. 2016;19(7):753–759.
4. American Nurses Association Professional Issues Panel in collaboration with the Hospice and Palliative Nurses Association. Call for action: nurses lead and transform palliative care. 2017. Accessed September 9, 2019.
5. Hospice and Palliative Nurses Association. Value of the advanced practice registered nurse in palliative care. Accessed September 2, 2019.
6. Jors K, Seibel K, Bardenheuer H, et al. Education in end-of-life care: what do experienced professionals find important. J Cancer Educ. 2016;31:272–278.
7. Theisen J, Sandau KE. Competency of new graduate nurses: a review of their weaknesses and strategies for success. J Contin Educ Nurs. 2013;44:406–414.
8. Kriebel-Gasparro A, Doll-Shaw M. Integration of palliative care into a nurse practitioner DNP program. JNP. 2017;13:e383–e388.
9. Pawlow P, Dahlin C, Doherty CA, Ersek M. The hospice and palliative care advanced practice registered nurse workforce: results of a national survey. J Hosp Palliat Nurs. 2018;20(4):349–357.
10. NewMediaWire. HPNA issues call to action in support of curricular guidelines for nursing education in palliative care. NewMediaWire Web site. Accessed September 2, 2019.
11. American Association of Colleges of Nursing. About ELNEC. Accessed September 2, 2019.
12. White KR, Roczen ML, Coyne PJ, Wiencek C. Acute and critical care nurses' perceptions of palliative care competencies: a pilot study. J Contin Educ Nurs. 2014;45(6):265–277.
13. Johnson SA, Romanello ML. Generational diversity: teaching and learning approaches. Nurse Educ. 2005;30(5):212–216.
14. Hampton D, Pearce PF, Moser DK. Preferred methods of learning for nursing students in an on-line degree program. J Prof Nurs. 2017;33(1):27–37.
15. Bradway C, Cotter VT, Darrah NJ, et al. An interprofessional education simulation workshop: health professions learning palliative care communication. J Nurs Educ. 2018;57(8):493–497.
16. Connell S, Yates P, Barrett L. Understanding the optimal learning environment in palliative care. Nurse Educ Today. 2011;31:472–476.
17. Ferrell B, Twaddle ML, Melnick A, Meier DE. National consensus project clinical practice guidelines for quality palliative care guidelines, 4th edition. J Palliat Med. 2018; doi:10.1089/jpm.2018.0431.
18. Ferrell B, Malloy P, Virani R. The end of life nursing education nursing consortium project. Ann Palliat Med. 2015;4(2):61–69.
19. Mangold K, Kunze KL, Quinonez MM, Taylor LM, Tenyson AJ. Learning style preferences of practicing nurses. J Nurses Prof Dev. 2018;34(4):212–218.
20. Hall K. Understanding validation: a way to communicate acceptance. Accessed September 2, 2019.

APRN; feedback; learning; nursing education; palliative care; program evaluation

Copyright © 2020 by The Hospice and Palliative Nurses Association. All rights reserved.