Secondary Logo

Journal Logo

Quality Improvement Report

Electronic primary care to specialist referral: A pilot project to evaluate referral expedience and efficacy

Olson, Ernestine DNP, APRN, FNP-BC (Nurse Practitioner)1,2; Fruh, Sharon PhD RN, FNP-BC (Assistant Dean)2; Kleinpell, Ruth PhD, RN, FAAN (Assistant Dean)3

Author Information
Journal of the American Association of Nurse Practitioners: April 2021 - Volume 33 - Issue 4 - p 318-323
doi: 10.1097/JXX.0000000000000377


In the current environment of escalating health care costs, third-party payers inquire as to the quality of care patients receive for the payments provided (McLaughlin, 2008). The Institute of Medicine (IOM, 2001) highlighted breakdowns in quality care and issued recommendations for resolving gaps in care delivery by dispatching six targets for resolution: (a) improved expedience, (b) patient influence, (c) augmented efficacy, (d) patient wellbeing and safety, (e) care provision equity, and (f) effective care. A quality improvement project was designed to explore an option for improving referral processes from primary to secondary and tertiary care with the intent of enhancing the targets identified by the IOM (IOM, 2001). This article will describe the changes that occurred with the project implementation because it related to referral processes within a multispecialty practice group.


The predominant point of care access is through a primary care visit and it is at the primary care level where diagnosis and treatment options including a need for specialist referral are conveyed to patients (Feldstein, 2011). Some disorders require emergent referral and those patients are typically routed to emergency centers. Other specialist interventions are elective and can be scheduled at will, but there are other referral needs which if delayed will result in increased health care expense burden, heightened morbidity, and extended rehabilitation time. Patients with gynecological tumors, for example, have improved outcomes when early referral is exercised (Greving, Vernooij, Heintz, van der Graaf, & Buskens, 2009), and patients with certain respiratory infections, such as aspergillosis, have improved outcomes and decreased morbidity with rapid referral (Karthaus, Neuperlach, & Ring, 2011). Additionally, several authors have described the benefits of early intervention in reducing cardiac morbidity and mortality (Chockalingam, Gnanavelu, Subramaniam, Dorairajan, & Chockalingam, 2005; Parikh et al., 2010). Delayed referrals for orthopedic and neurologic care can also contribute to lengthier recovery and can be associated with the development of chronic opioid use (Braden, 2010). Time lapse between renal insufficiency and hepatobiliary disease identification and referral to specialists has also shown a direct correlation to survivability (Cass et al., 2002; Croome, Chudzinski, & Hanto, 2010).

Electronic medical record referral

With the increasing availability of electronic medical record (EMR) systems, the capability of processing primary to specialist referrals via electronic means becomes a viable option. Electronic referrals have been shown to save time and provide impressive economic savings (Heimly, 2009), and activating the electronic referral component of an existing EMR adds economic value to the existing system (Erstad, 2003). Gibson and Singh (2003) recommended that organizations that provide health care interventions need to refocus on proactive practices aimed at improving patient outcomes expeditiously and equitably.

One study reported that 75% of involved physicians described the electronic referrals as less cumbersome than standard processes (Weiner et al., 2009). Another study reported a reduced referral time from 90 days to 5.5 days which in turn resulted in patient volume escalations up to 7 times the capacity prior to the e-referral intervention predominantly due to logjam reduction (Fischer, Martinez, Driscoll, & Conway, 2010). In addition to obvious financial benefits, rural patient satisfaction with health care systems was reportedly improved when referrals were quickly established and smoothly negotiated (Caldwell & Arthur, 2008).

Quality improvement initiative

At a multispecialty group practice meeting, the need for more efficient specialist referrals was identified when primary providers reported that patients often experienced worsening conditions during interim waits for specialist access. Attendees began a discussion of options to improve specialist access and a plan was put in place for primary practitioners to directly contact specialists by phone for urgent referrals. Once the urgent access issues had been addressed, attention was directed toward expedition of routine primary to secondary referrals. A needs assessment identified that 30% of group referrals were delayed over a month. Under the original system, patient referral specialist access requests were submitted by paper to a specialist's front office staff. All requests were then reviewed and patients were contacted for scheduling. Time delays occurred in both referring and receiving offices, which were further complicated by missed phone calls. Based on the need assessment, a project was proposed using an electronic referral system to complete primary to secondary referrals.

The quality improvement (QI) project intention was to accomplish a completed primary to secondary referral in less than 1 month for routine referrals while using specialist-specific guidelines. By including guidelines, the process would provide for appropriate preliminary patient workup and increased patient control of scheduling options. Accomplishing 90–100% of referrals in less than 1 month was identified as a reasonable improvement goal. The purpose of the QI project was to determine whether a direct electronic primary care to specialist referral system within a multispecialty group would decrease access wait times to less than 1 month in a manner that was acceptable to patients, primary care providers, and specialists.


The multispecialty group had 89 members covering 17 specialties and included both rural and urban family practice clinics affiliated with a 234-bed nonprofit acute care hospital in south central United States. The project champions who became the implementation team included five specialists, a practice management official, three office managers, and a primary care nurse practitioner project coordinator. The EMR used for the study was eClinicalWorks, which enables the administrator to designate which providers have access to a specialist's schedule. Any EMR scheduling software can be used once the information technology security sanctions a primary practitioner's access to a specialist schedule. Some EMRs have added a separate referral management system (Texas Tech University Health Sciences Center, 2017).


A convenience sample assembled from patients who self-scheduled for primary care at the clinic, were mentally competent, were more than 18 years, who were deemed in need of specialist referral were apprised of disease differentials and options for treatment, specialist referral, and second opinions. To elude bias toward specialist selection, the options were transparently and uniformly presented without pressure to participate, and each individual was given a choice to opt in or out of the project. This was done to ensure equal chance of participation and provide a sample representative of the population (Kirchhoff, 2009). Patients were placed in control of date selection, practitioner choice, and the option to change their mind in the event of a change in their clinical or personal circumstances. All patient information during the collection process was protected and only available to providers directly involved in care of individual patients. Once patients opted for specialist care, their information was deidentified and placed in the QI project aggregate data. After the development of the QI process (see Primary to Specialist EMR Referral Flow Map, Supplemental Digital Content 1,, institutional review board approval for an exempt study was obtained with the identified purpose of improving patient access and outcomes in concurrence with refining system patient flow.

Phase I of the QI project adhered to the Kitson conceptual framework (Kitson, Harvey, & McCormack, 1998), which is closely aligned with the Johns Hopkins Model (Newhouse, Dearholt, Poe, Pugh, & White, 2005) and used the assessment of projects in terms of evidence, context, and facilitation. The Kitson framework defines methodology for constructing a positive transformation of research into practice. It delineates operationalization of three elements and their dynamic forces with the utilization of the following multidimensional equation “Successful Implementation = f (ECF), where ST = successful implementation, E = evidence, C = context, F = facilitation, and f = function of” (Kitson et al., 1998, p 150). The level and nature of the evidence, the context or environment into which the research is placed, and the method or way in which the process is facilitated combine together for strength and support of the transformation. The framework guided the project through evidence evaluation in the context of the need for improved referrals and the ability to facilitate an improvement with enhanced use of EMR.

Phase II entailed project implementation following an appraisal of a balanced scorecard (Kaplan & Norton, 1992; International Center for Management and Business Administration, Inc., 2002-2010) and involved granting a primary care nurse practitioner electronic access to the participating specialist schedules. Guidelines detailing preferred visit time frames, preparatory workups, and type of acceptable visits were provided by the champion specialists. Patient demographics were automatically transferred within the group.

Implementation was accomplished in accordance with the Rosswurm model (Rosswurm & Larrabee, 1999). The plan for outcome evaluations followed a plan, do, study, act (PDSA) process (Naranjo & Kaimal, 2011) based on Donabedian's (1988) structure, plan, and outcome evaluation method. The PDSA for the EMR referral project had a multispecialty clinic for the structural foundation. A plan (P) was envisaged and reviewed for achievability, the do (D) that was the action component for project execution included the generation of evidence-based referral guidelines, e-referral procedure and policy development and enactment, a study (S) component integrated a review of results, and the act (A) constituent entailed an evaluation and revision of the process as indicated. More office staff education was revealed as a need for improved satisfaction with the process.

A process flow map provided a visual path for the project. Electronic schedule access was facilitated by practice management and limited to the referring provider. Education of providers, office managers, and patients was conducted by the project coordinator. Data collection that occurred over a 4-month period was carried out by the project coordinator as well. Evaluation tools included Likert scales for qualitative responses and a nominal scale for the quantitative log of the referral to specialist encounter time gaps. The evaluation process was predicated on the context, input, process, and product model.

The segue into Phase III began with the creation of a data set for analysis that included a progressive referral log for recording the time the electronic referral was activated and the time of the specialist encounter, as well as a notation designating whether the referral visit was accomplished as scheduled. The electronic referral to specialist visit time gaps were then plotted on a bar graph (Figure 1) along with the standard in system referral to specialist visit time gaps and outsourced referral to specialist visit time gap results. The project was designed to improve patient primary care to specialist referral processes. A project goal of completing at least 90% of the electronic referrals in less than 1 month was designated before onset of the project and was established due to the needs assessment, which revealed that only 70% of standard process referrals were accomplished in less than 1 month. Primary outcome improvement was defined as a completion of 90% of internal group system electronic referrals accomplished in less than 1 month and was established as a benchmark for the project.

Figure 1.:
Referral bar graph.

The project implementation was conducted over a 16-week period. Qualitative review of the electronic referral process was conducted at the completion of the project data collection, which included survey contributions from the 14 patients and the three participating provider office coordinators. Survey questions for both patients and providers included an inquiry into whether referrals processed smoothly and information was correctly communicated. In addition, the patients were asked to rate whether the referral was presented clearly and if they considered the time between referral and consult satisfactory. Providers were also asked to rate whether they felt the referral option was readily accessible and functional, and if the time gap between referral and consult met care criteria. Both groups rated their responses on a Likert scale and were given an opportunity to offer suggestions (Tables 1 and 2). Because the EMR was universally functional within the group, the implementation costs were associated with information systems access build time and staff training (Table 3).

Table 1. - Likert scale electronic referral patient survey results
Scale Total Responses/Surveys Sent Great, % Good, % OK, % Fair, % Poor, %
The referral was presented clearly 14/14 86 14
The referral processed smoothly 14/14 86 14
The time between referral and consult was satisfactory 14/14 86 14
Information was communicated clearly 14/14 86 14
Additional suggestions None

Table 2. - Likert scale electronic referral office manager survey results
Scale Total Responses/Surveys Sent Great, % Good, % OK, % Fair, % Poor, %
The referral option was readily accessible and the system was functional 3/3 33 66
Referrals processed smoothly 3/3 33 33 33
The time between referral and consult met care criteria 3/3 33 33 33
Information was communicated clearly 3/3 33 33 33
Additional comments More office manager education and ongoing collaboration between clinical and office staff

Table 3. - Budgetary items
Budget Category Budget Item Cost per Unit Totals
Electronic medical record system equipment (costs for referral component) Hardware: $25/month × 4 months = $100
Software: $3/month × 4 months = $12
$112 $112
Administrative supplies (paper, stamps, ink, copy costs) Educational materials = $5
Guideline materials = $5 (one-time cost)
Survey materials = $25
$35 $35
Labor Record keeping at 10 minutes per referral for 40 referrals = $6
Training for: 4 office coordinators $20/hour × 1 hour = $80
Information tech $23/hour for 16 hours = $368
Instructor salary $43/hour for 1 hour = $43 (start-up cost)
$497 $497
Total 4-month cost Start-up plus ongoing costs $644
Maintenance Ongoing cost per patient $2.80 × 10 patients/month = $28/month Monthly cost $28


A total of 74 patients were referred to specialists during the 16-week period. Sixteen (22%) patients were deemed in need of referral to one of the participating specialists, with two of those warranting same-day referral, which was accomplished through phone communication per protocol. Eleven (15%) patients were referred via the standard referral methods in system and 47 (64%) patients were referred outside the group using standard referral methods. A bar graph (Figure 1) depicting the percentage of standard referrals, electronic referrals, and outside standard referrals completed within the 1 month or less was used to display the efficacies in comparison with the benchmark of 90% within 4 weeks. Electronic referrals demonstrated a 100% scheduling completion rate in 1 month or less. One patient later cancelled and rescheduled due to transportation quandaries. Nine (82%) of the standard in system referrals were completed with the 1 month time frame and 36 (77%) of external standard referrals were completed in 1 month or less. The quantitative outcomes revealed a 30% improvement over the pre-project time gap, in which 70% of the referrals were completed within 1 month or less. Qualitative surveys (Table 2) returned from the three office managers were mixed, and comments revealed a need for increased specificity with guidelines and more detailed staff education in the specialist offices. All 14 surveys returned from the participating patients were positive, with 12 of the 14 (86%) respondents rating the experience as great.


A direct primary care nurse practitioner provider to specialist EMR schedule referral based on specialty referral needs resulted in a more expedient referral compared with those referred by the standard process. Additionally, patients reported an efficient and satisfactory process. All patients were given an option to select their preferred provider, so the electronic referral providers were selected by the patients deemed in need of specialist referral. The two patients needing urgent referral were seen the same day after a provider-to-provider phone call according to protocol in place. The project findings were consistent with findings by Fischer et al. (2010), which were supportive of electronic referral processes.

Moreover, a study published by Piscotty, Kalisch, and Gracey-Thomas (2015) made note that a blend of technology and high-level clinical reasoning has a high potential to not only upgrade “quality and safety” but also minimize medical care errors. Because the QI study was completed, very little has been published regarding the utilization of EMR to enhance the referral process. Patel et al. (2018) noted an ongoing correlation between the gap time between assessed referral need and completed referrals. The longer the patient had to wait, the less probable that a completed referral would transpire. Referral time delay regrettably remains a harmful and expensive error (Singh, Meyer, & Thomas, 2014).

Office manager buy in along with the provider commitment is critical to the process because there is a need on their part to control the office flow and involving them as an integral part of the planning team for change is essential (Senge, 1994). The office managers did not give specific reasons for the ratings they gave to the project, but all commented that they wanted more education and practice before being presented with the project.

The results of the project have application to multispecialty groups with interconnected practice systems. Recommendations for sustainability would include expansion of the planning team to include more specialties and development of more detailed guidelines for each specialty's diagnostic needs to expedite processes. Improving information sharing regarding schedule detail will also provide for smooth application when including additional clinics. Deeds, Dowdell, Chew, and Ackerman (2019) also identified a need for guidance and education regarding electronic consult programs for primary care providers. Future QI projects could also be directed toward coordinating referrals electronically between health care organizations and individual specialist groups such as cardiology and pulmonology. In summary, a nurse practitioner–led process of implementing electronic referrals offers a safe means of expediting effective referrals in an efficient and patient-centered approach, and can be best implemented with well-developed clinical and schedule guidelines.

Future QI projects could also be directed toward coordinating referrals electronically between health care organizations and individual specialist groups such as cardiology and pulmonology. In summary, a nurse practitioner–led process of implementing electronic referrals offers a safe means of expediting effective referrals in an efficient and patient-centered approach, and can be best implemented with well-developed clinical and schedule guidelines.


Braden J. B., Russo J., Fan M., Edlund M. J., Martin B. C., DeVries A., Sullivan M. D. (2010). Emergency department visits among recipients of chronic opioid therapy. Archives of Internal Medicine, 170, 1425–1432.
Caldwell P. H., Arthur H. M. (2008). The influence of a “cultural referral” on access to care in rural settings after myocardial infarction. Health and Place, 15, 180–185.
Cass A., Cunningham J., Arnold P. C., Snelling P., Wang Z., Hoy W. (2002). Delayed referral to a nephrologist: Outcomes among patients who survive at least one year on dialysis. The Medical Journal of Australia, 177, 135–138.
Chockalingam A., Gnanavelu G., Subramaniam T., Dorairajan S., Chockalingam V. (2005). Right ventricular myocardial infarction: Presentation and acute outcomes. Angiology, 56: 371–376.
Croome K. P., Chudzinski R., Hanto D. W. (2010). Increasing time delay from presentation until surgical referral for hepatobilitary malignancies. International Hepato-Pancreato-Biliary Association, (Oxford), 12, 644–648.
Deeds S. A., Dowell K. J., Chew L. D., Ackerman S. L. (2019). Implementing an opt-in eConsult program at seven academic medical centers: A qualitative analysis of primary care provider experiences. Journal of General Internal Medicine, 34, 1427–1433.
Donabedian A. (1988). The quality of care: How can it be assessed? Journal of the American Medical Association, 260, 1743–1748.
Erstad T. L. (2003). Analyzing computer based patient records: A review of literature. Journal of Healthcare Information Management, 17, 51–57.
Feldstein P. J.. (2011). Health policy issues: An economic perspective (5th ed.). Chicago, IL: Health Care Administration Press.
Fischer B. S., Martinez E., Driscoll M., Conway T. (2010). Chicago: Using evidence-based rules to make smarter referrals. Health Affairs, 5, 972–975.
Gibson R., Singh J. P. (2003). Wall of silence: The untold story of the medical mistakes that kill and injure millions of Americans. Washington, DC: A Regnery Publishing Company.
Greving J., Vernooij F., Heintz P., van der Graaf Y., Buskens E. (2009). Is centralization of ovarian cancer care warranted? A cost-effectiveness analysis. Gynecologic Oncology, 113, 68–74.
Heimly V. (2009). Electronic referrals in healthcare: A review. European Federation for Medical Informatics, 327–331. doi: 10.3233/978-1-60750-044-5-327.
Institute of Medicine. (2001). Crossing the quality chasm: A new health care system for the 21st century. Washington, DC: National Academy Press.
International Center for Management and Business Administration, Inc. (2002–2010). The balanced scorecard. Retrieved from
Kaplan R. S., Norton D. P. (1992). The balanced scorecard: Measures that drive performance. The Harvard Business Review, 70, 71–79.
Karthaus M., Neuperlach K., Ring O. (2011). Prophylaxis and treatment of invasive aspergillosis with voriconazole, posaconazole, and caspofungin: Review of the literature. European Journal of Medical Research, 16, 145–152.
Kirchhoff K. T.. (2009). Sampling methods. In Mateo M., Kirchhoff K. (Eds.) Research for advanced practice nurses: From evidence to practice (pp. 155–166). New York, NY: Springer Publishing Company.
Kitson A., Harvey G., McCormack B. (1998). Enabling the implementation of evidence based practice: A conceptual framework. Quality in Health Care, 7, 149–158.
McLaughlin C. P., McLaughlin C. D. (2008). Health policy analysis: An interdisciplinary approach. Sudburn, MA: Jones and Bartlett.
Naranjo L. L., Kaimal P. V. (2011). Applying Donabedian's theory as a framework for bariatric surgery accreditation. Bariatric Nursing and Surgical Patient Care. 6, 33–37.
Newhouse R., Dearholt S., Poe S., Pugh L. C., White K. M. (2005). Evidence-based practice: A practical approach to implementation. Journal of Nursing Administration, 35, 35–40.
Parikh S. V., Lemos J. A., Jessen M. E., Brilakis E. S., Ohman E. M., Chen A. Y., Holper E. M. (2010). Timing of in-hospital coronary artery bypass graft surgery for non-ST-segment elevation myocardial infarction patients' results from the national cardiovascular data registry action registry-GWTG (acute coronary treatment and intervention outcomes network registry-get with the guidelines). JACC, 3 428–430.
Patel M. P., Schettini P., O'Leary C. P., Bosworth H. B., Anderson J., B., Shah K. P. (2018). Closing the referral loop: An analysis of primary care referrals to specialists in a large health system. Journal of General Internal Medicine, 33, 715–772.
Piscotty R., Kalisch B., Gracey-Thomas A. (2015). Impact of healthcare information technology on nursing practice. Journal of Nursing Scholarship, 47, 287–293.
Rosswurm M., Larrabee J. (1999). A model for change to evidence-based practice. Journal of Nursing Scholarship, 31, 317–322.
Seng P. M.. (1994). The fifth discipline: The art and practice of the learning organization. New York, NY: Currency and Doubleday.
Singh H., Meyer A. N. D., Thomas E. J. (2014). The frequency of diagnostic errors in outpatient care: Estimations from three large observational studies involving US adult populations. BMJ Quality and Safety Journal, 23, 727–731.
Texas Tech University Health Sciences Center (2017). EMR Community News. Retrieved from
Weiner M., Hoyek E., Wang L., Dexter P., Zerr A. D., Perkins A., James F., Juneja R. (2009). A web-based generalist-specialist system to improve scheduling of outpatient specialty consultations in an academic center. Journal of General Internal Medicine, 24, 710–715.

Electronic referral; pilot project; primary care; quality improvement; rural; specialist

Supplemental Digital Content

© 2020 The Authors. Published by Wolters Kluwer on behalf of the American Association of Nurse Practitioners