Accessing ambulatory care in academic medical centers can be a struggle for patients from all strata of society.1,2 Calls reach answering machines. Schedulers fail to register patients effectively or behave rudely. Appointments offered lie weeks away. These delays in care can have serious consequences; in fact, the Institute of Medicine included timeliness as one of its six aims for improving health care quality.3 Timely appointments might be made when the referring physician asks the specialist to squeeze the patient in, but this has costs in terms of the physicians' and support staffs' time.4 Patients may be placed on waiting lists for sudden “openings.” However, schedulers untrained in triage certainly cannot determine over the telephone who needs prompt care and who does not.
To improve both the experience of calling for an appointment and access to physicians, Beth Israel Deaconess Medical Center (BIDMC) used an approach, modified from a successful experience at the University of Pittsburgh Medical Center. We report here the methods we used, their effects, the costs of implementation, and possible downstream benefits. We also describe obstacles to success and how we managed them. The methods we describe continue to be used at both institutions.
Learning About the Issues: Measuring Access and Quality of Telephone Service
Excellent telephone service means prompt pickup by a person, not a machine. Schedulers identify themselves and exhibit courtesy and compassion. They obtain all relevant registration information and schedule appointments within the rules of the particular practice. Excellent access is the ability to see a physician (not a particular physician) within one to three days of the call.
Mystery shoppers measure scheduling skills and access time
Before our intervention, we tracked access by measuring, in BIDMC's computerized scheduling system, such things as the time between the scheduling and occurrence of appointments and the numbers of available slots. We rapidly concluded that these measures were insufficiently robust to guide improvement. First, rapid appointments did not always reflect preferences; a patient or referring doctor might request a delay before the visit. Second, the scheduling system could not always generate accurate reports on the numbers of open slots. Third, the limited appointment availability of some physicians' overfilled practices skewed the results of their colleagues. Finally, computer systems cannot examine the quality of telephone interactions.
We decided instead to follow the successful experience of the University of Pittsburgh Medical Center, which used “mystery shopping” to measure the quality and effectiveness of their schedulers. Mystery shopping is a tool used to measure, among other things, quality of service. The mystery shopper, whose identity is not known by the service provider being evaluated, performs specific tasks such as buying a product, asking questions, or registering complaints. The mystery shopper then provides detailed feedback to the evaluator. We developed mystery shopping protocols in which a professional, usually a nurse, posed as a patient or family member. The mystery shoppers called the clinics with standard, urgent vignettes appropriate for the given clinic and asked to schedule an appointment. The mystery shoppers noted the time it took schedulers to pick up, the number of times they were put on hold, and the total time they spent on the telephone. They recorded the number of days between the call and the first appointment offered with any doctor in that specialty. They also rated the schedulers' interpersonal, scheduling, and registration skills. The conversations were recorded and transcribed verbatim.
Initially, the mystery shoppers made second calls to cancel their appointments, but as the project progressed, they identified themselves at the ends of the calls and gave supportive, constructive criticism. This immediate coaching not only saved the time and effort of canceling the appointment but also helped schedulers improve their performance and reinforced their sense of being essential members of the health care team. Indeed, at the request of schedulers, the mystery shoppers began to publish a quarterly online newsletter that provides helpful hints for improving patients' scheduling experiences.
Using mystery shopper results to improve quality
Although mystery shopping helps measure an institution's performance, it is up to institutional leaders to improve that performance. Academic medical centers often divide responsibility for ambulatory clinics between physicians and administrators, and this division can prevent either group from managing the scheduling process effectively. To avoid this, at BIDMC we as chair and administrative director of the Department of Medicine (where our improvement effort began) meet monthly with a group comprising both the physician and administrator in charge of each clinic, and the hospital's senior ambulatory leadership. A few days before each meeting, the participants receive the mystery shopping results. During the meetings, we note and applaud successes (prompt, effective service; ready access to the clinic) and discuss approaches for improving weaknesses.
Quantifying costs of mystery shopping and downstream benefits of improved access
Using accounting systems, we tracked clinic volumes (measured as visits), work relative value units (RVUs), and gross revenue (charges) for the year before the intervention and thereafter. Estimating downstream benefit to increased access is imprecise. However, we were interested in changes in downstream revenue potentially resulting from our interventions. On this basis, we tracked collections resulting from patients seen in the clinics within one year of the visit. To estimate the increment in downstream revenue from improved access, we used the collection data and data on direct and indirect costs to generate a contribution margin related to the patients seen in the clinics. Contribution margin is calculated as:
Contribution margin was tracked for patients for one year following their clinic visits. Data were tracked for two years (from January 2004 through December 2005) preceding the interventions to provide a baseline period, and for the several years following the beginning of the intervention in January 2006.
What We Learned and What We Did: How Interventions Improved Quality of Scheduling and Access
Telephone scheduling service
We start with a transcript report of a mystery shopper who called the Infectious Disease Clinic to make an appointment at the beginning of the improvement process.
The receptionist answered the phone. After she introduced herself and the unit, I said, “I'd like to make an appointment for my husband, who has had recurrent staph infections over the last two years that started with a spider bite and has been treated three times with antibiotics.”
She asked his name and if he'd been to the hospital before. I told her his name and then said, “He just moved from New York and tried to get an appointment at your hospital with a new primary care physician about eight weeks ago when he had symptoms again, but the wait was so long, he couldn't wait. So he called his own primary care physician in New York, who ordered the antibiotics and told him to get an appointment with an infectious disease doctor if the antibiotics didn't help. He told my husband to get an appointment with an infectious disease doctor anyway in the near future to better address this recurrent problem.”
Her reply was, “Yes it does take a long time to establish yourself with a new primary care physician, but he needs to do that and he should do it before he sees an infectious disease doctor. The number to call is—.”
I said, “Thank you, and I will call, but we always do annuals with our own doctor, and my husband just had a physical in the fall, and his primary care physician in New York told him to get an appointment with an infectious disease specialist as soon as possible.”
She then said, “Well, once he sees his new primary care physician, he or she can refer him to us.”
I said, “We don't need a referral. We have a PPO for an appointment.”
“Is the infection gone?” she said.
I said, “Yes, but that isn't the point. It obviously is dormant now. Eight weeks ago it was so gross he couldn't even shave, and honestly I didn't want to share the same bed! To break out with a rash like this isn't normal and we don't want it to happen again, so could you please give me the next available appointment?”
She said, “I know how bad staph can be, but we can't treat him if it isn't active. I'm not clinical, but that is how it works. The next time he has the infection, call us and we'll get him in that day or the next.”
I said, “So I'm hearing you won't give me an appointment because he doesn't have an active rash?”
She said, “Hold on a minute please.” She came back to the phone and said she checked and that is what we should do because “he would be just coming in and talking to a doctor who really couldn't do anything for him now.”
“You mean there is no historical information they could discuss? Blood tests or a plan of actions to talk about?”
“So basically you are telling me I can't have an appointment?”
“Not now, it wouldn't help really. He'd see one doctor, and then when he gets sick, that doctor may not be on, so it would be another doctor who he'd have to start all over with. So I suggest you call the number I gave you, get a primary care physician appointment set up, and if he gets sick before he sees the primary care physician, just call us and we'll get him in that day or the next. I promise, okay?”
I said, “Well I guess, but his primary care physician in New York made it sound like this was the appropriate next step—to see an infectious diseases doctor. Thank you.”
Clearly, this clerical employee (with or without the advice of her supervisor) was not equipped to determine who should and who should not see a physician. In other vignettes, we saw schedulers make inappropriate or rude comments or otherwise fail to display the kind of respect and compassion that we would want patients to receive. Clinic leaders and frontline staff were understandably shocked and, in some cases, ashamed at the quality of telephone service demonstrated during these baseline calls.
Before our intervention began and during the baseline period, academic secretaries scheduled appointments for many of the physicians and, because scheduling was an intermittent task for them, it was difficult to ensure that they all performed reliably; some had difficulty navigating the scheduling system. Further confusing the matter, different physicians within the same clinic often had different scheduling rules or templates, with varying durations for individual and follow-up visits. Finally, when an academic secretary was unavailable, patients' calls went to a message machine.
To solve these problems, all scheduling was moved to specialized schedulers, who were easier to coach, who could master the scheduling software, and who would reliably answer the telephone, covering for each other when necessary. In addition, templates for each clinic were standardized, rendering the scheduling more consistent and reliable. Now, when a mystery shopper fails to get a live answer or an appointment, we notify that clinic's director and administrator and ask him or her to prepare a root cause analysis of the failure in service for the next meeting.
Figures 1 and 2 show the changes in telephone service at BIDMC as a result of these quality improvements. At baseline, a large number of mystery shopping calls were answered by a machine, and many failed to result in appointments being scheduled. Aside from a blip in July 2007, we have markedly reduced the numbers of unanswered calls and calls that fail to result in appointments. In our measurements of the schedulers' telephone service and registration skills, we see that both courtesy and registration skills have improved markedly, reaching levels above 90% and remaining there through multiple surveys.
We measured access as the number of business days between the telephone call and the first appointment offered with a doctor in the specialty. Figure 3 displays the average access time for all calls to all clinics made in a given month. Before improvement efforts, access was often very poor, with some specialties measured in weeks or months. In some cases, schedulers failed to identify reliably available clinic slots. These issues were resolved as discussed above, by hiring dedicated schedulers, training them, and simplifying scheduling rules.
In most cases, however, long access times were simply the result of not enough slots to meet the demand. Increasing the number of slots required that physicians devote more effort to ambulatory care. To motivate that effort, we tabulated transparent profit and loss statements and reported them to individual physicians quarterly. We also instituted incentive plans tied directly to work RVUs. These interventions increased the number of physicians who were actively interested in seeing more patients and helped persuade many to add clinic sessions.
Over time, we also noticed that heavy vacation periods often led to access delays. To ensure prompt access to clinics during vacations, we created coverage systems, much like those on inpatient services, with physicians adding extra clinic sessions to make up for those canceled during vacations. Many clinics also introduced a Friday afternoon “clean up” session, when any patient not accommodated during the week could be seen by a rotating faculty member.
After all other interventions were exhausted, it became necessary to hire physicians to provide additional ambulatory care. In some areas, most notably hepatology, delays in access at BIDMC have persisted because of difficulties recruiting enough faculty members. Since our intervention, access time has fallen from 12 days to 6 to 8 days. Our analysis omits data from the hepatology clinic because of that specialty's specific recruiting difficulties.
Access to general internal medicine also deserves special comment. Shortly before our intervention, BIDMC's general medicine division set up episodic care clinics, where patients who call with urgent problems can be seen that same day or the next. The clinics, which can add new patients relatively easily, might be said to meet our goals by ensuring that patients' immediate needs are dealt with. They do not, however, provide easy access to ongoing primary care. Although the increasing focus of internal medicine training programs on ambulatory care has meant more resident slots in the general medicine clinic, and these added slots have improved our patients' access to ongoing primary care, our data reflect only access for urgent visits. In any case, improved access to urgent care ensures that patients' needs are met while they wait to begin their primary care relationships.
Growth in ambulatory volume
Table 1 shows the numbers of visits in medicine clinics both before and after our interventions. It is apparent that ambulatory volume grew strikingly. To maintain good access, we often had to recruit new clinicians in a variety of disciplines and, when their practices quickly filled, hire more. As a side benefit of our intervention, once a clinic's access has been improved to take only a few days, any lengthening of wait times becomes immediately apparent. When we see this happen reliably over several months, we know it is time to add more clinical staffing to accommodate the need.
The growth in ambulatory volume at BIDMC's clinics was not the result of marketing; no such marketing occurred. It may have been due to the incentive plan that encouraged faculty to schedule more appointments, or to more general market conditions. However, because volume had not increased before our interventions, but began to grow and continued to do so during these efforts, it is reasonable to believe that, offered a choice, patients opt for physicians who provide effective telephone service and rapid access. This is also supported anecdotally: We have heard of patients coming to our physicians, and to physicians at the University of Pittsburgh Medical Center, when they could not be seen promptly elsewhere.
Economic impact of improved scheduling and access
The costs of the interventions described here include the costs of the mystery shopping program and of the time that leadership and frontline staff used to improve service. The mystery shopper program at BIDMC costs $50,000 per year. And, although the cost of increased time is difficult to estimate, we would argue that improving quality of service is a major responsibility of all staff engaged in ambulatory care and should, therefore, not be regarded as additional cost.
The economic impact of increased ambulatory volume can be difficult to gauge. Academic medical centers usually “lose money” on outpatient practices because reimbursements for clinic visits do not cover the costs of physician time, clinic staff, and other expenses (although the recent Medicare adjustments for work RVU favor evaluation and management services, including ambulatory care). From this perspective, academic medical centers may not want to improve telephone services, access times, or waiting room experiences because they see no economic upside to the resulting expansion of ambulatory volume. Such a view ignores the role that “patient satisfaction” plays in well-being, but it is still important to make an economic case for improved “customer service” in ambulatory care.
It is important to note that the faculty practice plan, from which physicians are compensated, often bears the brunt of the expense of improving access to ambulatory care. If, as occurs in nonprocedural disciplines, payments to physicians in ambulatory practices are relatively low, adding new physicians can lead to significant losses. Unless the academic medical center covers these losses, it is strongly against the financial interests of the department and the practice plan to improve access.
Of course, increasing ambulatory volume can also have important financial benefits for an academic medical center. Ancillary services, such as imaging, lab work, and other testing, which increase along with volume, represent a major source of income. More ambulatory volume also leads to more inpatient hospitalizations, operations and procedures, and downstream revenue. During our interventions, the number of inpatients at BIDMC grew strongly. Because other factors may have led to this growth, we have tried to measure the downstream impact of increased ambulatory volume by tabulating the revenues or margin coming from the patients seen in the clinics during the year following their visit.
We estimated that, before our intervention, the downstream contribution margin for specialty medicine at BIDMC averaged $2,465 per patient visit per year, and the net margin (the downstream collections minus all costs and overheads) averaged $71. General medicine visits yielded an average annual downstream contribution margin of $972 and an average annual net margin value of $33. Using these figures, in the two years preceding the intervention, the total downstream contribution margin from the medicine clinics was $381 million for specialty clinics and $83 million for general medicine, whereas net margins were $16.9 million and $2.8 million, respectively. With the increases in volume over the two years of the intervention, contribution margins have risen to $435 million for specialty clinics and $85 million for general medicine, whereas net margins have risen to $19.2 million and $2.9 million, respectively. Clearly, this approach is imprecise on many counts, including the fact that it double counts patients seen in multiple clinics. However, because we used the same approach to calculate both pre- and postintervention figures, many of those imprecisions should tend to cancel out.
It is apparent that important growth in revenue or margin can be attributed to growth in ambulatory volume. To determine the costs of growth in ambulatory volume, we have used data available at BIDMC to total up hospital and practice plan costs. We have then compared these costs with the increase in contribution margin realized by the hospital. The increase in contribution margin far outstrips the increase in cost, even using a limited view of the downstream revenue coming from increased ambulatory volume.
A Culture of Quality
The movement to improve the quality of health care, which rightly emphasizes patients' safety, has tended to focus on inpatient care. Telephone responsiveness and access to timely appointments may be less urgent than preventing central line infections or treating community acquired pneumonia, but it is important to note that many more people receive ambulatory care each year than receive inpatient care. BIDMC, for example, annually admits 12,000 patients for 16,500 medical and surgical procedures, whereas the physicians in the Department of Medicine alone see 90,000 individual patients across 150,000 visits. Clearly, improving the experience of ambulatory care can have a huge impact on the quality of life for many people. In addition, effective ambulatory care can prevent or ameliorate serious illnesses, thus avoiding hospital admissions in the first place.
Our study has several important limitations. Like many quality improvement efforts, there was no randomization, and, because the investigators cannot be blinded to the intervention, observer bias may have entered the study. Although we gathered the data prospectively, our comparisons are before and after, with no crossover in design. In addition, as often occurs in quality improvement efforts, the clinics undertook a variety of interventions, including limiting the numbers of schedulers, standardizing rules for making appointments, adding clinical sessions, and instituting incentive plans. Finally, the outcomes we tracked do not measure actual patient outcomes such as mortality or morbidity but focus instead on direct measurement of service quality and convenience for patients.
Because of these limitations, we do not know whether improved access reduced illness or death in our patients. We also do not know which interventions actually improved access and telephone skills. We cannot conclude definitively that improved customer service alone led to increased ambulatory volumes. Finally, we cannot determine precisely the financial impact of these interventions—first, because some of the increased volume may have been due to other factors, and, second, because the downstream benefits of increased ambulatory volume are hard to calculate.
Nonetheless, we believe that the interventions we describe are sensible approaches to improving quality of care and can be readily applied at other centers. Others apparently agree. When the Boston Globe covered our efforts,5,6 we received inquiries from hospitals around the country seeking to emulate our program. In response, we have held two blog-assisted phone conferences to describe and discuss the interventions. The nationwide interest also led us to prepare this article.
We regularly describe these efforts in detail to our graduate trainees and medical students as part of their quality improvement curricula, to cultivate a culture of quality within our programs and the institution.7 It is our hope that they, like us, will desire to treat patients the way we would want our own families treated: with humane, effective care that starts from the first telephone call and continues through all of a patient's experiences with the health care system.
1. Levine DM, Becker DM, Bone LR, Hill MN, Tuggle MB, Zager SL. Community–academic health center partnerships for underserved minority populations: One solution to a national crisis. JAMA. 1994;272:309–311.
2. Cohen JJ, Dickler RM, Griner PF, et al.. Meeting the Needs of Communities: How Medical Schools and Teaching Hospitals Ensure Access to Clinical Services. Washington, DC: Association of American Medical Colleges; 1998.
3. Committee on Quality of Health Care in America, Institute of Medicine. Crossing the Quality Chasm. Washington, DC: National Academies Press; 2001.
4. Alexander GC, Kurlander J, Wynia MK. Physicians in retainer (“concierge”) practice: A national survey of physician, patient, and practice characteristics. J Gen Intern Med. 2005;20:1079–1083.
5. Leddy KM, Kaldenberg DO, Becker BW. Timeliness in ambulatory care treatment: An examination of patient satisfaction and wait times in medical practices and outpatient test and treatment facilities. J Ambul Care Manage. 2003;26:138–149.
6. Kowalczyk L. Long waits for doctors targeted: Hospitals urge faster response. Boston Globe. , 2005.
7. Kowalczyk L. On sly, workers rate hospital service. Boston Globe. , 2007.
8. Aronson MD, Neeman N, Carbo A, et al.. A model for quality improvement programs in academic departments of medicine. Am J Med. 2008;121:922–929.
Funding for this quality improvement effort was provided by Beth Israel Deaconess Medical Center.
Not applicable.© 2012 Association of American Medical Colleges