This study was a retrospective electronic chart review study. All eConsults within the defined study period (394 eConsults) were reviewed and categorized retrospectively by clinical topic and type of question by one rater (F.S.). To ascertain agreement on the categorization of each eConsult, a random selection of 54 eConsults (approximately 14% of the total) was independently reviewed by the specialist (G.P.). Disagreements (five eConsults) were mainly regarding clinical topic chosen and were resolved through rereviewing and discussing the eConsult in detail until agreement was achieved between the lead author and the specialist. All data were exported into an Excel database for analysis. Descriptive analyses were used to quantify the most common clinical topics or question types.
After each eConsult, a mandatory survey is completed by the primary care provider. The primary care provider could not close a case and receive a transcript of the consultation without completing the survey. The survey was comprised of five questions (Box 1). Questions ranged from assessing the value of the eConsult service to determining whether a referral was avoided as a result of the eConsult.
Box 1 Post–Electronic Consultation Survey Detailed Questions and Answers Cited Here...
- Q1. Which of the following best describes the outcome of this eConsult for your patient:
- 1. I was able to confirm a course of action that I originally had in mind
- 2. I got good advice for a new or additional course of action
- 3. I did not find the response very useful
- 4. None of the above (please comment)
- Q2. As a result of this eConsult, would you say that:
- 1. Referral was originally contemplated but now avoided at this stage
- 2. Referral was originally contemplated and is still needed - this eConsult likely leads to a more effective visit
- 3. Referral was not originally contemplated and is still not needed - this eConsult provided useful feedback/information
- 4. Referral was not originally contemplated, but eConsult process resulted in a referral being initiated
- 5. There was no particular benefit to using eConsult in this case
- 6. Other (please comment)
- Q3. Please rate the overall value of the eConsult service in this case for your patient:
- Minimal 1 2 3 4 5 Excellent
- Q4. Please rate the overall value of the eConsult service in this case for you as a primary care provider:
- Minimal 1 2 3 4 5 Excellent
- Q5. We would value any additional feedback you provide
eConsult, electronic consultation.
Reprinted from Liddy C, Maranger J, Afkham A, Keely E. Ten steps to establishing an e-consultation service to improve access to specialist care. Telemed J E Health 2013;19:982–90.
Two time elements were analyzed; first, the amount of time between the primary care provider's submitting the eConsult until an answer was received (response time) and, second, the amount of time reported by the specialist to complete the eConsult. The specialist was given four timeframes to choose from: less than 10 minutes, 10–15 minutes, 15–20 minutes, and more than 20 minutes. Initially, these data were collected for the purposes of remunerating physicians for their time but they now serve as an outcome of interest in terms of the time commitment and feasibility of the service.
Of the 5,597 eConsults submitted to the service, 394 (7.0%) were directed to ob-gyns. Questions were submitted by 151 primary care providers—126 medical doctors and 25 nurse practitioners—91% with urban practices and 9% with rural practices. The mean age of patients included was 43.2±15.4 years (range 8–95 years old).
Pregnancy issues and gynecologic cancer screening issues were the most common topics in our study, accounting for 18% each (Fig. 2). Pregnancy issues included infertility, preconception planning, antepartum bleeding, antenatal screening, infections in pregnancy, medication safety as well as other symptoms and complaints of pregnancy. Obstetric concerns were grouped together because there were low numbers in each category and most related to antenatal care. Gynecologic cancer screening topics included abnormal Pap tests, endometrial hyperplasia, and postmenopausal bleeding. Vulvovaginal symptoms comprised 13%, abnormal uterine bleeding 12%, contraception 10%, menopause 9%, abnormal pelvic ultrasonography 8%, and uterine fibroid 3%. All eConsult topics that occurred less than 10 times were grouped into a separate category, “other.” Other topics included vulvovaginal symptoms (13%), abnormal uterine bleeding (12%), contraception (10%), menopause (9%), abnormal pelvic ultrasonography (8%), and uterine leiomyoma (3%).
Questions about drug treatments and general management questions were the most common question types in our study, comprising 28% and 27%, respectively. Drug treatment questions included inquiries about the drug of choice for a specific condition, safety of the drug in pregnancy as well as how specific drugs are usually prescribed (Fig. 1).
In the post-eConsult surveys, which were all completed in full, primary care providers were asked whether referral was originally contemplated but avoided as a result of this eConsult. In 135 of 394 (34.3%) of eConsults, a referral was originally contemplated but now avoided as a result of the eConsult. In 165 of 394 (41.9%) of cases, a referral was not planned but the eConsult provided useful feedback. In 73 of 394 (18.5%) of cases, referral was still needed based on specialist advice, but the eConsult resulted in a more effective visit, because the specialist provided recommendations for necessary workup before referring the patient. A second question in the post-eConsult survey asked primary care providers to evaluate the value of the eConsult for the patient. In more than 50% of cases, primary care providers got good advice for a new or additional course of action; in 46.7% of cases, primary care providers indicated that the eConsult helped them to confirm their existing plan of action or treatment.
The vast majority of primary care providers rated the overall value of the eConsult service very highly. On a scale from 1 (minimal) to 5 (excellent), 93.4% of primary care providers gave the service a rating of 4 or 5 on its overall value for patients, and 94.2% gave it a rating of 4 or 5 on its value for them as health care providers. Examples of comments provided by the primary care providers included: “clear response and good teaching for future cases,” “efficient, speedy, and reassuring. My patient was surprised about the technology and how quickly a response was obtained,” “fast and effective way of communicating with a specialist,” “having the opinion of the specialist helped in reassuring me and the patient about the next course of action,” “I have used eConsult several times now and find it extremely efficient and a resource that we are so lucky to have in our day-to-day practice,” “my patients are appreciating the quick replies to the eConsults,” “this eConsult provided an immediate plan of care, which both I and my patient were very pleased with,” “this is the perfect use of eConsult. Avoided a consult and helped adjust my overall practice in this situation based on the advice provided (ie, I was overscreening),” “timely answer. Great feedback. Offered useful resources.”
The vast majority of the consults required less than 10 minutes (80.5%) for the ob-gyn to complete, and nearly all of them were completed within 15 minutes (98.8%). Most of the remaining eConsults were completed within 20 minutes (Fig. 3).
This study reports on a secure, web-based eConsult service in obstetrics and gynecology. This highly valued, efficient system demonstrated change in care for patients by reducing the need for face-to-face referrals and providing advice that resulted in a new course of action by the primary care provider. Evidence for this change in referral practice from evaluation data is a surrogate outcome based on self-report by primary care providers, but numbers of actual consultations avoided in our province are not measurable. Moreover, the eConsult project would still be a success if the absolute number of consultations was not reduced but the actual consultations initiated were more efficient (ie, correct investigations ordered before the visit) or if patients who truly require face-to-face consultations were being seen. This may have vast implications for the future as e-health becomes more integrated into our health care system.
In this project, the eConsultant is paid a stipend based on the amount of time they report spending on the eConsult. From a practical perspective, could there be pushback from specialists who enjoy billing for easy consults? Certainly—but we believe that it is incumbent on physicians to try to be part of the solution to our health care woes, one of which being timely access to care for patients who really need it. We will all benefit from a more efficient and optimized health care delivery system.
A measure of the effectiveness of the eConsult service can be stipulated through the one-on-one teaching that occurs in each eConsult. It can also be drawn from the satisfaction reported by the primary care providers. In addition, the eConsult system provided a platform in which the primary care provider can communicate ultrasound reports, images, or laboratory values to the specialist. This could not have been possible if a phone consultation system was used.
Analysis of the types of questions and clinical topics received is a unique opportunity to understand clinical scenarios that primary care providers have questions about. This information could be used to inform planning of continuing medical and professional development events for primary care providers. Interestingly, primary care providers asked more questions regarding pregnancy issues than any other topic. In the gynecologic cancer screening topic, questions about endometrial hyperplasia and the normal width of the endometrium on ultrasonography were very common and for each eConsult on that topic, the specialist forwarded the specialty society guidelines on that topic in an effort to provide education. Therefore, eConsults are not only an efficient way of consulting a specialist, but may be used as a forum to share knowledge and increase the capacity of primary care providers to manage these types of clinical problems independently. Future research is needed to help determine whether focused, directed primary care provider education through eConsults or other focused medical education sessions based on the content of eConsults improves overall primary care provider knowledge in obstetrics and gynecology.
The eConsult service changed primary care provider referral patterns and the clinical course of action. The proportion of all cases in which referral was originally contemplated but now avoided as a result of the eConsult advice, 34%, is in a similar range as the average across all specialties within the Champlain BASE service (40%).11 If this service were more broadly available, there are huge potential savings for our health care system, both in terms of avoided face-to-face referrals, and also in terms of proving more timely care for patients and potentially reducing risks of further degradation or complication. In an economic analysis of the overall Champlain BASE service, factoring in its cost and the cost of traditional referral prevented, is expected to break even after approximately 7,800 eConsults.11
Our study showed that primary care providers highly valued the eConsult service. In 50.8% of the time, a suggestion for a new or additional course of action was received as a result of the eConsult regardless of whether a traditional referral was eventually recommended. In 46.7%, it confirmed an action that they already had in mind, which may help reassure the primary care provider and the patient that they are on the right path. The eConsult service was recognized as a valuable educational tool by primary care providers because they were more engaged in patient care through participating in the thought process involved in each consult.
Finally, it is important to notice that the eConsult service did not consume much of the specialist's time because it took less than 15 minutes to finish 98.8% of eConsults in our study, and in fact, the vast majority (80.5%) of eConsults were completed in less than 10 minutes. This is important because it can be used to educate new specialists who are interested to join the project and give them an estimate of how much time they need to commit in their schedule for this kind of service. It also provides useful information for deciding on the best payment models.
One limitation of our study is that all eConsults were answered by a single gynecologist. Potentially, the feedback for this one gynecologist's consultations might not be reflective of the service that would be provided by multiple gynecologists. However, the Champlain BASE service has processed more than 10,000 eConsults with similar results across all specialty services.
The eConsult service for obstetrics and gynecology has been successfully implemented in our region and addresses a wide range of clinical topics. It has the potential to reduce both the wait times to see a specialist and the frustrations of patients and primary care providers in our current congested health care system.
1. Carrière G, Sanmartin C. Waiting time for medical specialist consultations in Canada, 2007. Health Rep 2010;21:7–14.
2. Manuel DG, Maaten S, Thiruchelvam D, Jaakkimainen L, Upshu R. Primary care in the health care system. In: Jaakkimainen L, Upshur R, Klein-Geltink JE, Leong A, Maaten S, Schultz SE, et al., editors. Primary care in Ontario: ICES atlas. Toronto (Canada): Institute for Clinical Evaluative Sciences; 2006.
4. Berthelot JM, Sanmartin C. Access to health care services in Canada, January to December 2005. Ottawa (Canada): Statistics Canada; 2006.
6. Liddy C, Rowan MS, Afkham A, Maranger J, Keely E. Building access to specialist care through e-consultation. Open Med 2013;7:e1–8.
7. Keely E, Liddy C, Afkham A. Utilization, benefits, and impact of an e-consultation service across diverse specialties and primary care providers. Telemed J E Health 2013;19:733–8.
8. Liddy C, Maranger J, Afkham A, Keely E. Ten steps to establishing an e-consultation service to improve access to specialist care. Telemed J E Health 2013;19:982–90.
9. Straus SG, Chen AH, Yee H Jr, Kushel MB, Bell DS. Implementation of an electronic referral system for outpatient specialty care. AMIA Annu Symp Proc 2011;2011:1337–46.
10. Gray BH, Bowden T, Johansen I, Koch S. Issues in international health policy: electronic health records: an international perspective on “meaningful use.” Issue Brief (Commonw Fund) 2011;28:1–18.
© 2016 by The American College of Obstetricians and Gynecologists. Published by Wolters Kluwer Health, Inc. All rights reserved.
11. Liddy C, Deri Armstrong C, Drosinis P, Mito-Yobo F, Afkham A, Keely E. What are the Costs of Improving Access to Specialists through eConsultation? The Champlain BASE Experience. Stud Health Technol Inform 2015;209:67–74.