Caring for linguistically diverse patients and families is of growing importance to all medical providers. Nowhere is the population of linguistically diverse patients growing more rapidly than in pediatrics. In 2006, 13.7% of U.S. children under five years old were living in a home where a parent or guardian spoke English less than “very well.”1 These families are considered limited English proficient and are likely to need interpretation when communicating with medical providers.
Many students and residents enter careers in medicine with some level of skill in a second language. In a 2009 national survey of U.S. third-year pediatric residents, 72% had learned a second language prior to residency, and 26% reported learning a language during residency.2 Providers with limited second language proficiency report using these language skills in clinical contexts.3–6 Use of nonproficient language skills may significantly impede communication with patients and families, increasing the risk for misunderstandings,7,8 medical errors,9,10 and decreased patient and family satisfaction.11,12 Federal law and accrediting bodies consequently require that hospitals ensure the competence of individuals providing interpretation or bilingual medical care to patients.13,14
For all of these reasons, it is important that providers not use nonproficient language skills in clinical settings. However, providers’ ability to accurately assess their own skills and competencies has been found to be poor compared with external evaluations, especially among those with the lowest skill levels.15,16 Whether that is also the case for self-assessment of language proficiency is unknown. The lack of an accepted standard for defining the degree of proficiency needed to provide safe medical care complicates the task of assessment. Most authors advocate for a stringent definition to decrease the risk of medical errors and ensure equitable care, but no consensus exists on how to arrive at such a definition.17–20
This study explores several related questions about pediatric resident use of clinical Spanish when caring for patients. Our first objective was to explore the relationship of resident Spanish proficiency level (tested and self-reported) with residents’ use of and comfort in using their Spanish language skills in clinical contexts. Second, we wanted to determine the positive predictive value (PPV) and negative predictive value (NPV) of self-reported language proficiency compared with an objective test of Spanish proficiency. Given the lack of a standard cutoff for defining proficiency, we aimed to evaluate predictive values of self-report for several levels of tested proficiency.
This was a cross-sectional study that was approved by the institutional review board at each site. Study coordinators at each site linked participant survey and proficiency testing data, then deidentified all records prior to review and analysis by study investigators.
Protocol and participants
In March 2010, we e-mailed all 247 pediatric residents training at the University of Washington/Seattle Children’s Hospital (Seattle, Washington), Johns Hopkins School of Medicine (Baltimore, Maryland), and Children’s Mercy Hospitals and Clinics (Kansas City, Missouri). The e-mail notified them of the study and invited those with any level of Spanish language ability to participate. Residents who did not respond to the initial e-mail received up to two reminder e-mails. We obtained participants’ consent as they enrolled. All participating residents received a small gift of chocolates.
Our analysis includes only those residents who completed both a baseline pretest survey, administered via a secure online server through the University of Washington, and a telephone-based test of their Spanish oral proficiency and comprehension. We confidentially shared the results with each resident several days after testing; those results were neither added to residents’ official files nor shared with program directors.
The residency program coordinators provided us summaries of the demographic data for the cohorts of all pediatric residents at each institution, which we used for comparison with our study’s participants.
Baseline pretest survey measures
The baseline survey, developed and pilot-tested before the study, collected demographic data (year of residency, sex, age, and self-reported race/ethnicity) and asked questions regarding Spanish language skill and use. Participants rated their ability to speak about health and pediatric medicine in Spanish (none, rudimentary, basic, conversational, proficient, or fluent). They reported their average monthly frequency of Spanish use (less than once per month, 1–7 days, 8–14 days, 15–21 days, or >21 days) and how often they used an interpreter (always, usually, about half the time, usually not, or never). They then rated on a Likert scale (from 1 = “very uncomfortable” to 5 = “very comfortable”) how comfortable they would be using Spanish in 14 hypothetical clinical scenarios. We grouped these scenarios into four levels of complexity: no clinical content (e.g., introducing yourself and making small talk), straightforward clinical content (e.g., management of wheezing), complex clinical content (e.g., management of wheezing in a child with midline thoracic scar), and clinical content with legal implications (e.g., obtaining consent for a procedure). We placed the scenarios into these categories for analysis because residents have been shown to assess their need for interpreters based on their prediction of the complexity of an encounter, despite the fact that one can never know ahead of time how straightforward or complex an actual encounter will be.4,6 The complete list of scenarios can be found in the Appendix.
Participants completed the Spanish Speaking and Listening Test, a standardized 20-minute oral proficiency test administered by ALTA language testing service. The telephonic test consisted of 12 questions selected from a pool of 120 items. Each question was chosen to evaluate one or more expression or comprehension skill so that the complete test addressed all target skills. Although the general test did not include specific medical content, it has been previously used to evaluate language proficiency in medical trainees.20 Since completing our study, a language test incorporating medical content has become available from the same company.21 No data exist comparing performance of medical personnel on the two tests.
Tests were administered and scored by a trained native Spanish speaker, educated at the university level in a Spanish-speaking country. They had no information about each participant apart from his or her name and organization and based their evaluations on general vocabulary, grammar, speaking speed, sentence structure complexity, and ability to convey ideas in Spanish. ALTA provided us with the scores and brief qualitative evaluations.
Scores ranged from 1 (no knowledge of the language) to 12 (native speaker) and can be mapped to other language proficiency test scales, such as the Interagency Language Roundtable scale.22 On the basis of previous research, we labeled an ALTA score of 9 or above as “proficient.”20 The qualitative description provided by ALTA for a score of 9 is that the individual can successfully handle a variety of communicative tasks, with some errors of grammar or syntax that do not significantly impact understanding.22 Because a provider with a score of 9 or 10 still makes errors that might impact meaning and may have difficulty with advanced tenses and particular dialects or slang, we also conducted analyses for participants scoring 11 or 12, labeled for this study as “highly proficient.” These individuals are described as nearly fluent or fluent, and the few errors they do make do not impact understanding. Given the importance of communication for safe medical care, some institutions might choose to define proficiency using this more stringent criteria.
We present demographic data from the pretest survey and residency program coordinators as percentages. We grouped data relating to self-reported proficiency by tested proficiency and analyzed them using Fisher exact test. We describe further analyses below by objective.
Use of Spanish in clinical contexts.
We evaluated self-reported frequency of Spanish use and frequency of interpreter use, by self-reported and tested proficiency, with chi-square and Fisher exact tests. To facilitate analysis, we collapsed frequency of Spanish use responses from five categories into two (≤7 days per month and >7 days per month).
We dichotomized residents’ responses relating to comfort using Spanish in clinical and nonclinical scenarios into “yes” for Likert responses of “comfortable” and “very comfortable,” and “no” for all other responses. If a resident answered “yes” to any scenario in one of the four categories, we labeled that category “yes” in the clinical scenario analysis. We analyzed differences in percentages of respondents comfortable with each category by tested proficiency using chi-square and Fisher exact tests.
Self-reported versus tested proficiency.
We dichotomized self-reported proficiency, categorizing responses of “proficient” or “fluent” as “proficient” and responses of “rudimentary,” “basic,” and “conversational” as “not proficient.” Similarly, with tested proficiency, we categorized scores of ≥9 as “proficient” and scores of <9 as “not proficient.” We defined scores of ≥11 as “highly proficient,” a subset of the “proficient” group. These categories were determined based on ALTA’s published skill level descriptions and correspond to those used by other authors.20 We calculated the PPV and NPV of self-assessed proficiency predicting tested proficiency using both score cutoffs (≥9 and ≥11), along with the corresponding confidence intervals (CIs).
Of the 247 pediatric residents at the three training institutions, 84 agreed to participate in the study. Six residents who reported having some Spanish language skills in the pretest were unavailable to take the oral language test because of scheduling challenges with out-of-state/out-of-country rotations and so were excluded from analysis. This resulted in 78 participants, or 32% of all residents. Not knowing precisely how many residents spoke any Spanish and so were eligible to participate, we cannot know the true study response rate, but, on the basis of our familiarity with the three cohorts, we believe that nearly all residents with Spanish-speaking ability did participate.
Participants’ distribution of sex, age, training year, and self-reported race/ethnicity was similar to that of all residents at the three sites (Table 1). Participants were also similar across the three training institutions (data not shown).
Use of Spanish for clinical care by self-reported proficiency
Residents at every level of self-assessed proficiency reported sometimes using Spanish to communicate directly with patients and families. Over one-quarter of providers with self-described “rudimentary” Spanish (6 of 23), and over three-quarters of those with “basic” Spanish (13 of 17), communicated without an interpreter at least some of the time (Table 2). Residents who considered their Spanish to be conversational were almost universally comfortable using it in straightforward clinical scenarios (95%, 19 of 20), although only 30% of those residents tested at a level considered proficient enough for clinical care (6 of 20). Among residents rating their Spanish as proficient, all were comfortable using it for nonclinical and straightforward clinical care, and large majorities were comfortable using it in complex and legal scenarios (77% and 85%, respectively). Just over half (54%, 7 of 13) of these residents tested at a proficient level.
Use of Spanish for clinical care by tested level
Over half (58%) of residents who tested at a proficient level (11 of 19) and 15% of residents who did not (9 of 59) reported using Spanish more than seven days per month (Table 3). All residents who tested proficient reported that they sometimes discussed clinical care with families with limited English proficiency without a professional interpreter. Of the residents testing nonproficient, 63% reported that they sometimes discuss clinical care with such families without an interpreter (37 of 59). Less than one-third (28%) of participating residents (22 of 78) reported using an interpreter for all clinical communications with families with limited English proficiency.
Nearly two-thirds of all participants (64%, 50 of 78) reported comfort using Spanish in straightforward medical scenarios. Residents were much less likely (23%) to feel comfortable using Spanish when presented with a medically complex scenario (18 of 78). Twenty-four percent of all participating residents and 68% of residents who tested proficient (13 of 19) reported comfort using Spanish in scenarios with potential legal implications, such as consent for procedures or disclosing a medical error. Among the 59 nonproficient residents, 10% (6) reported that they were comfortable using Spanish in complex clinical scenarios, and 10% (6) reported comfort in medical-legal scenarios. Residents who tested proficient were significantly more likely to report comfort using Spanish in all types of clinical scenarios (nonclinical P < .001; straightforward clinical P = .012; complex clinical P < .0001; legal P < .0001).
Association between self-reported and tested language proficiency
Tested language proficiency scores ranged from 2 to 12. Twenty-four percent of participants (19 of 78) scored at a level considered proficient or better (9–12) on the ALTA test. However, only 6% (5 of 78) were considered “highly proficient” with a score of 11 or higher. When defining proficiency with the cutoff of 9, the PPV of self-reporting Spanish proficiency was 67% (95% CI 41%–86%). Thus, one in three residents self-reporting proficiency did not test at that level. The NPV was 88% (95% CI 77%–95%). Using the more stringent criteria for high proficiency of 11 or 12, the PPV dropped to 22% (95% CI 7%–48%), and the NPV increased to 98% (95% CI 90%–99.9%).
Pediatric residents possess a wide range of Spanish language skills. Similar to other studies, we found that residents with all levels of Spanish language skills routinely care for Spanish-speaking families without professional interpretation.3,4,6 Even residents who are aware that they do not speak Spanish well deliver care without professional interpretation, including in situations involving complex medical issues and potential legal ramifications. Nearly half of the residents who considered their Spanish skills to be basic were comfortable providing clinical care without an interpreter, as were nearly all residents with self-assessed “conversational” skills. However, less than a third of the residents with self-reported conversational skills, and none with self-reported basic skills, tested at a level considered safe for clinical use. Many residents may feel that common or straightforward medical problems require only straightforward language for adequate communication with a family. Serious concerns or questions that a family raises may be poorly understood or poorly addressed by a nonproficient provider who was expecting a “straightforward” medical encounter.
When providers with limited proficiency use non-English language skills in a clinical context, they risk miscommunication. Language barriers are associated with decreased patient and family satisfaction,12,23 decreased understanding of and adherence to medical therapy,24,25 increased expenditures of time and money,26,27 increased risk of medical errors,8–10,28 increased length of stay and hospital readmission,29 and heightened risk of costly litigation.30 Professional interpreters improve communication with families and quality of care.31–34 In contrast, the common practice of ad hoc interpretation by family members or untrained bilingual staff carries significant risk of miscommunication from errors in interpretation.8,31,35,36
Previous studies among other types of medical providers have reported high degrees of concordance between self-reported and externally measured proficiency.20,37 In this study among pediatric residents, however, we found self-report to have moderate to poor predictive validity compared with a standardized test. Within our small group of residents reporting proficiency, 67% had ALTA test scores in the “proficient” range. Using the more stringent cutoff of 11, only 22% qualified as “highly proficient.” The residents who self-reported conversational or proficient skills—a group with relatively high levels of comfort using Spanish in clinical settings—had rates of tested proficiency of only 30% and 54%, respectively. On the basis of these findings, we concluded that self-report of Spanish proficiency is not an adequate predictor of measured general proficiency among residents. This finding has important implications for residency program directors and medical educators entrusted with modeling and ensuring high-quality communication with patients and families with limited English proficiency cared for by residents. In particular, it draws attention to the group of residents reporting conversational or proficient (but not fluent) language skills, who may be very comfortable using their Spanish clinically but may not have the skills to do so safely. Although universal proficiency testing would be preferable, residency programs with limited resources could begin by targeting testing to the group who self-report conversational or proficient skill levels.
Our study had several limitations. The number of participants within each category of self-reported and tested proficiency was small, limiting the precision of the calculated predictive values. Residents’ use of medical Spanish in clinical settings was self-reported and not directly observed. Regarding our choice of test, an important limitation of the ALTA Speaking and Listening test was the lack of medical content. The test measures general Spanish proficiency and comprehension, which is important in understanding the concerns of patients and families, but it may not accurately assess the skills necessary for effective clinical care. The results of a test including medical content might differ in two ways from those we obtained: Residents with good general proficiency in Spanish might lack the medical vocabulary necessary to communicate effectively about health; or, as many of our participants reported, medical Spanish might be a strength for residents, and so performance could be better than what we observed. An additional challenge is the lack of evidence to guide criteria for defining what degree of proficiency is needed to provide safe, equitable care to patients with limited English proficiency. As hospitals develop policies that measure the Spanish proficiency of employees and providers, they must carefully define the specific communication skills needed by various hospital personnel and how those skills should be tested.
Implications and Conclusions
Effective communication is essential for reducing medical errors and providing excellent care in our increasingly diverse communities. The Joint Commission has identified interpreted communication as a core measure of quality care for patients with limited English proficiency.13 In addition, the U.S. Department of Health and Human Services has mandated that all hospitals and health care providers receiving federal money “must assure the competence of language assistance provided to limited English proficient patients/consumers by interpreters and bilingual staff.”14
Our study suggests that many residents who know they are not proficient nonetheless provide care without professional interpretation, and an additional cohort report themselves to be proficient but do not demonstrate proficiency on an objective test. Medical educators and hospital administrators are faced with the challenge of identifying both groups of residents and intervening to ensure safe, effective, high-quality communication with families with limited English proficiency.
Although ensuring appropriate use of professional interpretation is the most immediate solution, many residents express a desire to improve their language proficiency in order to provide safe bilingual care. Medical Spanish courses may inadvertently increase disparities for families with limited English proficiency by decreasing interpreter use without leading to true proficiency.38,39 A recently described novel model for safe, supervised language skill development within the context of a resident continuity clinic includes objective language proficiency testing to measure progress and the presence of a professional interpreter in the room while the resident is providing care (for those not yet proficient), in order to remedy any miscommunications that may occur and to provide feedback.17 The model has resulted in improved continuity of care and satisfaction for Spanish-speaking families and improved Spanish proficiency for all participating residents. This program serves as an excellent example of safe, supervised language use and skill development that conveys a clear message to trainees that demonstrating language competence is as important to high-quality patient care as any other skill taught and assessed during residency.
Ensuring the best care for every patient will require consistent screening for language need, efficient professional interpretation, and recognition and support for the unique skills that bilingual providers can contribute. Hospitals and residency programs can move towards achieving those goals by establishing “language credentialing” for providers: objectively measuring provider language skill, and establishing and enforcing expectations of professional interpretation for those who are not proficient. Just as residents are expected to document proficiency in performing technical procedures such as intubation or central line placement before performing these tasks without supervision, documented language proficiency is a testable skill that should be measured and reported.
2. Laraque D, Mendoza F, Dreyer B, Frintner MP, Cull W Cross cultural and linguistically appropriate care: pediatric resident preparedness. 2010Paper presented at: 2010 Pediatric Academic Societies Annual Meeting Vancouver, British Columbia, Canada
3. Burbano O’Leary SC, Federico S, Hampers LC. The truth about language barriers: One residency program’s experience. Pediatrics. 2003;111(5 pt 1):e569–e573
4. Diamond LC, Schenker Y, Curry L, Bradley EH, Fernandez A. Getting by: Underuse of interpreters by resident physicians. J Gen Intern Med. 2009;24:256–262
5. Yawman D, McIntosh S, Fernandez D, Auinger P, Allan M, Weitzman M. The use of Spanish by medical students and residents at one university hospital. Acad Med. 2006;81:468–473
6. Evans Y, Rafton S, Hencz P, Michel E, Fernandez J, Ebel BE. Provider language proficiency and use of interpretation in caring for limited English proficiency children and families. 2008 Pediatric Academic Societies Annual Meeting. May 5, 2008
7. Flores G, Abreu M, Schwartz I, Hill M. The importance of language and culture in pediatric care: Case studies from the Latino community. J Pediatr. 2000;137:842–848
8. Flores G, Laws MB, Mayo SJ, et al. Errors in medical interpretation and their potential clinical consequences in pediatric encounters. Pediatrics. 2003;111:6–14
9. Cohen AL, Rivara F, Marcuse EK, McPhillips H, Davis R. Are language barriers associated with serious medical events in hospitalized pediatric patients? Pediatrics. 2005;116:575–579
10. Divi C, Koss RG, Schmaltz SP, Loeb JM. Language proficiency and adverse events in US hospitals: A pilot study. Int J Qual Health Care. 2007;19:60–67
11. Flores G. Language barriers to health care in the United States. N Engl J Med. 2006;355:229–231
12. Morales LS, Cunningham WE, Brown JA, Liu H, Hays RD. Are Latinos less satisfied with communication by health care providers? J Gen Intern Med. 1999;14:409–417
13. Joint Commission. Advancing Effective Communication, Cultural Competence, and Patient- and Family-Centered Care: A Roadmap for Hospitals. 2010 Oakbrook Terrace, Ill Joint Commission;
14. U.S. Department of Health and Human Services Office of Minority Health. National Standards for Culturally and Linguistically Appropriate Services in Health Care. 2001 Washington, DC U.S. Department of Health and Human Services Office of Minority Health
15. Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L. Accuracy of physician self-assessment compared with observed measures of competence: A systematic review. JAMA. 2006;296:1094–1102
16. Colthart I, Bagnall G, Evans A, et al. The effectiveness of self-assessment on the identification of learner needs, learner activity, and impact on clinical practice: BEME guide no. 10. Med Teach. 2008;30:124–145
17. Cowden JD, Thompson DA, Ellzey J, Artman M. Getting past getting by: Training culturally and linguistically competent bilingual physicians. J Pediatr. 2012;160:891–892.e891
18. Diamond LC, Luft HS, Chung S, Jacobs EA. “Does this doctor speak my language?” Improving the characterization of physician non-English language skills. Health Serv Res. 2012;47(1 pt 2):556–569
19. Ferguson WJ. Un Poquito. Health Aff (Millwood). 2008;27:1695–1700
20. Reuland DS, Frasier PY, Olson MD, Slatt LM, Aleman MA, Fernandez A. Accuracy of self-assessed Spanish fluency in medical students. Teach Learn Med. 2009;21:305–309
22. ALTA Language Testing Services. Oral Language Proficiency Testing. 2003 Atlanta, Ga ALTA Language Testing Services
23. Mazor SS, Hampers LC, Chande VT, Krug SE. Teaching Spanish to pediatric emergency physicians: Effects on patient satisfaction. Arch Pediatr Adolesc Med. 2002;156:693–695
24. Kravitz RL, Helms LJ, Azari R, Antonius D, Melnikow J. Comparing the use of physician time and health care resources among patients speaking English, Spanish, and Russian. Med Care. 2000;38:728–738
25. Wilson E, Chen AH, Grumbach K, Wang F, Fernandez A. Effects of limited English proficiency and physician language on health care comprehension. J Gen Intern Med. 2005;20:800–806
26. Hampers LC, McNulty JE. Professional interpreters and bilingual physicians in a pediatric emergency department: Effect on resource utilization. Arch Pediatr Adolesc Med. 2002;156:1108–1113
27. John-Baptiste A, Naglie G, Tomlinson G, et al. The effect of English language proficiency on length of stay and in-hospital mortality. J Gen Intern Med. 2004;19:221–228
28. Jackson JC, Nguyen D, Hu N, Harris R, Terasaki GS. Alterations in medical interpretation during routine primary care. J Gen Intern Med. 2011;26:259–264
29. Lindholm M, Hargraves JL, Ferguson WJ, Reed G. Professional language interpretation and inpatient length of stay and readmission rates. J Gen Intern Med. 2012;27:1294–1299
31. Flores G. The impact of medical interpreter services on the quality of health care: A systematic review. Med Care Res Rev. 2005;62:255–299
32. Karliner LS, Jacobs EA, Chen AH, Mutha S. Do professional interpreters improve clinical care for patients with limited English proficiency? A systematic review of the literature. Health Serv Res. 2007;42:727–754
33. Moreno G, Tarn DM, Morales LS. Impact of interpreters on the receipt of new prescription medication information among Spanish-speaking Latinos. Med Care. 2009;47:1201–1208
34. Moreno G, Morales LS. Hablamos juntos (together we speak): Interpreters, provider communication, and satisfaction with care. J Gen Intern Med. 2010;25:1282–1288
35. Moreno MR, Otero-Sabogal R, Newman J. Assessing dual-role staff-interpreter linguistic competency in an integrated healthcare system. J Gen Intern Med. 2007;22(suppl 2):331–335
36. Flores G, Abreu M, Barone CP, Bachur R, Lin H. Errors of medical interpretation and their potential clinical consequences: A comparison of professional versus ad hoc versus no interpreters. Ann Emerg Med. 2012;60:545–553
37. Rosenthal A, Wang F, Schillinger D, Pérez Stable EJ, Fernandez A. Accuracy of physician self-report of Spanish language proficiency. J Immigr Minor Health. 2011;13:239–243
38. Diamond LC, Jacobs EA. Let’s not contribute to disparities: The best methods for teaching clinicians how to overcome language barriers to health care. J Gen Intern Med. 2010;25(suppl 2):S189–S193
39. Prince D, Nelson M. Teaching Spanish to emergency medicine residents. Acad Emerg Med. 1995;2:32–36
Appendix Text of Instructions and Clinical Scenarios for Which Participants Were Asked to RateTheir Comfort Using Spanish With Patients and Families, Grouped by Analysis Category, Three Institutions, 2010
Please rate your level of comfort in providing the following types of care in spoken Spanish on a scale of 1 to 5, with 1 meaning very uncomfortable, and 5 meaning very comfortable.
1. Introducing yourself and making small talk with a patient’s family
2. Giving a family verbal directions to the cafeteria
3. Asking about a child’s pain overnight
4. Discussing the initiation of care for a wheezing child before the interpreter arrives
5. Midnight rounds with a family who speaks some English
6. Discussing treatment for constipation in a hospitalized patient
7. Answering brief questions about healthy newborn care
8. Discussing the initiation of care for a wheezing child with midline thoracic scar before the interpreter arrives
9. Discussing concern for intimate partner violence with a patient’s mother
10. Discussing a transition to comfort care for a child in the ICU
Clinical Scenarios With Legal Implications
11. Discussing risks and benefits with a family who is refusing the recommended treatment
12. Consent for PICC line placement
13. Consent for biopsy of a lymph node
14. Discussing a medication error without clinical consequence