Measurement and Correlates of Physicians’ Lifelong Learning

Abstract

Purpose 

To examine the psychometric properties and correlates of an instrument to measure physicians’ orientation toward lifelong learning with attention to differences between full-time and academic clinicians.

Method 

The authors mailed a survey in 2006 to a national sample of 5,349 alumni of Jefferson Medical College who graduated between 1975 and 2000; 3,195 (60%) responded. The respondents were classified as full-time clinicians (n = 1,127) and academic clinicians (n = 1,612). The other 456 respondents were involved in administration or research. The revised Jefferson Scale of Physician Lifelong Learning (JeffSPLL) was included in the survey. Factor analysis, regression analysis, and analysis of variance were used to examine the construct- and criterion-related validities of the scale.

Results 

Factor analysis of the JeffSPLL items resulted in three factors designated as “learning beliefs and motivation,” “attention to learning opportunities,” and “skills in seeking information,” which supported its construct validity. Alpha reliability coefficients were 0.85 and 0.86, and test-retest reliability coefficients were 0.72 and 0.77 for full-time clinicians and academic clinicians, respectively. For full-time clinicians and academic clinicians, scores on the JeffSPLL were significantly (P < .01) correlated with measures of learning motivation, professional accomplishments, career satisfaction, and commitment to lifelong learning, which supported the criterion-related validity of the scale.

Conclusions 

The findings indicate that the JeffSPLL is a psychometrically sound instrument that measures physicians’ orientation toward lifelong learning among full-time clinicians and academic clinicians. The instrument can be used to monitor educational programs, assess educational outcomes, and examine group differences.

Lifelong learning has been identified as an element of professionalism in medicine.¹ Educating medical students and physicians to become lifelong learners has been the most consistent recommendation made by professional organizations.^2,3 However, empirical research on the topic is scarce because of the ambiguity associated with its definition as well as the lack of an instrument to measure it among physicians.

In previous research, we reviewed the literature and defined lifelong learning as an attribute involving a set of self-initiated activities and information-seeking skills with sustained motivation to learn and the ability to recognize one’s own learning needs.^4,5 The four key concepts in this definition that have frequently been described in the literature^6,7 are italicized to underscore their importance.

Although a few instruments have been used to measure self-directed learning in the general adult population,^8–10 none is specific to physicians. Added to the conceptual and measurement complexities is the fact that the learning priorities, activities, and outcomes for physicians involved exclusively in patient care can be different from physicians who, in addition to patient care, are also involved in research and teaching. We designed the study reported here to examine the psychometric properties of an instrument developed specifically to measure orientation toward lifelong learning in different groups of physicians, and to investigate correlations with indicators of academic achievement, motivation, professional accomplishment, and career satisfaction.

Method

Study sample and instruments

The total sample included all 5,612 physicians who graduated from Jefferson Medical College between 1975 and 2000, excluding 59 who were deceased as of 2006, the year of our survey.

The survey included a revised version of the Jefferson Scale of Physician Lifelong Learning (see Chart 1). The scale’s scores can range from a minimum of 14 to a maximum of 56. Higher scores indicate more orientation toward lifelong learning. Step-by-step procedures in the development and psychometrics (validity, reliability) of the original scale have been reported.^4,5 The original scale contained 19 items that targeted mostly physicians who were involved in patient care as well as research and teaching.

Chart 1 Revised Jefferson Scale of Physician Lifelong Learning

Additional questions in the survey asked about time devoted to patient care, research, and teaching. Also included were

• 13 questions (answered on a four-point scale) as indicators of learning motivation (e.g., “Regardless of my busy professional schedule, I always feel the motivation to learn about new advances in medicine”);
• 11 questions as indicators of professional accomplishments (e.g., “Presented patient education/research findings on radio, on TV, in a newspaper or before a community group”);
• self-reported rating of commitment to lifelong learning (“Please rate yourself in lifelong learning by checking a point of the following 10-point scale,” 1 = Not committed to lifelong learning, 10 = Very much committed to lifelong learning); and
• a single question on career satisfaction (“In general, how satisfied are you with your career?”); a 10-point scale was given, with 1 = not satisfied at all, and 10 = extremely satisfied.

Medical school class rank (top 25%, middle 50%, or bottom 25%) was calculated by retrieving course and clerkship grades from the database of the Jefferson Longitudinal Study.¹¹ The algorithm for calculation of class rank and evidence in support of its predictive validity have been reported.^12,13

Procedures

After approval by the university’s institutional review board, we undertook a pilot study to refine the content of the original version of the Jefferson Scale of Physician Lifelong Learning (JSPLL) to make it content-appropriate for physicians who were exclusively involved in patient care (full-time clinicians) as well as other patient-care physicians who were also involved in teaching and research (academic clinicians). We asked 50 clinical faculty in the Jefferson Health System to review the 19 items of the original JSPLL and to judge the “face validity,” or soundness, of each item as being relevant to physicians who are full-time clinicians and academic clinicians. They were also asked to make suggestions to improve the clarity of the items. On the basis of feedback from 30 physicians, we made modifications in the scale.

We excluded five items which were judged by more than 33% of physicians in the pilot study as being inappropriate indicators of lifelong learning for physicians who were exclusively involved in patient care. In our previous factor analytic studies of the original 19-item scale, these five items had significant factor coefficients on a construct entitled “research interest and academic activities.”^4,5 Therefore, the revised version of the Jefferson Scale of Physician Lifelong Learning (JeffSPLL) used in this study contained 14 items that were equally suitable for physicians involved exclusively in patient care and others involved in patient care as well as research and teaching.

Using addresses obtained from our alumni office, we sent postcards describing the study to inform physicians that the survey would be mailed in two weeks. The purpose of this mailing was to identify undeliverable addresses or forwarding addresses. Subsequently, we sent the survey accompanied by a cover letter and a postage-paid return envelope to 5,349 living graduates with valid addresses in 2006. We assigned a code to each survey to conceal respondents’ identities during data collection and processing to enable us to follow up with nonrespondents and to correlate responses to medical school data. We explained the purpose of the code in a cover letter. The following statement was printed on the survey: “Your completion and return of this survey is an indication of your agreement to voluntarily participate in this study, approved by the Thomas Jefferson University Institutional Review Board.” After three follow-up reminders, we received 3,195 completed surveys. To examine the stability of responses (test-retest reliability), we mailed a second copy of the survey to 200 physicians who responded to the first mailing.

Statistical analyses

We used factor analysis (principal component factor extraction, varimax rotation), bivariate correlation, multivariate regression analysis, t test, analysis of variance, and chi-square for statistical analyses. We used nondirectional (two-tailed) tests in statistical analyses. Because of the large sample size, negligible relationships could become statistically significant but practically unimportant.^14,15 For the purpose of determining the “practical” significance of the findings, we used the operational definition of the effect-size estimates suggested by Cohen.¹⁴ Thus, correlation coefficients smaller than 0.10 were considered “practically” unimportant. Statistical analyses were performed separately for full-time clinicians and academic clinicians. Statistical Analysis System (SAS, version 9.1) was used for statistical analyses.

Results

Respondents

Of 5,553 living graduates, 5,349 had deliverable mailing addresses. Research participants were the 3,195 physicians from across the United States who responded to the survey. This represented 60% of those with deliverable mailing addresses and 58% of all living graduates in the selected time period. The respondents’ ages (as of 2006) ranged from 29 to 66 years, with a mean and median of 46 and a standard deviation of 7.3. A total of 826 women (26%) responded. Out of 200 physicians who received a second copy of the survey for the purpose of test-retest reliability study, 132 (66%) completed the second survey.

The respondents and nonrespondents were compared on variables retrieved from the Jefferson Longitudinal Study of Medical Education.¹¹ Differences observed between respondents and nonrespondents on class rank, gender, ethnicity, and specialty variables were practically unimportant as suggested by their negligible effect sizes.

Determining full-time clinicians and academic clinicians

On the basis of the respondents’ reports of time spent in patient care, teaching, research, and administration, we identified full-time clinicians as physicians who spent at least 28 hours per week in patient care, without any involvement in teaching and research (n = 1,127; 35%). Another group of physicians who reported spending at least eight hours per week in patient care and other time spent on teaching or research were identified as academic clinicians (n = 1,612; 51%). The rest (n = 456; 14%), who were primarily involved in administration or research, were excluded from analysis.

The underlying factors of the JeffSPLL

The underlying construct of the JeffSPLL was examined for each group of physicians using factor analysis (Table 1). Three factors emerged in each group with eigenvalues greater than one,¹⁶ which accounted for 52% of the variance among full-time clinicians and 54% of the variance among academic clinicians. The three-factor solution was also supported by the scree plot test.¹⁷

Table 1: Summary Results of Factor Analysis of the Revised Jefferson Scale of Physician Lifelong Learning for Full-Time Clinicians and Academic Clinicians, Jefferson Medical College, 1975–2000

The factor structure was similar in both groups. Six items (1–6 in Table 1) had the largest coefficients on factor 1 in both groups. Based on the magnitude of the eigenvalue and the content of items with largest coefficients, factor 1 was considered to be a prominent factor involving “learning beliefs and motivation.”

Three items (7–9 in Table 1) had the largest coefficients on factor 2, which was regarded as a construct involving “attention to learning opportunities.” Finally, two items (10–11 in Table 1) had the largest coefficients on factor 3, which was entitled “technical skills in seeking information.” Three remaining items (12–14 in Table 1) were bifactorial with relatively large coefficients on factors 1 and 2 in both groups.

The three factors of the JeffSPLL seem to correspond, respectively, to the key notions of “learning needs and motivation,” “self-initiated activities,” and “information-seeking skills” that we underscored in the definition of lifelong learning in medicine. Thus, the underlying construct of the JeffSPLL is consistent with the definition of the concept it purports to measure. Furthermore, the fact that these mathematically derived factors are also conceptually relevant to the elements of lifelong learning described by others^7,8,18–22 provides support for the construct validity of the JeffSPLL. In her doctoral dissertation research, Brahmi asked students at Indiana University School of Medicine about their perceptions of features of lifelong learning (Brahmi FA. Medical students’ perceptions of lifelong learning at Indiana University of School of Medicine. Doctoral dissertation completed at the School of Library and Information Science, Indiana University, December 2007). Brahmi concluded that content analysis of students’ responses fully supported the underlying construct of physician lifelong learning reported in our previous studies^4,5 and also confirmed in our study.

Descriptive statistics and reliability coefficients

An examination of responses to each item of the JeffSPLL indicated that the respondents chose the full range of responses. The item mean scores ranged from a low of 2.5 (item 10 in Table 1) to a high of 3.7 (item 4 in Table 1) for full-time clinicians, and from 2.9 to 3.8 for the same items for academic clinicians.

We also examined the item-total score correlations (e.g., correlations between the scores of each item with those of the remaining items in the scale). These were all positive and of practical significance, ranging from a low of 0.45 (item 11 in Table 1) to 0.70 (item 8 in Table 1), with a median correlation of 0.61 for the full-time clinicians. These correlations for the academic clinicians ranged from 0.48 (item 11 in Table 1) to 0.70 (item 2 in Table 1), with a median of 0.62.

Descriptive statistics and reliability coefficients of the JeffSPLL are reported in Table 2.

Table 2: Descriptive Statistics and Reliability Coefficients of the Revised Jefferson Scale of Physician Lifelong Learning for Full-Time Clinicians and Academic Clinicians, Jefferson Medical College, 1975–2000

The full-time clinicians obtained a lower mean score (M = 44.5) than did academic clinicians (M = 47.1, t_(2,752) = 11.2, P < .01, effect size = 0.37), indicating that academic clinicians, as a group, expressed a stronger orientation toward lifelong learning. The coefficient alpha reliability was equally large for both groups (0.85 and 0.86, respectively), and test-retest reliability coefficients (approximately four months between testing) were at an acceptable range for psychological testing (0.72 and 0.77, respectively). The magnitudes of the reliability coefficients indicate that the scores of the JeffSPLL were not only internally consistent (coefficient alpha) but also relatively stable overtime (test-retest).

Indicators of learning motivation

Scores on the JeffSPLL were correlated with responses to 13 indicators of learning motivation^19–22 (Table 3). Measurement properties of these indicators (e.g., construct validity) have been reported elsewhere.^4,5 All of the correlations were statistically significant for both groups (P < .01). The largest correlations in both groups were obtained for “Regardless of my busy professional schedule, I always feel the motivation to learn about new advances in medicine” (r = 0.57 and 0.53, respectively). The correlations for three questions about interest in research (items 1, 2, and 3 in Table 3) were expectedly higher for the academic clinicians than for full-time clinicians, because of the lack of research activities by full-time clinicians at the time of this study.

Table 3: Summary Results of Correlational Analyses of Scores of the Revised Jefferson Scale of Physician Lifelong Learning, and Indicators of Learning Motivation, for Full-Time Clinicians and Academic Clinicians, Jefferson Medical College, 1975–2000

Negative correlations were obtained for responses to two questions (12 and 13 in Table 3). These questions address extrinsic (as opposed to intrinsic) learning motivation,^4,5 indicating that learning for the sake of certification requirement, for example, is not positively related to having an orientation toward lifelong learning for the sake of learning. The standardized regression coefficients (β) showed that some of the 13 indicators of learning motivation had a unique contribution to the prediction of the JeffSPLL scores in the two groups of physicians. The resulting multivariate correlations of 0.70 and 0.73 for full-time and academic clinicians indicate a substantial overlap between scores of the JeffSPLL and the degree of respondents’ agreement with the indicators of learning motivation (an overlap of 49% and 53%, respectively, R²).

Medical school class rank

We compared the scores on the JeffSPLL for physicians classified in the top 25%, middle 50%, and bottom 25% class rank (Table 4). The mean scores on the JeffSPLL increased with increasing levels of class rank. The Duncan post hoc mean comparison test showed that the mean score for physicians in the top 25% was significantly higher than the mean score for the bottom 25% in both groups of physicians. For the combined group, the top 25% scored significantly higher on the JeffSPLL than did the middle 50%, who in turn scored significantly higher than did the bottom 25% (P < .01). Although the effect sizes of differences were marginal (0.09, 0.07, and 0.10 in full-time clinicians, academic clinicians, and the combined groups), these findings are consistent with a report of positive correlations between scores of a self-directed learning measure⁸ and performance in medical school.²³

Table 4: Scores of the Revised Jefferson Scale of Physician Lifelong Learning by Top 25%, Middle 50%, and Bottom 25% of Medical School Rank, Full-Time Clinicians and Academic Clinicians, Jefferson Medical College, 1975–2000

Indicators of professional accomplishments

Results of correlational analyses of the JeffSPLL scores and 11 indicators of professional accomplishments are reported in Table 5. The obtained bivariate correlations were “practically” significant for six indicators of professional accomplishments in full-time clinicians and for all accomplishment indicators in academic clinicians. For full-time clinicians, the largest correlation was found between scores of the JeffSPLL and responses to a question (11, Table 5) about physicians’ presenting patient education or medical research findings in public media (r = 0.23). For academic clinicians, the largest correlation was obtained between scores of the JeffSPLL and responses to a question (9, Table 5) about physicians’ serving as a reviewer for a professional journal (r = 0.28) and also for responses to questions (2, 3, and 4, Table 5) about physicians’ publication and research activities (r = 0.27).

Table 5: Summary Results of Correlational Analyses of Scores on the Revised Jefferson Scale of Physician Lifelong Learning and Professional Accomplishments for Full-Time Clinicians and Academic Clinicians, Jefferson Medical College, 1975–2000

In the multivariate statistical model, presentation of patient education and medical research findings had the largest unique contribution to the scores of the JeffSPLL in full-time clinicians (β = 0.18). The largest standardized regression coefficients for academic clinicians were obtained for conducting research, receiving research or training grants, and serving as a reviewer of a professional journal (β = 0.09). For the combined groups, conducting research and appearing in public media to present patient education and medical research findings yielded the largest standardized regression coefficients (β = 0.11). The adjusted multiple Rs were 0.26, 0.36, and 0.37 for full-time clinicians, academic clinicians, and the combined groups, respectively (P < .01). These findings demonstrate that the unique contributions of the indicators of accomplishment were not similar in full-time clinicians and academic clinicians, which may reflect the learning priorities in the two groups.

Career satisfaction and commitment to lifelong learning

We examined the correlation between JeffSPLL scores and physician ratings of career satisfaction and the self-report of physicians’ commitment to lifelong learning. Significant correlations were obtained in both groups between scores of the JeffSPLL and ratings of career satisfaction (r = 0.23, P < .01). Also, correlations with larger magnitudes were observed between JeffSPLL scores and physicians’ self-ratings of commitment to lifelong learning (r = 0.57 and 0.58 for full-time clinicians and academic clinicians, respectively, P < .01).

The findings of a significant link between JeffSPLL scores and measures of career satisfaction and commitment to lifelong learning, as well as statistically and practically significant correlations with indicators of learning motivation and professional accomplishments, provide support for the instrument’s criterion-related validity.

Examining response accuracy

To examine the accuracy of responses, we compared physicians’ self-reported publications and inventions with relevant information recorded in electronic databases for selected groups of 44 respondents who reported more than five publications in peer review journals in the past five years, and 54 respondents who reported that they had developed at least three new medical/surgical procedures, instruments, drugs, or techniques. We asked our medical-record librarian (blind to respondents’ self-report information) to search relevant databases for information about the publications and patents of the aforementioned selected physicians.

A high concordance rate was observed between the respondents’ own reports of the number of publications and information extracted from electronic databases. We obtained a correlation of 0.85 (P < .01) for the self-report number of publications and publication record in electronic databases. Also, the association between self-report and electronic record of registered patents was statistically significant (χ₍₁₎ = 7.3, P < .01).

Additional analyses

In additional analyses we examined the relationships between scores of the JeffSPLL and age (r = 0.04) and years after graduation (determined by subtracting 2006 from the graduation year) (r = 0.03). None were statistically significant. Also, no significant difference was observed on the scores of the JeffSPLL between men (M = 46.0) and women (M = 45.8) or among ethnic minority groups (e.g., 45 African American, M = 46.7; 44 Hispanic-Mexican, M = 46.2; 182 Asian-Oriental, M = 46.5) compared with the rest of the sample (M = 46.2).

Physicians who also had combined MD-PhD degrees (n = 47) obtained a significantly higher JeffSPLL mean score (M = 48.4) than the rest of the sample (M = 46.1) (F_{(1, 2,726)} = 7.7, P < .01, effect size = 0.40). The pattern of findings was similar in both full-time and academic clinicians.

Discussion

During this era of exponential growth of medical knowledge and rapid development of biomedical advances, it is important and timely to empirically study physicians’ lifelong learning, its development, its predictors, and its outcomes. The instrument evaluated in this study meets the key standards for test development, test content, validity, and reliability of educational and psychological testing.²⁴ The fact that medical school class rank was found to yield a statistically significant link with an orientation toward lifelong learning strengthens the belief that past performance is a good indicator of future achievement.²⁵

This is our third in a series of studies on physicians’ lifelong learning. The first was a study of 160 physicians in which step-by-step development of the original version of the JSPLL was described.⁴ The second study, with 444 physicians, was conducted to examine the psychometrics of the original version.⁵ In addition to a nationwide sample, the present study is unique in making a distinction between clinicians who are exclusively involved in patient care and those with additional involvement in research and teaching. The JeffSPLL that was created for this study is not only equally applicable to both groups of physicians, but the strong psychometric evidence supports its usefulness in either group.

Although the fact that the sample of physicians graduated from one medical school may be a limitation of this study, other factors strengthen the results. First, the study respondents had completed their residency training in 391 institutions throughout the United States and were practicing medicine in 49 states in 2006. In addition, Jefferson Medical College, established in 1824, is similar to other large private medical schools in the country with regard to its four-year curriculum, composition of student body, attrition rate, and students’ career choices. For example, the gender composition of the study sample (26% women) and specialty distribution (33% in primary care specialties of general internal medicine, family medicine, and general pediatrics, 5% in obstetrics–gynecology, 4% each in anesthesiology, psychiatry, and general surgery, and 2% in pathology) were similar to national data for U.S. physicians.²⁶ Also, the two types of career paths that were studied are the career paths of the majority of active physicians in this country. These factors strengthen the external validity (generalization) of the findings and mitigate the single-institution limitation.

Second, as with any self-report instrument, concern about the accuracy of responses and the effect of the “social desirability response bias” on results can be raised. Our findings of significant associations between self-reported numbers of publication and inventions and relevant information from electronic databases suggest that respondents were truthful; thus, the internal validity of the results was unlikely to be substantially distorted by social desirability responses.

Another concern is that individuals’ self-reports of orientation toward lifelong learning may not necessarily reflect their actual lifelong learning behavior. The fact that we found significant correlations between scores on orientation toward lifelong learning and behavioral manifestations of lifelong learning (e.g., indicators of professional accomplishment) supports a strong link between orientation and its behavioral manifestation. It is important to note that measures of professional accomplishments are important behavioral manifestations of physicians’ lifelong learning; however, the ultimate consequences of lifelong learning in medicine can be demonstrated by optimal patient care and positive clinical outcomes. That will set an agenda for future research.

Another limitation is that the JeffSPLL reported in this study may be inappropriate for administration to medical students in evaluating medical education curriculum outcomes. In response to a need for a psychometrically sound instrument to measure orientation toward lifelong learning in medical students, a group of medical education researchers at the Virginia Commonwealth University School of Medicine recently began collaborating with our team to develop a new version of the JeffSPLL for medical students. Psychometric results will be reported in due course.

In conclusion, our study provides evidence in support of the validity and reliability of the instrument used, and, more importantly, it documents that the surveyed physicians have a commitment to lifelong learning demonstrated by their motivation and professional accomplishments.^4,5 Medical schools and residency programs should monitor the quality of their graduates using a variety of instruments in a longitudinal study design, including orientation toward lifelong learning, to ensure that their educational goals have been achieved and to provide feedback to their faculty and graduates. The instrument evaluated in this study has practical value to monitor educational programs and to empirically study the predictors and outcomes of physicians’ lifelong learning. The JeffSPLL can also be used in the assessment of specific medical education programs (such as the problem-based learning curriculum) to validate the claim that such programs could lead to a stronger orientation toward lifelong learning. Also, physicians with different demographic characteristics in various specialties, types of practice, and different settings can be compared on their orientation toward lifelong learning through the use of the JeffSPLL.

Acknowledgments

Thomas J. Nasca, MD, played a significant role in the initial development of the Jefferson Scale of Physician Lifelong Learning when he was the dean of Jefferson Medical College. He is now chief executive officer at the Accreditation Council for Graduate Medical Education. The authors also would like to thank Dorissa Bolinski for her editorial assistance.

This study was funded in part by an invitational grant from the National Board of Medical Examiners (NBME) Edward J. Stemmler, MD Medical Education Research Fund.

Disclaimer

This study, its findings, and interpretations of the outcomes do not necessarily reflect NBME policy, and NBME support provides no official endorsement.