Physicians, especially obstetricians and pediatricians, are frequently placed in the unenviable position of informing parents that they have delivered a child with a disability, most commonly Down syndrome. Despite the fact that more than 1 in 733 live births are diagnosed each year with Down syndrome,1 physicians in residence as well as medical students rarely receive training related to communicating with parents who receive a prenatal or postnatal diagnosis of this chromosomal condition.2,3 In fact, a 2004 survey of 1,105 American College of Obstetrics and Gynecologists Fellows and Junior Fellows reflected their perceptions that their training had been deficient in regard to Down syndrome screening.4 It is as if new physicians are expected to understand immediately parents' reactions, as well as their own, to unplanned events such as Down syndrome. As a result, families have consistently expressed dissatisfaction with both the content and the manner in which information has been delivered to them.3–7 Not surprisingly, in one large-scale study, very few mothers rated the experience of receiving this news a positive one, and the majority of mothers reported a sense of fear, anxiety, or both upon receiving the diagnosis.6
Since 1975,8 researchers have repeatedly noted this critical need for accurate information and emotional support for families of newly diagnosed children with Down syndrome,2–11 but little progress toward this goal has occurred. Compared with the 1950s when most parents of children with Down syndrome viewed physicians' advice in the neonatal period as the most important factor in choosing their child's place of rearing, only 3% thought that physicians' opinions were critical by 1980.8 Parents question the reliability and usefulness of physicians' advice in this arena. In one study, only one half of the families felt well supported, and only 30% of the families of children with Down syndrome felt that the information they received was sufficient.8 Many reported that physicians often relied on textbook information and stressed medical conditions and deficits rather than the ways in which children with Down syndrome were similar to other children. In fact, parents from Sweden reported that physicians often provided diagnostic information to them in an approach that seemed like a “checklist” of deficits rather than a caring relationship.8,9 Offering alternatives, families stressed the importance of both how they were informed and the quality of the information they received.8
In the largest-ever survey to date involving 1,250 mothers of children with Down syndrome, Skotko6 found that very few health care professionals gave up-to-date printed information or provided telephone numbers for parent support groups. When physicians did emphasize positive aspects of Down syndrome, their efforts resulted in increased optimism on the mothers' part. Although mothers with infants born most recently were more apt to report that their physicians had talked about or emphasized the positive aspects of Down syndrome, improvements in physician behaviors have been slow and incomplete.6,12
All of these studies have documented the need for physician training in knowledge, attitudes, and communication skills in regard to Down syndrome. How to provide accurate, balanced, and empathic information on Down syndrome may not be an intuitive area of expertise for most physicians. The need for such training is clearly aligned with the communication competencies of the Association of American Medical Colleges (1999), including such critical skills as giving the patient (family) time and opportunity to talk, discussing lifestyle issues and health risks, exploring psychosocial and emotional factors, checking and clarifying information, encouraging the patient (family) to ask questions and adapting to the family's level of understanding, being direct and avoiding jargon, maintaining a respectful tone, asking open-ended questions, using silence, reflecting families' comments and acknowledging their reactions, asking if the family has anything else to discuss, and reviewing next steps.13 This need is also aligned with the Core Competencies of Patient Care (“Counsel and educate patients and their families”) and Interpersonal and Communication Skills (“Residents must be able to demonstrate interpersonal and communication skills that assist in effective information exchange and be able to team with patients, patients' families, and professional associates”).14 In addition, Educational Objectives for Residency Programs in Obstetrics and Gynecology outline the need for residents to “describe the clinical significance of karyotype abnormalities such as trisomy 21” and to “counsel a patient about manifestations of common genetic disorders.”15 Further, learning objectives for the training of pediatric residents indicate that they must have experience in a “Normal/Term Newborn Nursery” which should include (among other experiences) “identification of common anomalies, birth defects and syndromes, including counseling the parents.”16 Just how this training should be delivered, however, is left unspecified.
In response to this need for culturally and emotionally sensitive information at strategic intervals after a diagnosis of Down syndrome, the National Centers for Disease Control, National Center on Birth Defects and Developmental Disabilities recently funded Brighter Tomorrows, which incorporates two projects related to physicians communicating with patients about Down syndrome diagnoses in utero and postnatally. These projects have been developed over a 2-year period. In this study, we report the effectiveness of the materials developed from one of these two projects, an interactive CD-ROM to teach physicians how to provide accurate and balanced information in a compassionate manner to families when the initial diagnosis of Down syndrome is made unexpectedly after the birth of the child. The purpose of this study was to assess obstetrics and gynecology residents and pediatrics residents' cognitive knowledge about Down syndrome as well as their comfort level in counseling parents with a newborn child with Down syndrome before and after intervention (resident viewing and interaction with the educational material). Further we desired to evaluate similarities and differences between the two specialties. Our null hypothesis was that after the development of this educational tool, there would be no difference in previewing and postviewing scores when first and second year residents were assessed about their cognitive knowledge of Down syndrome and comfort level in communicating this diagnosis to a family.
MATERIALS AND METHODS
This study was approved by the University of Kentucky Institutional Research Board (IRB 03–0711-P4S). It was conducted from November of 2003 through June of 2005. As noted above, funding was provided through National Center on Birth Defects and Developmental Disabilities in cooperation with the Association of University Centers on Disabilities.
We gathered together an interdisciplinary medical team, an expert advisory group of parents of children with Down syndrome, and faculty who were experts in the fields of special education and psychology to develop an interactive CD-ROM, an educational tool for physicians. The purpose of this CD-ROM was to 1) update information on survival and the outlook of Down syndrome, including potential education, social, and adult outcomes; 2) teach appropriate methods for counseling patients at the time immediately after the diagnosis; 3) increase knowledge about available community resources; 4) define necessary medical and educational referrals; and 5) educate about the availability of healthcare guidelines, in conformance with the AAMC competencies and RRC objectives outlined above.
In brief, we used two expert teams—a Core Medical Team and an Expert Parent Team. Six parents of children with Down syndrome were recruited from the local Down Syndrome Association; their children represented a range of ages, outcomes and functioning levels for children with Down syndrome. The parents' primary role was to describe their perspectives about how parents should be told of the diagnosis. Additionally, they helped develop the “virtual patient and family scripts, provided stories of how they were informed of their child's diagnosis and developed “life glimpses” of their child's current status, including his or her other achievements, contributions to the family, and participation in school and community environments, all of which were included in the module. The responses and perspectives were provided to and reviewed by the Core Medical Team. The Core Medical Team and Expert Parents Team worked in an iterative fashion to refine the teaching module.
The Core Medical Team consisted of a multidisciplinary–departmental group of faculty members from the University of Kentucky College of Medicine who were interested in and knowledgeable about Down syndrome. This team 1) formulated the key teaching objectives for residents in obstetrics and gynecology, pediatrics, and family practice; 2) determined the key elements of a script in delivering the diagnosis of Down syndrome to families who had not expected their child would have Down syndrome; 3) ensured the medical accuracy of the script and related resource documents for physicians; and 4) defined the critical elements of communication essential in supporting families.
Specifically, we developed an interactive, standardized “virtual patient and family” module which was consistent with fostering self-directed learning as suggested by Knowles.17 This module required learners to respond at “decision points” about how they could best communicate with the family in the context of delivering the diagnosis. Physicians were asked to respond to both multiple choice and open-ended (typewritten) response formats during the course of the patient–physician “interviews.” Open-ended responses were assessed by core concepts rather than written eloquence, and preferred answers were provided to physicians using critical components of an optimal answer. All responses were saved by the program, and were provided to the physician upon the completion of the module. The module also contained video and narrative portraits of individual children and young adults at different ages, written by their families; narrative examples (in story form) of how physicians had given the news and how families had received the diagnosis; and links to related topics and resources. The materials were recorded for use on a CD-ROM format.
The preliminary module was then critiqued in a series of focus groups with third- and fourth-year residents and community physicians in obstetrics and gynecology, pediatrics, and family practice, as well as parents of children with Down syndrome. Further revision ensued, which incorporated feedback, and the final version was developed.
Thirty first- and second-year residents specializing in either obstetrics and gynecology or pediatrics participated in this effectiveness study. Because credits were listed on the CD-ROM, these residents could have been aware that two of their faculty members had assisted with the construction of the materials. This group represented a response rate of 100% in our resident pool. Ten were residents in obstetrics and gynecology, and 20 were residents in pediatrics. The study was limited by the number of available obstetrics and gynecology residents in any given year.
The Core Medical Team and Expert Parent Group also assisted in the development of the evaluation tools for this module. These tools consisted of a situational inventory to assess physicians' perceived confidence and competence in assisting and supporting families, a cognitive inventory to assess physicians' knowledge of the most important aspects of Down syndrome, and a module satisfaction scale that asked viewers to respond to the ease of use of the CD-ROM format as well as accuracy and comprehensiveness of the information.
We used this situational inventory to assess residents' perceived comfort level in dealing with difficult questions or situations involving a postnatal diagnosis of Down syndrome. We developed our own scale for this measure, based upon two considerations: 1) what areas of need had been reported in the literature as reflected by a search in MEDLINE and WebSPIRS in June of 2004; and 2) the results of a focus group conducted with our own Parent Expert Team that identified key issues in how they were informed, the communication skills they most valued in physicians, and how they wish they had been informed. The resulting instrument was a 5-point Likert scale ranging from “not at all uncomfortable” (1) to “extremely uncomfortable” (5). Thus, we were able to compare before and after differences in residents' discomfort in potentially interacting with parents of children with Down syndrome. To determine an overall comfort score, we added the scores on the questions together for each participant. Sample items from the situational scale are provided in the box, “Sample Items From the Situational Inventory.”
The content or knowledge test had 15 multiple-choice items based upon the content of the module. Knowledge items clustered in the areas of 1) the genetics of Down syndrome, 2) diagnosing Down syndrome and communicating that diagnosis to families, 3) secondary conditions commonly associated with Down syndrome, and 4) current research in life outcomes for children and adults with Down syndrome.
The usability scale asked participants to rate: 1) the need for this program for medical students, residents, and practicing physicians; 2) general ease of use and navigational features; 3) the accuracy and comprehensiveness of the content of the program; 4) the value of interacting with the “virtual family” in delivering the diagnosis; and 5) the value of the additional elements of the module, including the parent and physician stories of receiving and giving the news the “life glimpses” of individual children and the community, national, and informational resources section.
We analyzed data using Statistical Package For Social Sciences 14.0 (SPSS, SPSS Inc., Chicago, IL) for Windows. Since we used two repeated measures instruments, we used a paired sample t test from SPSS to determine change scores on residents' knowledge and comfort. We also conducted a post hoc test of power, using nQuery Advisor v5 (Statistical Solutions, Saugus, MA). The observed power for both pretest compared with posttest and precomfort compared with postcomfort test is greater than 99%. We then computed the sample sizes needed to show that the mean obtained difference of 2.97 for pretest total compared with posttest total and 14.48 for precomfort total compared with postcomfort total are statistically significant at P<.01 and then P<.001 levels. These tests enabled us to estimate that a sample as small as 22 subjects would have allowed us to detect a difference in means of 2.97, assuming a standard deviation (SD) of 2.11. Similarly, for comparing precomfort and postcomfort totals, we would have needed only 19 subjects to have a 99% power to detect a difference in means of 14.48, assuming a standard deviation of 9.23. These analyses then allowed us to proceed with further tests of residents' performance on knowledge and comfort level after using the CD-ROM. Finally, since we wanted to compare obstetrics and gynecology residents and pediatrics residents and sample sizes were small, we used nonparametric tests for analyses on these groups, including the Wilcoxon and the Mann-Whitney tests with two independent samples.
A full description of the subjects can be found in Table 1. Thirty residents in our pool participated in this study, with all 30 completing both the knowledge pretest and posttest measures, and 29 of the 30 completing both the situational or comfort level measure. Initially, of these 30, 28 (93%) of the residents indicated that they felt at least some need to be exposed to this material. After completion, all indicated that this module would be useful for medical students, interns, and residents alike. Each of the measures is discussed in turn.
As noted above, the content test had 15 multiple-choice items. Pretest results ranged from 7 correct items to 13 correct items. Posttest results ranged from 10 to 15 correct items. Of the 30 individuals who completed the pretests and posttests, 26 improved (Table 2). The mean improvement was 2.97, with a SD of 2.11. When we conducted a paired samples Student t test on pre–post test differences, it was significant at the P<.001 level (df 29, SD 2.108, t −.705).
A total of 29 residents (10 obstetrics and gynecology and 19 pediatrics) completed both comfort scales (Table 3). Of a possible 100 for the greatest discomfort, scores ranged from 35 to 78 (with a mean of 52.27) on the presituational scale and from 24 to 55 (with a mean of 37.45) on the postsituational scale, suggesting a decrease in residents' discomfort with these situations. When we looked at individual improvement, we discovered that 28 of the 29 participants became more comfortable with these situations. The mean difference between the pretest and posttest scores for these 29 residents was 14.48, with a SD of 9.22. Using a paired sample Student t test, this difference was significant at the P<.001 level (df 28, SD 9.230, t 8.450). Average mean score for each item in the pretest was 2.60, indicating a moderate overall degree of discomfort; average mean score of all items in the posttest was 1.87 (a score of 1 indicates “Not at all uncomfortable”).
Considerable evidence documents the importance of engaging adult learners in interactive processes.16,17 When we designed this interactive CD-ROM, we wanted to assess improvement in physicians' knowledge and attitudes, but we also wanted to evaluate the effectiveness of this interactive method of delivery. As noted previously, we constructed a usability scale that asked physicians to evaluate the CD-ROM on the following dimensions: 1) need for this program, 2) navigation between sections, 3) accuracy of content, 4) ease of using interactive components, 5) helpfulness of “Life Glimpses” and stories of “Giving and Receiving the News”, 6) helpfulness of the “Resources Section”, 7) usability of multimedia components, and 8) perceived overall value for participants at each level of learning, that is, medical students, interns, second- and third-year residents, practicing physicians, and Continuing Medical Education units for practicing physicians.
For most questions on this scale, we used a 5-point Likert scale ranging from strongly agree (5) to strongly disagree (1). On all except two measures, the median response was 4, the score between “agree” and “strongly agree.” The only exceptions were 1) the residents' sense that the CD-ROM might not be quite as valuable for medical students in their early training (mean 3.5, SD 1.13) and 2) the helpfulness of the Life Glimpses section (mean 3.6, SD .93). Overall, then, the residents validated the effectiveness of this interactive method of educational delivery. In fact, the most frequent response to the question of the value of Brighter Tomorrows for physicians in residence was “5, Strongly Agree.” Selected data are listed in Table 4.
When we compared differences between the 15 second-year residents and the 14 first-year residents (one resident did not designate a year) using a Wilcoxon nonparametric test within SPSS, we found no significant differences in either knowledge or comfort level improvement. We did, however, find significant differences in two measures when we controlled for residents' specialty. Using a Wilcoxon test to assess the residents' felt need for this type of information and training, there was a significant difference between the two groups, with pediatrics residents voicing a significantly greater need for this information and training than did obstetrics–gynecology residents (exact significance with two-tailed test, P<.01). Using a Wilcoxon and a Mann-Whitney test to compare differences in comfort of each specialist group, residents in pediatrics showed greater improvement in comfort level than did residents in obstetrics–gynecology. As stated previously, the lower the score on the situations inventory, the less discomfort felt. Pediatric residents' scores changed from 4 points (least change) to 38 points with a mean change of 17.4 and SD of 8.25. Obstetrics–gynecology residents' change ranged from −1.00 to 27 points, with a mean change of 9.00 and SD of 8.82 (rank mean 17.71 and 9.85, respectively; z −2.37, P<.01). In the comparison of these two resident groups, then, we found a greater difference in the effect of the CD-ROM on the comfort level of pediatric residents compared with the obstetrician–gynecologists. Differences between pediatrics residents and obstetrics–gynecology residents on knowledge measures were, however, nonsignificant using a Mann-Whitney test (z −1.18, P<.25).
Overall, then, first and second year residents improved significantly on both content and comfort-level measures and recommended the use of this module for all levels of medical education. We also wanted to determine whether there were differences in usability experience between the pediatrics residents and the obstetrics and gynecology residents. We found few differences. These differences included the perceived need for this information for themselves, as previously noted. In addition, their perceptions of usefulness for practicing physicians and for Continuing Medical Education units were different. Since Interactive case 1 deals with informing the family after the baby is born, it seems predictable that pediatricians would expect practicing physicians to find it more useful than would specialists in obstetrics–gynecology practice.
Because of the size of the pediatrics residents' sample, we also analyzed the pediatricians alone and found similar results. Using a nonparametric test, the Wilcoxon, we found pediatrics residents' improvement from pretest to posttest for content to be significant at the P<.001 level (z −3.586, and for precomfort to postcomfort to be significant at the P<.001 level (z −3.825).
Researchers have consistently found that families of children with Down syndrome report both a lack of essential support and accurate information from their physicians at the initial point of diagnosis after birth.2–11 Parents have also noted the need for information that they can digest after this initial encounter. Written information can be especially helpful; unfortunately, a lack of written information is one of the needs mentioned as frequently unmet.6,10 Educational programs to support physicians and other health care providers to inform and support parents at the point of initial diagnosis should include written, balanced informational resources for physicians to use in addition to a focus on basic knowledge and communication competencies. Effective learning tools that can simulate the critical physician–family interview at the point of initial diagnosis and that subsequently result in both increased knowledge and comfort level of resident physicians concerning Down syndrome hold the promise of more positive change for future families. Further, such tools are clearly aligned with the intent of Accreditation Council for Graduate Medical Education and Resident Review Committees in terms of learning objectives for residents in obstetrics and gynecology as well as essential learning objectives in pediatrics. Indeed, our tool may be a practical approach in which to provide such training to resident physicians, within a realistic simulation that prepares residents for this situation before they actually encounter it. This tool allows residents to learn on their own, to test their new knowledge, and then to compare their attitudes and knowledge to what is being modeled by expert physicians in their training. All of these skills are important parts of comprehensive adult learning in our new information age.17,18 “See one, do one, teach one” may have limited value in an arena such as Down syndrome that previously has been misunderstood and misrepresented in practice. Training by CD-ROM can have a significant place in the education of residents as it relates to learning how to provide balanced and accurate information in a supportive fashion at the point of initial diagnosis.
The purpose of the present study was to assess obstetrics and gynecology residents' and pediatrics residents' cognitive knowledge about Down syndrome and comfort in counseling parents with a newborn child with Down syndrome before and after intervention (resident viewing and interaction with the educational material). We found highly significant changes (P<.001) in both knowledge and comfort levels for both obstetrics and gynecology residents and pediatrics residents, as a result of completing this learning activity. Although obstetrics and gynecology and pediatrics residents improved similarly on both the knowledge and comfort level measures, we did, however, find two significant differences in the responses of these groups overall. Pediatrics residents demonstrated that they felt a greater need for this type of information than did the obstetrics and gynecology specialists, with a mean of 4.14 and 3.2 respectively on a scale of 1 to 5 (P<.008). In that this module dealt specifically with a diagnosis after birth, this result is not surprising. At least one study has found that pediatricians are more typically involved in delivering a postnatal diagnosis than any other specialty or health care deliverer.8
Further, residents in pediatrics showed greater improvement in comfort level than obstetrics and gynecology residents with improvement means of 17.4 and 9.0 respectively (.017 significance level). Again, given that pediatric residents may see delivery of a postnatal diagnosis as more in their province of practice than do obstetrics and gynecology residents, the fact that participation in this simulated “virtual family” model resulted in a greater increase in perceived comfort level for pediatric residents is also not surprising.
We also wanted to examine the effectiveness of a multimedia interactive CD-ROM as a teaching tool for physicians in residence. Both groups of residents rated the module as easy to use, with clear navigation aids, and important to them overall. For sophisticated students such as these physicians, it seems that this type of approach to learning could be expanded to other topics in addition to treatment for families with children with Down syndrome.
The second part of this learning tool—a virtual family interview for a couple who receives an in utero diagnosis—has now been developed as a part of the project, and is currently being piloted with obstetrics and gynecology, pediatrics, and family practice residents. It will be of value to determine whether obstetrics and gynecology and family practice residents rate the need for the in utero module more highly, given the greater likelihood of their involvement in interviews and assessments for patients receiving an in utero diagnosis.
A key limitation to this study is that we had to rely on measures of knowledge and perceived comfort level for the participating residents, and not upon more direct measures of actual changes in physician-family interviews. However, direct measures of actual family-physician encounters at this critical juncture in families' lives are not clinically practical. Still, standardized, pre–post patient interviews could be used to validate whether these knowledge and attitudinal measures carry over into actual changes in communication and interview behaviors in at least simulated patient settings.
Second, we did not use a previously validated measure of comfort level in dealing with similar situations, nor could we find an already developed scale that would measure comfort level in the literature in the sense in which we wanted to study it. Rather, we developed scenarios based upon parental perspectives and issues noted in the literature and our expert family panel's descriptions of “real-life” physician encounters to construct the items in our scale.
Third, although this study yielded a response rate of 100%, it was conducted at just one university and hospital setting. Although we are confident that other residency training programs would obtain similar results (given the high level of statistical significance of our findings), we cannot state with certainty that our results would generalize to other residency training programs.
Teaching physicians to impart “difficult” news in a balanced, compassionate, and accurate fashion is not an easy task. Yet that skill is critically needed when physicians are called upon to inform a family that their newborn child has Down syndrome or another developmental disability. Although the present training tool was designed to teach physicians up-to-date and balanced information about Down syndrome, it is suggested that the communication skills required to impart that information empathically, with a guarded sense of optimism and hope, are ones that generalize broadly to the diagnosis of other disabilities, and to the provision of other types of unexpected, life-altering news. In addition, the interactive CD-ROM format of training delivery has proven to be effective in at least one university medical center setting. Future research is needed to determine the most effective educational tools for ensuring that residents are equipped with generalizable skills to communicate unexpected news to patients and families concerning the diagnosis of Down syndrome and other developmental disabilities, as well as other potentially life-altering situations for families.
1.Centers for Disease Control and Prevention (CDC). Improved national prevalence estimates for 18 selected major birth defects—United States, 1999–2001. MMWR Morb Mortal Wkly Rep 2006;54:1301–5. Available at www.cdc.gov/mmwr/preview/mmwrhtml/mm5451a2.htm
. Retrieved April 7, 2006.
2.Krahn GL, Hallum A, Kime C. Are there good ways to give “bad news”? Pediatrics 1993;91:578–82.
3.Klein SD. The challenge of communicating with parents. J Dev Behav Pediatr 1993;14:184–91.
4.Cleary-Goldman J, Morgan MA, Malone FD, Robinson JN, D'Alton ME, Schulkin J. Screening for Down syndrome: practice patterns and knowledge of obstetricians and gynecologists. Obstet Gynecol 2006;107:11–7.
5.Springer A, Steele MW. Effects of physicians' early parental counseling on rearing of Down syndrome children. Am J Ment Defic 1980;85:1–5.
6.Skotko B Mothers of children with Down syndrome reflect on their postnatal support. Pediatrics 2005;115:64–77.
7.Drotar D, Baskiewicz A, Irvin N, Kennell J, Klaus M. The adaptation of parents to the birth of an infant with a congenital malformation: a hypothetical model. Pediatrics 1975;56:710–7.
8.Hedov G, Wikblad K, Anneren G. First information and support provided to parents of children with Down syndrome in Sweden: clinical goals and parental experiences. Acta Paediatr 2002;91:1344–9.
9.Williams C, Alderson P, Farsides B. What constitutes “balanced” information in the practitioners' portrayals of Down's syndrome? Midwifery 2002;18:230–7.
10.Garwick AW, Patterson J, Bennett FC, Blum RW. Breaking the news: how families first learn about their child's chronic condition. Arch Pediatr Adolesc Med 1995;149:991–7.
11.Sharp MC, Strauss RP, Lorch SC. Communicating medical bad news: parents' experiences and preferences. J Pediatr 1992;121:539–46.
12.Skotko B, Bedia RC. Postnatal support for mothers of children with Down syndrome. Ment Retard 2005;43:196–213.
13.Association of American Medical Colleges. Report III: Contemporary issues in medicine: communication in medicine. Washington (DC): Association of American Medical Colleges; 1999.
14.The Accreditation Council for Graduate Medical Education. ACGME Outcome Project: general competencies. Available at: http://www.acgme.org/outcome/comp/compFull.asp
. Retrieved November 21, 2003.
15.Council on Resident Education in Obstetrics and Gynecology. Educational objectives: a core curriculum in obstetrics and gynecology. New York (NY): Professional Publishing Group, Ltd; 2005.
16.The Accreditation Council for Graduate Medical Education. Residency Review Committee for pediatrics core competencies. Available at http://www.acgme.org/acWebsite/RRC_320/320_genComp.pdf
. Retrieved February 10, 2006.
17.Knowles MS. Fostering competence in self-directed learning. In: Smith RM and associates, editors. Learning to learn across the life span. 1st ed. San Francisco (CA): Jossey Bass; 1990. p.123–36.
18.Gooler DD. Changing the way we live and learn in an information age. In: Smith RM and associates, editors. Learning to learn across the life span. 1st ed. San Francisco (CA): Jossey Bass; 1990. p. 307–27.