There is a shortage in the number of pediatric neurologists in Canada1–4 and in the United States.5,6 Recent published articles1–3 have demonstrated an increasing demand for Canadian pediatric neurological services, without an equal increase in qualified personnel to fill this demand. This increasing demand is attributable, at least in part, to a change in the role of pediatric neurologists. Pediatric neurologists now play a key role in family counseling, planning and implementation of treatment, and longitudinal follow-up, in contrast to a previous focus that was primarily on diagnosis, with the pediatrician providing the ongoing care.1,7
The most common diagnoses of neurological patients seen by general pediatricians are attention-deficit/hyperactivity disorder (ADHD), developmental delay, headaches, seizures, and epilepsy. The most common diagnoses of patients seen in neurological consultations are epilepsy, nonepileptic paroxysmal disorders, disorders of development and behavior (including mental retardation, learning disabilities, and psychiatric disorders), cerebral palsy, head injuries, peripheral nervous system and cranial nerve disorders, and headaches.2,8 There is clearly some overlap between the patient groups. Although support from an expert in pediatric neurology is essential to maintain standards of modern-day pediatric practice, management of more routine cases by primary care or consultant general pediatricians could shorten the pediatric neurologists' wait lists so that they may see patients with more complex diagnoses.
Pediatricians who rate their neurological assessment and management skills as inadequate are more likely to refer their neurological patients to pediatric neurologists.9 Hence, increasing pediatricians' abilities, as well as their perception of their abilities, in managing the more common aspects of pediatric neurology could significantly alter referral patterns for pediatric neurology consultations.10 Traditionally, Canadian pediatric residency programs have focused training predominantly on inpatient acute and complex chronic medical concerns. However, in recent years, residency educators in Canada and the United States have recognized the need for further training in management of behavioral, developmental, and general pediatric neurological issues.11–13
The University of British Columbia (UBC) pediatric residency program in Vancouver, British Columbia, is a four-year program, as are other Canadian pediatric residency programs. The UBC residency program requires the completion of two months of tertiary care neurology. In addition, residents encounter patients with neurological conditions during the required three months of community practice and six weeks of combined psychiatry and tertiary care developmental assessments. Once a doctor has begun to practice, the spectrum of neurological disorders that practitioners encounter clinically may be determined somewhat by whether a graduate is a subspecialist or a general pediatrician and by whether he or she practices in a rural community or in a city. The Canadian four-year program contrasts with the three-year program in the United States, during which residents are required to include at least one month of behavioral–developmental pediatrics, with an additional month that may be in pediatric neurology.
To our knowledge, there are no recent studies assessing the abilities of Canadian pediatric residents' competency in neurological and neurodevelopmental skills. The purpose of this study was to determine which areas of pediatric neurology should be given greater emphasis during pediatric residency training to achieve more appropriate referral for consultation. We surveyed graduates of the UBC pediatric residency training program to determine how they perceived their level of competence in diagnosis and management of common neurological issues, using self-assessment scores. The neurology and developmental pediatrics components of the UBC pediatric residency curriculum was consistent over the years the survey respondents were training.
The survey questions were developed iteratively by the residents, residency program director, and the neurology service to ensure relevance of content, unambiguous wording, and appropriateness to training opportunities. The questionnaire was based on a previous survey of our program in 1998 that was used to enhance the relevance of education for residents in other aspects of pediatric practice.11 We received ethical approval from the UBC behavioral research ethics board, certificate number B02-0364. There was no funding provided.
To optimize cooperation in survey completion, we contacted graduates by both telephone and e-mail and asked them to state their preferred method of receiving the survey (hard copy by mail, or electronic by e-mail). Surveys were sent between October 2002 and September 2004 to all 46 graduates from a seven-year period (1998–2004) of the UBC pediatric residency training program. However, we excluded graduates who had additional subspecialty training in pediatric neurology or developmental pediatrics (because these individuals would have received additional training in the areas to be assessed). We used the Dillman survey method to maximize return rates, with questionnaires numbered to allow repeat mail out to nonresponders.14 No other identifying information was displayed on the forms. Accompanying each questionnaire was a cover letter that included an outline of the purpose of the project, assurance of confidentiality, instructions not to put a name on the form, and a statement that completion of the questionnaire would constitute consent. Respondents were asked to indicate their year of graduation from the UBC pediatrics residency program, whether they were in general or subspecialty practice, and how many years they had practiced to date.
The survey consisted of a total of 29 questions. The first 26 questions used a six-point rating scale for respondents to rate their ability to manage specific neurological symptoms and conditions, to perform procedures, and to interpret a variety of neurological tests (1=inadequate,2=marginal in some areas, 3=adequate, 4=comprehensive, 5=perfect, 6=excessive). The category “excessive” was intended to allow respondents to indicate that their training in a particular area had been overemphasized compared with other aspects of their pediatric neurology training.
In addition, we asked three separate open-ended questions: which areas of neurology needed more emphasis in residency, which areas needed less emphasis in residency, and which areas would be most appropriate for continuing medical education (CME) sessions. Because there was only sufficient space for a short answer to each of these three open-ended questions, each respondent's answer was classified in only one category for each of the three questions. The answers to these questions were remarkably similar across respondents, greatly facilitating categorization. When respondents provided an extra comment in addition to the response that was categorized, we have included that comment in the results section.
We performed descriptive statistical analyses to identify areas of deficit as well as areas of overemphasis in training. The rating scale in this questionnaire uses numbers to indicate qualitative statements. These numbers, therefore, do not provide a continuous data set with a clear linear relationship but, rather, an ordinal rating scale. Thus, we chose descriptive, not inferential, analyses. The use of means allows for a comparison of the responses from variable to variable, but it has no numeric interpretation because these are qualitative data.
From 46 surveys sent out, 39 were returned (overall response rate, 85%). Twenty-eight subjects requested the survey by post and 18 by e-mail. Return rates for each method were comparable: 24/28 (86%) for the mailed surveys and 15/18 (83%) for the e-mailed surveys.
Sixty-seven percent (26/39) of respondents were in general pediatric practice, 15% (6/39) were in pediatric emergency medicine, and 18% (7/39) were in other pediatric subspecialty practice (Table 1). Seven of the respondents in general pediatric practice had completed part or all of a two-year subspecialty training program, but none of this subspecialty training involved pediatric neurology or developmental pediatrics. Seven (35%) of the 20 respondents who had undertaken subspecialty training (including pediatric emergency medicine), which was either partially or fully completed, eventually practiced as general pediatricians.
The only information available about the seven nonresponders was the date of graduation. They had a mean of 2.71±1.21 years since graduation, similar to our general pediatric group, and the graduation dates were distributed across the seven years.
Overall, the highest self-ratings for competency (Table 2) were in management of febrile seizures (mean, 4.40 ± 0.94; range, 3–6), performance of lumbar punctures (mean, 4.38 ± 0.78; range, 3–6), and management of status epilepticus (mean, 4.19 ± 0.72; range, 3–5). The lowest self-ratings for competency were for electroencephalogram (EEG) request and interpretation (mean, 2.10 ± 1.23; range, 1–6), head computed tomography (CT) scan request and interpretation (mean, 2.79 ± 1.08; range, 1–6), and management of tics/Tourette syndrome (mean, 2.90 ± 1.01; range, 2–5).
In addition to high self-rating competency scores for febrile seizures, lumbar punctures, and status epilepticus, the pediatric emergency physician group also rated their competence highly for management of syncope (4.67±0.52), elevated intracranial pressure (4.50±0.55), coma (4.33±0.82), minor head injury (4.33±0.41), and migraine (4.17±0.98). In contrast, the group of general pediatricians rated their competence highly for management of neonatal seizures (4.31±0.84) and full neurological examination (4.15±0.54).
Overall, in the open-ended questions, 28% (11/39) of respondents indicated that the areas of neurology that required less time and emphasis in residency education than they currently received were complex quaternary neurological cases (e.g., complex refractory seizure disorders) and those diagnoses requiring inpatient management in a tertiary care setting. This perception was most pronounced among general pediatricians (35% ) compared with pediatric emergency physicians (17% ) and pediatric subspecialists (14% ), as would be expected from the profile of patients seen by each group on a daily basis. In addition, 8% (3) of the overall group thought that full-day, tertiary-level developmental assessments of children with developmental delay should be less emphasized, and 18% (7) of the overall group thought that complex cases of other subspecialty services should be less emphasized. One respondent also stated that the interpretation of complex specialized neurological tests should receive less emphasis. Another respondent also mentioned the need for those working in small communities, rather than a tertiary care center, to have the necessary skills to provide acute management for neurological emergencies. Forty-six percent (18) either reported that there were no topics that required less emphasis or left this question blank.
The majority of respondents (74%  of the overall group) indicated in the open-ended questions that education and training in the diagnosis and management of common general pediatric neurobehavioral problems merited greater emphasis during residency training. These problems included office-based behavioral and developmental assessments; for example, assessment of developmental delay, pervasive developmental disorders, ADHD, learning disabilities, and school failure. Individual comments emphasized the need to learn how to perform an efficient assessment of these disorders in a general pediatric office setting as opposed to the process required in a tertiary care context. The need for greater focus on general neurobehavioral issues, as seen in an office-based setting during residency, was widely perceived by all subgroups, with 77% (20) of general pediatricians, 67% (4) of pediatric emergency physicians, and 71% (5) of subspecialists stating that such assessments required greater emphasis. Other topics cited as requiring more emphasis in residency include therapeutic use of antiepileptic drugs (8%  of the overall group), the neurological exam (8%  of the overall group), management of the child presenting with a seizure disorder for the first time (5% [two] of the overall group), and evidence-based medicine (EBM) (3%  of the overall group). Overall, 3%  left this question blank.
The perceived deficiencies in training, evident in the responses to the open-ended questions from the general pediatrician group, are also evident in the overall scores of the rating scale items; in particular, the ability to perform developmental assessments (2.90 ± 0.96), manage learning disabilities (3.12±1.11), investigate pervasive developmental disorders (3.13±0.98), and manage tics and Tourette syndrome (3.15±0.97).
Respondents indicated that their CME needs for neurology knowledge and skills were in the same topic areas as the deficiencies they had identified in their residency training. The major themes for CME were developmental delay, ADHD, and pervasive developmental disorders.
In this study, we examined how well a four-year Canadian pediatric residency program prepared graduates tomanage common neurological and neurodevelopmental problems once they were in practice. Important deficiencies in knowledge and ability were identified. It is the responsibility of every pediatric training program to provide the education and clinical learning opportunities necessary to prepare trainees for their future career responsibilities. It is important to get feedback from graduates to determine whether they have been well prepared to treat patients in practice. To do this, educators must evaluate how adequately their trainees are able to function after they have begun practicing, assess graduates' perceptions of the strengths and weaknesses of the education they have received, and determine which areas of the training program require greater or lesser emphasis. With this evidence, constructive changes can then be made.
Respondents' answers to the questions self-assessing competency in neurological skills (Table 2) demonstrate a wide variability in perceived competence, with the variability evident between different skills within the same group and for the same skills across different groups.
All groups scored highly in self-assessment of febrile seizure management, lumbar puncture, and status epilepticus management, probably because these skills are generally mastered during the early years of residency training. It is quite reasonable that head CT scan and EEG request and interpretation are rated at the lowest end of the perceived competency self-rating scale by all groups because these tests are frequently ordered and interpreted with the assistance of neurologists and/or radiologists, and pediatricians and nonneurology subspecialists would almost never be expected to fully interpret these studies independently. Hence, a lower level of actual and perceived competence is acceptable in this area, which may be why request and interpretation of both head CT and EEG were not listed by any respondents as requiring more emphasis in the pediatric residency training program.
The pediatric emergency physician group rated their ability to manage neurological emergencies such as syncope, coma, minor head injury, and elevated intracranial pressure highly, as would be expected from the patients seen in their practice. This is in addition to their high self-ratings of competency with skills that were also rated highly by the overall group (febrile seizure management, lumbar puncture, and status epilepticus management).
The most noteworthy findings of this study are the responses to the open-ended questions. These responses indicate that the majority (74% overall) of graduates from our pediatric residency program recognize a need, once they are in practice, for more training in office-based assessments of behavioral, developmental, and general neurological complaints and conditions.
Although the objective of pediatric residency training is to prepare trainees for their future career in effectively treating patients, the majority of pediatric residency training occurs at tertiary care centers. The reasons for this are primarily logistic and are related to the availability of faculty and education facilities and the intent to provide access to the greatest number and range of pediatric patients. However, as a result, most training occurs in the context of inpatient acute care. Although inpatient acute care is an essential aspect of pediatric training, once they have begun practicing, many general consulting pediatricians spend a large proportion of their time doing office-based practice, particularly developmental and behavioral assessments.
A previous study of our program in 199811 indicated that 37% of trainees who undertook pediatric subspecialty training before 1989 and from 1989 to 1996 ultimately entered practice as general pediatricians. Our data indicate that 35% of respondents to our survey followed this same career path. For this reason, education in an office-based setting needs to have a more central role in pediatric training, even for those residents intending to follow residency with subspecialty training. The previous Canadian survey of residents within the UBC program in 1998,11 a U.S. study in 1993,12 and a U.S. survey of program directors in 200013 indicated a relative lack of education in behavioral and developmental medicine at that time. This is consistent with our present findings, suggesting that there is an ongoing need to improve education in these areas. Our survey responses indicate that this need should be addressed at the community level, ideally in an office setting, rather than in a tertiary care setting. This is in keeping with the previously recognized need to increase the amount of office-based education included in pediatric residency programs in a 1993 U.S. study.9
The educational challenges that many pediatric trainees currently experience with the tertiary care developmental assessment training model are that the patients seen are frequently complex and require full-day assessments by a multidisciplinary team working at a developmental center. Such teams combine the expertise of physiotherapists, occupational therapists, psychologists, social workers, and developmental pediatricians. Although it is useful to see the team approach, understand the benefits of such assessments, and know that these assessments are available, trainees are rarely able to be involved in them in a meaningful way. Moreover, such teams are unlikely to develop a trainee's expertise in the type of one-hour, single-handed assessment of developmental and behavioral problems required in the office setting. Such skills may be best learned in an ambulatory clinic or office with a community pediatrician. This could be accomplished either by adding further community office rotations (as mandatory rotations or electives) or by asking several community pediatric offices to schedule developmental assessments on a particular day of the week; such assessments could be incorporated into a developmental pediatrics or neurology rotation. Previous surveys of Canadian pediatric residents in other fields of pediatrics indicated that the residents' sense of competency improved with the number of cases they were exposed to,15 and a survey of U.S. pediatric residents indicated that measured competency also increased with increased exposure to relevant cases.16 Moreover, evaluation of other skills in Canadian pediatric residents showed that residents who had little opportunity to practice a skill felt uncomfortable with it.17
A pediatrician, like other lifelong learners, is expected to continue to acquire relevant knowledge as her or his career progresses. One respondent commented that only after beginning practicing did the knowledge of developmental pediatrics become more refined, and that much relevant learning occurs after beginning to practice (e.g., through CME). However, it is inappropriate for skills as central to office-based practice as a developmental assessment to be left to be acquired in this way.
Respondents did identify their CME priority topics in neurology to be the areas perceived as deficient in residency training. Although it is encouraging that this need is recognized, a prior study has shown that there is little difference in self-assessment scores between pediatricians who have practiced for more than 10 years and those who have practiced for 10 years or less, suggesting that, in fact, little improvement in these skills occurs after a doctor has begun practicing.9 However, it is possible that the standards that individuals are applying in comparison with their own skills rise over time with increasing familiarity with the subject matter, but this standard is difficult to assess. Possibly, the pediatricians involved in teaching the office-based developmental assessments could be required to have CME in these assessments, especially if they have been out of the academic setting for a substantial amount of time.
In contrast to our findings, it might also be argued that the time spent during residency training at a tertiary care center is best spent taking advantage of the unique opportunities to learn from those most highly qualified in particular areas of pediatrics. Furthermore, the opportunity to see rare, complex cases similar to those which one may be called on to manage in the community could be seen as a valuable experience that is unlikely to be replicated after a doctor has begun practicing. Also, the privilege of being involved with full-day developmental assessments with developmental pediatricians during residency will allow trainees to thoroughly learn the important components of these assessments from those most familiar and experienced with them and will allow trainees to draw on this knowledge to later formulate their own office-based assessments. It could also be suggested that the skills for office-based neurobehavioral assessments can indeed be further developed after doctors have begun practicing, whereas it is much more difficult to find opportunities for hands-on bedside training with subspecialists after beginning to practice; for example, assessment of a pediatric patient with a stroke in the emergency department with a pediatric neurologist. Although graduates expressed concern about the emphasis on quaternary-level patients during training, exposure to these patients should not be eliminated from the training program, because general pediatricians often manage aspects of care such as feeding and quality of life for patients whose refractory seizures, for example, are managed by pediatric neurologists. Hence, the opportunity to work with subspecialists with expert knowledge should continue to be highly valued in training programs, because these individuals frequently have vast experience with the particular patients in their area of subspecialization and, thus, have finely tuned their approach to these patients. Because these valuable learning opportunities will be less easily repeated after a doctor has begun practicing, it may be concluded that a proportion of residency training should still continue to focus on subspecialty areas. However, because common neurological problems are very frequently referred to pediatric neurologists,2,8,9 who are in short supply relative to the demand, in Canada1–4 and the United States,5,6 the exposures to subspecialty training opportunities must be balanced by more training in garden-variety neurobehavioral assessments in residency training and CME.
Limitations of this study are the small sample size and the inclusion of only one residency program. Because the UBC pediatrics program has between five and seven Canadian graduates per year, the sample size is limited, particularly when the groups are further divided by career path. To have a larger sample size, we would either have to survey residents who graduated before 1998, or survey graduates of other pediatrics programs. Either of these additions would confound the study because these graduates would have gone through a much more variable set of rotations during training, and a subanalysis of each group would be necessary in both cases. However, the restriction of the survey to only the UBC pediatric residency program limits the application of the conclusions of this study to other pediatric residency programs. In particular, our findings have limited applicability to programs with a different structure, such as the three-year programs in the United States, and further studies are needed to examine the specific needs of both U.S. and Canadian pediatric residents.
As the field of pediatrics becomes broader, it often seems that more responsibilities are being added to the curriculum, with the risk that traditional skills involving inpatient acute care, intensive care, neonatology, and other pediatric subspecialties are being eroded. The challenge of every pediatric program is to determine the best combination of training opportunities to stimulate effective learning in the areas most relevant to the spectrum of patients to be seen in trainees' future careers. Although this study suggests that additional training in this area would be beneficial, it is important that the improvements to neurology training for pediatric residents not occur at the expense of training in other areas that residents may also perceive as deficient in their training.
Conclusions and Future Directions
In this study, we have looked at how graduates of one program perceived their preparation to care for pediatric patients with neurological/neurobehavioral problems, and we have identified several areas of training that should be further emphasized in pediatric residency. The insights obtained from this study should serve to stimulate further studies with more detailed quantitative analysis to help in residency program curriculum design for future trainees. Remaining questions include how to best implement this training, which aspect (if any) of the current training it should replace, and whether this should be a mandatory or elective component of a pediatric residency program. Further studies could compare self-rating of neurobehavioral assessment skills between graduates of programs with greater and lesser emphasis in this area and could also evaluate the effectiveness of CME neurobehavioral training. In addition, we have not addressed the differing needs of residents planning to practice in rural versus urban settings, as the support of a nearby tertiary care center can significantly alter the range of clinical problems a general pediatrician is presented with and may also change the management of those problems. Generally, a more comprehensive look at neurology and neurodevelopmental training in a broader spectrum of pediatric residency programs is needed.
Guidance and statistical support were kindly provided by Faith Gagnon, Parveen Thind, Wendy Cannon, and Ruth Milner.