David Savitz and Roberta Ness have authored a useful critique1 of the National Children's Study. Having been involved with the study since its inception in 2000, I can affirm that the genuinely thorny issues raised by this critique have been grappled with by National Children's Study investigators and others for years. In my response, I will underline 4 points:
- The study's protocol and sampling plan did not evolve in a vacuum. Understanding the present situation requires an analysis of the forces that shaped the study's evolution.
- It is encouraging to see the authors express support for sampling from prenatal care settings.
- The recommendation to assemble “a small group of carefully chosen experts” to direct the study must avoid recapitulating the missteps that led to the present situation. This is best done by entrusting the study to a leadership team consisting of the study director, the Program Office, and the investigators now in the field.
- The study is now committed to an empirical approach for assessing sampling, recruitment, and protocol content. Though I have favored prenatal-care sampling since 2004, the ultimate decision of how best to sample and recruit must be based on empirical evidence.
THE POLITICAL WORLD OF THE NATIONAL CHILDREN'S STUDY
The National Children's Study was a Congressional mandate initiated during the Clinton presidency. The Children's Act of 2000 (PL 106-310) instructed the NIH and other federal agencies (CDC and EPA) to “. . . plan, develop, and implement a prospective cohort study, from birth to adulthood, to . . . incorporate behavioral, emotional, educational, and contextual consequences to enable a complete assessment of the physical, chemical, biologic, and psychosocial environmental influences on children's well-being . . . .” This very broad directive was not, however, accompanied by funding.
NIH cannot ignore Congressional mandates. The NICHD's director, Duane Alexander, worked tirelessly to organize planning for the study, expecting that, as momentum grew, Congress would not only recommend but also fund this critically needed study. The planning phase was supported by internal funds of the agencies involved, unassisted by either the President or Congress. The National Children's Study, never a Bush White House priority, was omitted from the federal budget in each of the 7 fiscal years from 2000 through 2006.
The 7 lean years in which the study languished without Congressional funding proved a mixed blessing. On the one hand, a great deal of time was available to enlist expertise from the many fields of human endeavor needed to address the all-encompassing agenda Congress had mandated for the National Children's Study. Committees were created, planning meetings held, and small pilot studies were undertaken. An assembly of more than 20 working groups enlisted hundreds of academics to examine study methods and measures of exposure and outcome, reflecting an admirably liberal and open spirit on the part of the federal scientists involved in planning. In a fiscally constrained environment, it was also an efficient way to plan, because this massive academic effort was entirely unpaid. But, as a ship lying in port too long becomes encrusted with barnacles, so the National Children's Study became weighted down with a myriad of measures and instruments.
In 2004, a stratified random sample of all 3141 US counties yielded 105 study locations in 43 states to represent the US. Each location will recruit 250 births a year for 4 years, yielding the sample of 100,000. Establishing this large national constituency took the National Children's Study into a new political realm. Congressional interest in the study is now widespread; it is unlikely that any location can be dropped from the National Children's Study without paying a substantial political price.
In fiscal 2007 and 2008, Congress and the President could not agree on a budget, and a continuing resolution was needed to fund federal operations. Continuing resolutions can include new legislation and are virtually veto-proof. This allowed Congressional supporters of the National Children's Study to insert funding for the study into the 2 resolutions. These allocations enabled the study's program office at NICHD to establish a large coordinating center and 7 Vanguard Centers to initiate recruitment. Requests for proposals were issued for other centers, and the teams responsible for conducting the study in most other study locations began to take shape.
With the Presidential election of 2008, the political map changed entirely. Close to $200 million was allocated to the National Children's Study in the White House budgets for both fiscal 2009 and 2010. White House Science Advisor Kei Koizumi has stated that the National Children's Study is a White House scientific priority (conversation at Michigan State University, 21 April 2010).
SAMPLING PLANS AND EXPERT OPINION
At the first consultative meeting on the National Children's Study, in 2000—attended by, among others, Jørn Olsen of the Danish National Birth Cohort—there was agreement that the study population should be diverse, but no consensus as to whether it should be nationally representative. To obtain an answer, an expert panel was assembled in 2004. This panel, chaired by David Savitz, asserted that the most desirable sampling approach would be to obtain “a full national probability sample of households, recruitment of reproductive age potentially fertile women residing in these households, and prospective monitoring of these women over some period of years for pregnancies and births.”2 The panel recommended studies to assess the feasibility of this approach, which it acknowledged was unprecedented in the US. However, neither resources nor authority were then available to undertake the required field work.
Some thought from the start that this plan was not feasible. Mark Klebanoff, then director of the Division of Epidemiology, Statistics and Prevention Research at NICHD, and I each presented alternative sampling plans to the sampling workshop in March 2004. We believed that only a fraction of potentially-fertile or pregnant women could be efficiently recruited from households. We both felt that some form of sampling of pregnant women in prenatal care was the best pragmatic alternative. Klebanoff argued that population representativeness might not be so critical, especially if that focus siphoned away effort that could be used more effectively to increase the depth of data collection. I argued that it should be possible to create a reasonable degree of representativeness by sampling prenatal care settings and hospitals within a geographic area.
In 2007, NICHD asked the National Research Council and the Institute of Medicine to review the scientific rigor of the National Children's Study. The panel's report, in May 2008, raised several concerns, but listed “The well-designed national probability sample” in which births would be “. . . identified from a probability sample of households . . .” as one of the study's key strengths.3
The household sampling plan had thus been recommended and endorsed by 2 panels of nationally recognized experts. No member of either panel was from the National Children's Study program office, the coordinating center or the current roster of principal investigators in the field. The authority conveyed by these 2 expert committees made their recommendations impossible for the Program Office to ignore. In effect, these experts were the “saviors” of the National Children's Study. They invested the study's choice of sampling methodology with the allure of pure and exact science. The argument that, at its core, the study was theoretically sound had a powerful resonance in many quarters. Paradoxically, however, as Savitz and Ness1 recognize, the sampling plan may have created a study design that, from a practical standpoint, is very difficult to carry out.
THE SITUATION NOW
The National Children's Study, under new leadership, has recognized that household sampling and recruitment have been less efficient than hoped in the Vanguard counties, though not quite as inefficient as Savitz and Ness assert. The figure of 10-40 households per pregnancy from the Pivetz et al document describes the results of 4 years of recruitment. Vanguard data available in January 2010 cited by Savitz and Ness7 reflect recruitment in the field for considerably less than 1 year. Moreover, updated Vanguard data as of May show more enrolled pregnancies than in the January report. Still, current leadership is taking steps to investigate alternatives. In the current phase of the study, decisions are being guided by empirical results. For the next 9-12 months, 10 study locations will each study 3 forms of recruitment: an enhanced form of household sampling; census-like recruitment that begins with questionnaires to a large population; and recruitment through health-care-provider offices.
This latter option, which will begin in Wayne County, MI and 9 other locations this summer, separates sampling and recruitment. Sampling remains household-based, but once the sample is identified, address-matching is used to identify geographically eligible women in prenatal care. This process should work well in relatively small counties, such as Grand Traverse County, MI, where 25% of all women are eligible for the study, 5 provider practices deliver all prenatal care, and 1 hospital delivers 96% of county births.
In Wayne County (and probably all large urban counties), the situation is different. Fewer than 1% of county births are eligible. In 2008, the 383 births of our assigned segments were delivered by 279 practitioners working in 140 different practice sites and took place in 26 hospitals. We have our work cut out for us!
With uncertain funding, a plethora of experts eager to provide advice, and just too much time before field work started, the extraordinary potential of the National Children's Study has been carried along for a decade by a mix of enthusiasm and ivory-tower idealism. The current leadership has recognized this problem, and has developed the decisive modifications to the study design that are currently being tested, but that are not reflected in the Savitz and Ness1 critique. Field experiments yielding empirical data are far more valuable than the enlistment of yet another set of experts to save the study.
The study's 37 principal investigators represent an extraordinary talent pool that includes considerable experience in the enrollment of pregnant women and babies in longitudinal research. They also have the huge advantage of having their feet on the ground in the locations in which the study takes place. In a study so intensely dependent on practitioners, clinics and hospitals, on neighborhood organizations and houses of worship, and ultimately on the goodwill and trust of women all over the country, there is no substitute for investigators, project directors and data collectors who intimately understand the streets and the people of the communities in which they live and work.
Ten of the study's investigators are on the executive steering committee that meets monthly and that works jointly with the study director to make key decisions about study direction. Should none of the current sampling and recruitment options prove optimal, or the protocol prove too burdensome, then other options, including sampling of prenatal care settings, will surely be considered and studied. Indeed, prenatal-care sampling has already been scheduled for discussion in meetings of the executive steering committee and in the next all-investigator meeting, scheduled for August of this year.
I am grateful to my friends David Savitz and Roberta Ness1 for prodding the study with their thoughts and suggestions, and especially for adding their voices to earlier ones suggesting that sampling of prenatal-care settings could be a viable option for the National Children's Study. Vigorous and challenging arguments over study methodology can only help.
Is the National Children's Study “in trouble”? That depends on what is meant by the phrase “in trouble.” In some sense, a project as large and ambitious as this will probably always be in trouble, or at least in enough trouble to prevent complacency. That kind of trouble is good for a study to have.
ABOUT THE AUTHOR
NIGEL PANETH is Principal Investigator of the Michigan Alliance for the National Children's Study, a consortium of institutions organized to conduct the National Children's Study in Michigan. He was a consultant to the first National Children's Study planning meeting in 2000, provided an overview of cohort study designs to the first National Children's Study study assembly in 2000, and cochaired the National Children's Study Study Design Working Group from 2001 to 2004. He currently serves on the Executive Steering Committee of the National Children's Study.