A SIGNIFICANT GAP exists between the availability of child evaluation tools developed for use with culturally and economically diverse populations (Peña & Halle, 2011; Spiker, Hebbeler, & Barton, 2011) that are accessible to both clinicians and paraprofessionals across the spectrum of child-serving agencies, including those that are low resource. In response, we are building NEST—Neurodevelopmental Ecological Screening Tool (Early Childhood). NEST aims to bridge the gaps in community-based child developmental screening and surveillance and address the pressing need for an easy-to-implement, psychometrically validated tool that reflects the reality of the challenges facing children, families, and the providers that serve them. NEST is being developed using a nationwide sample that is representative of the needs of racially and culturally diverse children 3–5 years of age, living in conditions of deep poverty, and being served by providers in homeless and housing programs. In accordance with recommendations by the Council on Children with Disabilities, Section on Developmental Behavioral Pediatrics, Bright Futures Steering Committee, Medical Home Initiatives for Children with Special Needs Project Advisory (2006),Johnson, Myers, and American Academy of Pediatrics Council on Children With Disabilities (2007), and Foy, Kelleher, Laraque, and American Academy of Pediatrics Task Force on Mental Health (2010), and with the needs of the paraprofessional workforce in mind, NEST is designed to serve as a communication tool, enabling staff and parents to engage in a structured dialogue about child and family needs while screening for potential developmental risk. Ultimately, NEST uses a computer-based scoring algorithm to match the child's needs with actionable recommendations aimed at minimizing risk factors and building resilience for children faced with early adversity.
In this article, we outline Phase I of instrument development. First, we discuss our initial conceptualization of the constructs, detail the item development process, and describe the method for implementing and evaluating the feasibility of the tool in low-resourced settings. Next, we review existing theory and research, as well as gold standard instruments to identify domain-specific constructs and subconstructs unique to young children (aged 3–5 years) and their caregivers living in poverty. We then describe the cognitive interviewing process used to further refine items, discuss how we developed the platform to deliver NEST to be family- and provider-friendly and secure, and provide an overview of the pilot study, Finally, we discuss data analyses and present preliminary psychometric data collected on a pilot sample of 60 children served by Massachusetts organizations working with homeless children in 2017. Through the pilot study, we tested the reliability and validity of the initial items and the feasibility of implementing an ecological screener in community settings. The results guided, shaped, and justified the national validation study with 241 children across seven states, the results of which are reported in subsequent papers under review. Implications for practice and policy for children in poverty are discussed.
THE INFLUENCE OF ECOLOGICAL FACTORS ON EARLY CHILDHOOD DEVELOPMENT
The development of young children is both magical and complex. Developmental pathways and future well-being are most strongly influenced by the nature of the primary caregiving relationship and the context in which it occurs (Bassuk & Beardslee, 2014; Grant & Brito, 2010; Perlman, Cowan, Gewirtz, Haskett, & Stokes, 2012; Shinn et al., 2008; Weinreb, Buckner, Williams, & Nicholson, 2006). The intricate dance between caregiver and child (Schore & Schore, 2008), and the “serve and return” nature of this primary attachment (Center on the Developing Child at Harvard University, 2016; National Scientific Council on the Developing Child, 2004), lays the foundation for future developmental outcomes. Surrounded by a supportive environment, children thrive (Bassuk & Beardslee, 2014; Grant & Brito, 2010; Perlman et al., 2012; Weinreb et al., 2006). The accumulation of protective factors buffers a child from certain risks, supporting healthy development; exposure to certain risk factors at the child, parent, or environmental levels that can sway a child's development toward more adverse outcomes (Fantuzzo, LeBoeuf, Brumely, & Perlman, 2013; Friedman, 2000; Hostinar & Gunnar, 2015; Rouse & Fantuzzo, 2009).
For children growing up in impoverished circumstances, healthy development may be threatened by an ecology lacking the protective benefit of strong supports (Nandi, Sweet, Kawachi, & Galea, 2014; Williams, Neighbors, & Jackson, 2003). According to Jiang, Ekono, and Skinner (2016), of the 72.4 million U.S. children younger than 18 years in the United States, 41% live at or below the federal poverty line (FPL). Children are more than twice as likely as older adults to be poor, with the youngest being at greatest risk. More than 5 million children 3–5 years of age grow up in low-income families, another 2.4 million are poor, and 1.1 million live in deep poverty. Racial minorities are disproportionately affected by poverty; 56% of children are either Black or Hispanic, as are those born to immigrant parents, residing with a single parent, or whose parents lack a high school degree (Jiang et al., 2016).
Poverty is a well-established risk factor for child development (Bradley & Cowyn, 2002; Eamon, 2001; Shonkoff & Phillips, 2000; Yoshikawa, Aber, & Beardslee, 2012) and is associated with more residential instability (Jiang & Koball, 2018; Rog & Buckner, 2007), a greater risk of exposure to environmental toxins and lead (Perlman & Fantuzzo, 2010), and more exposure to violence in the community (Finkelhor, Turner, Ormrod, & Hamby, 2009). In addition, children living in poverty often grow up in households with caregivers who have high rates of trauma exposure and associated mental health challenges, such as parental depression (Bassuk & Beardslee, 2014; Beardslee, Gladstone, Wright, & Cooper, 2003; Essex, Klein, Cho, & Kalin, 2002; Pachter, Auinger, Palmer, & Weitzman, 2006). With inadequate access to quality treatment and fractured familial and community support systems (Bassuk & Beardslee, 2014), these children are also at a greater risk for child maltreatment (Finkelhor et al., 2009).
The damaging impact of early adversity on human development is also well-established (Felitti et al., 1998; Gerhardt, 2004; National Scientific Council on the Developing Child, 2015; Shonkoff & Phillips, 2000). Adverse childhood experiences (ACEs) have been linked to an increased risk for health and mental health problems, as well as a variety of social and behavioral problems later in life (Chapman et al., 2004; Dube et al., 2001; Felitti et al., 1998). The presence of ACEs is higher among low-income and minority populations (Cronholm et al., 2015). For example, children who live below the FPL are five times more likely to experience four or more ACEs than their middle/upper-income peers (Halfon, Larson, Son, Lu, & Bethell, 2017). When exposed to six or seven early life adversities, a young child has a 90% chance of manifesting a developmental delay of some kind (National Scientific Council on the Developing Child, 2012).
Children who experience homelessness or housing instability are a high-risk group for developmental challenges. Poverty, ACEs, unstable housing, and experiences of cultural and racial discrimination place an enormous toll on the parents of young homeless children (Lupien, McEwen, Gunnar, & Heim, 2009; Perlman et al., 2012); the weight and relative impact of these stressors increase for single parents (Cairney, Boyle, Offord, & Racine, 2003). Under these circumstances, children who are homeless are more likely to be exposed to domestic violence (Bassuk, Dawson, & Huntington, 2006; The United States Conference of Mayors, 2016). They are six times more likely to have health illnesses such as asthma than their housed counterparts (McLean et al., 2004). Among preschoolers who are homeless, up to 20% demonstrate a mental health condition (Bassuk, Richard, & Tsertsvadze, 2015) and up to 25% demonstrate a developmental delay (Haskett, Armstrong, & Tisdale, 2015). Older children may struggle in school (Buckner, Bassuk, & Weinreb, 2001; Fantuzzo, LeBoeuf, Chen, Rouse, & Culhane, 2012; Fantuzzo & Perlman, 2007; Institute for Children, Poverty & Homelessness, 2017; Obradović et al., 2009). A recent study found that homelessness interferes with academic achievement through its relationship to deep poverty; some current housing interventions (e.g., rapid rehousing) can negatively affect educational outcomes (Cutuli & Hebers, 2019).
Parenting can mediate the impact of poverty, homelessness, and associated stressors on child development, leading to more resilient outcomes (Gewirtz & Taylor, 2009; National Research Council & Institute of Medicine, 2009; Perlman et al., 2012). However, many parents who are homeless are simply too depleted to buffer the impact. Among these families, rates of posttraumatic stress disorder (PTSD) and maternal depression are three to four times those of the general population (Bassuk & Beardslee, 2014; Hayes, Zonneville, & Bassuk, 2013), making parenting more challenging. Healthy nutrition, good sleep, and exercise—all critical for healthy development (Lopresti, Hood, & Drummond, 2013; Walsh, 2011)—are difficult for parents and children to obtain consistently when living in shelters, in motels, or in overcrowded situations with family or friends.
SCREENING AND ASSESSMENT OF CHILDREN
For young children, the cascade of consequences associated with growing up in underresourced settings can be mitigated by early identification, intervention, and support (Kazdin, 2010). Some children may not need any intervention, whereas others may have extensive need. Screening helps quickly identify and triage who may need support and may also provide guidance on the level and type. In settings with little resources, screening helps better target referrals and the provision of services.
Various compendiums have been published to help practitioners weed through the field to find an instrument to conduct child evaluations (Moodie et al., 2014; Ringwalt, 2008), and numerous tools and instruments are available to evaluate children's neurodevelopmental status. For example, the Ohio Department of Education (2010) catalogued 80 tools for use with children birth to 5 years of age and 93 tools specifically for children between birth to 3 years of age. In addition, the California Statewide Screening Collaborative, through its Race to the Top—Early Learning Challenge, developed The Developmental and Behavioral Screening Guide for Early Care and Education Providers, a guide for parents and educators on when and how to screen children developmentally (WestEd Center for Prevention & Early Intervention, 2015). In 2014, the Administration of Children and Families published Birth to 5: Watch Me Thrive! A Compendium of Screening Measures for Young Children. This federal publication narrowed the field to 11 evidence-based standardized tools with minimum sensitivity and specificity scores of 0.7. The recommended measures require parent/caregiver report of child functioning, include subscales to measure social and emotional development for the child, and are constructed to reduce false-negatives or false-positives to accurately identify children who may be at risk for developmental concerns (U.S. Department of Health and Human Services, 2014).
Despite the plethora of instruments, none are normed specifically on children in poverty or simultaneously assess ecological factors as predictors of developmental well-being. In addition, providers do not routinely use them. Barriers to utility are many: accessibility, time, and cost, to name a few. Most instruments can be difficult to administer and interpret (Moodie et al., 2014). Typically, administrators must possess an advanced educational degree (often in the clinical or medical sciences) and/or be licensed. Instruments can also be lengthy and require administrators to reference a manual to interpret results, making it difficult for even more well-resourced providers to use. Although some measures now provide online scoring and administration options, purchase of computerized scoring may be cost prohibitive.
Ethical and educational guidelines indicate that providers conducting standardized assessments should be familiar with certain principles of measurement (Meyer et al., 2001). The Standards for Educational and Psychological Testing (American Educational Research Association, American Psychological Association, & the National Council on Measurement in Education, 2014) have been considered the gold standard in guidance on testing for more than 50 years. Jointly developed by the American Educational Research Association, the American Psychological Association, and the National Council on Measurement in Education, these well-established standards guide providers on understanding the psychometric properties of testing, issues of fairness in testing, administration and interpretation, and the rights and responsibilities of administrators and consumers. Providers who administer screening or assessment instruments need to be well trained to understand a test's intended use and psychometric properties, to know its limitations, and to know how to interpret results in the context of the child's life (Cicchetti, 1994). This level of training is not readily available for the 2.4 million direct service workers across service sectors serving young vulnerable children (Bureau of Labor Statistics, U.S. Department of Labor, 2012,2014a,2014b,2014c; Mullen & Leginski, 2010).
For economically and culturally diverse populations, screening and assessment may be further limited by factors beyond basic access. Not all instruments are “culture fair” in their construction. Many of the most common developmental assessments available today were derived from normative samples that are representative of U.S. census trends of primarily White, middle-class children (Glascoe, 2008,2010; Sparrow, Balla, & Cicchetti, 2005; Squires & Bricker, 2009). Where no supplemental normative group data are available, tests may not be considered appropriate for use with certain minority group populations due to potential for bias in test construction. In addition, bias in testing can exist in how a person responds to the testing context as well as in how an examiner interprets the results. Artifacts of the socialization process related to issues of class, gender, and race can greatly influence testing across cultural groups (Helms, 2006).
EXPANDING SCREENING FOR AT-RISK CHILDREN
Traditional assessments examine key aspects of children's neurodevelopmental functioning including elements of motor, cognitive, language, and social-emotional functioning (Moodie et al., 2014; Ringwalt, 2008; Squires & Bricker, 2009). Common assessment instruments do a very good job of assessing child functioning in these areas. For example, the Vineland Adaptive Behavior Scales has been around for nearly 30 years and is well regarded as a gold standard for assessing overall child development and autism spectrum disorders (Kwok et al., 2017; Sparrow, Cicchetti, & Saulnier, 2016).
Despite their strengths, many tools do not fully assess functioning within a broad ecological context. In a review of Bronfenbrenner's model of human development, Eamon (2001) suggests that practitioners who work with low-income families and children should assess aspects of the caregiver's functioning and the family's environmental context to get a more accurate understanding of factors impacting the child who may require intervention.
In many “low-resource” settings—community organizations strapped by inadequate economic and human resources—the staff are primarily paraprofessionals. Second only to caregivers, this cross-sector workforce may be the first line of defense against adverse developmental consequences for children who are poor and otherwise vulnerable to risk. However, because of the barriers discussed, children in these settings are either not routinely screened or, at best, inadequately evaluated. The result is often missed opportunities for early intervention or a poor match with service referrals and low-service utilization thresholds (Council on Children with Disabilities, Section on Developmental Behavioral Pediatrics, Bright Futures Steering Committee, Medical Home Initiatives for Children with Special Needs Project Advisory, 2006; Foy, Kelleher, Laraque, & American Academy of Pediatrics Task Force on Mental Health, 2010; Guerro, Garro, Chang, & Kuo, 2010; Marks, Glascoe, & Macias, 2011).
When children's developmental needs go unnoticed and unaddressed, lags turn into delays, laying an insecure foundation for later development (Shonkoff & Phillips, 2000). For children receiving services in low-resource settings, contextual factors exert tremendous influence on developmental outcomes (Bronfenbrenner & Morris, 1998; Hoswarth, 2010; Morello-Frosch, Zuk, Jerrett, Shamasunder, & Kyle, 2011; Nandi et al., 2014; Williams et al., 2003). Without valid and reliable data on these factors, and an understanding of their relative influence on the child's development, service referrals may be misinformed and interventions may miss the mark altogether.
In direct response to the challenges, NEST is being developed to fill a gap in the field. NEST is meant to be used in low-resource settings, where personnel have limited time and capacity for in-depth counseling/interviews. In addition, NEST is meant to be used by the paraprofessional staff—day care providers, Head Start teachers, and homeless service staff—who may not be formally trained to conduct in-depth assessments beyond the scope of their service system (e.g., education, childcare, housing). In addition, NEST is brief taking less than 30 minutes to complete and is in line with screening best practices (Mackrides & Ryherd, 2011; Moodie et al., 2014). Multiple-choice questions are utilized to ultimately inform the scoring algorithm and to be completed after Phase II data collection is analyzed. The scoring algorithm will act as a decision matrix, combining hundreds of data points to identify potential risks to a child's development across multiple levels—child, caregiver, and environment. Use of a standardized tool can significantly reduce the potential for provider bias and increase effectiveness of identifying the type and level of service intervention needed. Provided in real time, recommendations are immediate. In addition, NEST offers a user-friendly format for providers and caregivers to have a conversation about the child's well-being.
Constructing a culturally relevant and developmentally targeted instrument that is psychometrically valid and appropriate for use in low-resource settings, where it can be administered by paraprofessionals, is a challenging task. To begin the initial creation, our goals were threefold: (1) Develop an item list across three domains (child, caregiver, and environment); (2) construct an online, user-friendly delivery platform; and (3) pilot the tool with a group of parents and providers to test its feasibility. Our goals and outcomes are summarized in Figure 1.
Conceptualizing the constructs
The first step to developing NEST was to identify core constructs and a pool of items. As we were taking an ecological approach, we knew we wanted to have three domains: child, caregiver, and environment. Within those domains, we drew from scientific knowledge on child development and research on risk and protective factors for children to identify subconstructs that were most relevant to the developmental trajectories of children experiencing poverty (Buckner, Mezzacappa, & Beardslee, 2009; Center on the Developing Child at Harvard University, 2007,2009,2010; Gerhardt, 2004; Ginsburg, 2007; National Research Council & Institute of Medicine, 2009; National Scientific Council on the Developing Child, 2015; Shonkoff, 2010; Shonkoff & Phillips, 2000). Our initial domain and construct list is described below.
Within the child domain, we targeted activities of daily living (ADL), gross and fine motor skills, and expressive and receptive language skills, as these are routinely measured to assess child development. We did not include items related to vocabulary due to the high likelihood of interference by extraneous factors such as culture and bilingual home environment. We also targeted executive functioning (cognitive processing, problem solving, attention, and self-regulation), social-emotional skills (adaptability/coping, peer relationships), and play; it is well established that these constructs are related to developmental risk and resilience for children (Bergen, 2002; Bierman, Nix, Greenberg, Blair, & Domitrovich, 2008; Cutuli & Herbers, 2014; Ginsburg, 2007; Hoswarth, 2010,Luckey & Fabes, 2006; Masten, 2001; Moore, 2013; Ringwalt, 2008; Ryan, 1999; Squires & Bricker, 2009). In the caregiver domain, we included items to assess risk factors known to impact children's developmental well-being, including risk for PTSD and depression, and parenting ability and style. The environment domain focused on residential instability, housing/homelessness, exposure to trauma, and food insecurity.
Research for item development
Once the constructs and subconstructs were established, our next task was to develop the items. We first set some parameters about the structure of the questions themselves. We determined that each item had to be multiple-choice so that it could be easily and consistently administered online. Items had to be of low cognitive burden (De Jong, 2009; Hollender, Hofmann, Deneke, & Schmitz, 2010; Sweller, Van Merriënboer, & Paas, 1998), and, whenever possible, minimize any stigma that caregivers may feel in responding. The initial pool of items consisted of 268 questions (child: n = 130; caregiver: n = 94; and environment: n = 44).
To develop the items, we reviewed a variety of sources, including gold standard measures. This is a common step to take during the development of an item pool (Tucker, 2010). For the child domain, these included Ages and Stages Questionnaire—Third Edition (Squires & Bricker, 2009); Brigance Early Childhood Screen III; social-emotional and self-help scales (Brigance & French, 2013); Parents' Evaluation of Developmental Status (Glascoe, 1997,2008); and the Behavior Assessment System for Children—Third Edition (Kamphaus & Reynolds, 2015). Each of these tools has its own conceptual model, constructs, and subconstructs, which helped refine our own thinking on how best to structure NEST to meet our objectives. We then proceeded to create items that fit our structure (multiple-choice, low cognitive burden, consistently administered online, ecologically focused, and predictive of risk or resilience) and to minimize stigma. Many items on gold standard measures referenced have skill ceilings/floors—that is, administrators ask a number of increasingly difficult questions and only stop asking when a child has answered a certain number incorrectly. This setup does not work well for an online tool administered by paraprofessionals and so we crafted our items accordingly. We describe our refinement process further in the “Item Refinement: Cognitive Interviewing” section.
For the caregiver domain, we reviewed the Trauma Symptom Inventory II (Briere, 2011), the Posttraumatic Maladaptive Beliefs Scale (Vogt, Shepherd, & Resick, 2012), the Adult–Adolescent Parenting Inventory (AAPI-2; Bavolek & Keene, 1999), the Beck Depression Inventory II, and the Beck Hopelessness Scale (Beck, Steer, & Brown, 1996; Dozois & Covin, 2004), and various free screening tools. We incorporated three publicly available, psychometrically valid screening instruments: the Patient Health Questionnaire-9 (PHQ-9) to assess caregiver risk for depression (Kroenke & Sptizer, 2002), and the Abbreviated PTSD Checklist 6 for the DSM-5 (PCL-C for civilians) to assess care-giver risk for posttraumatic stress responses (based on the full PCL-5 by Weathers et al., 2013). We chose the PHQ-9 and the PCL-5 as both are well-established, psychometrically sound instruments and available in the public domain for use by clinicians and paraprofessionals for the purpose of screening for risk, making them a good fit with NEST. In addition, inclusion of these validated screening tools further enhances the psychometric properties of NEST. The PHQ-9 has a reliability of .89 in primary care studies and has a sensitivity of 88% and a specificity of 88% for major depressive disorder. Adults who score high on the PHQ-9 are up to 13 times more likely to be diagnosed with depression by the mental health professional (Kroenke, Spitzer, & Williams, 2001). The PCL-5 is a self-report screening measure that assesses the presence and severity of PTSD symptoms. Items on the PCL-5 correspond with DSM-5 criteria for PTSD. The PCL-5 test scores demonstrate good internal consistency (α = .94), test–retest reliability (r = .82), and convergent (rs = .74–.85) and discriminant (rs = .31–.60) validity (Blevins, Weathers, Davis, Witte, & Domino, 2015).
Finally, we included items on parenting, as caregivers exposed to significant interpersonal trauma or struggling with depression may also demonstrate parenting styles that can negatively impact child development (Felitti et al., 1998; Vogt et al., 2012). We reviewed the AAPI-2 (Bavolek & Keene, 1999) and the Parental Stress Inventory IV (Abidin, 2012) and developed items thought to impact parenting outcomes, including sense of competence in the parental role and quality of attachment to the child (Bendell et al., 1994; Bendell, Stone, Field, & Goldstein, 1989; Mash & Johnston, 1983; Mowbray, Oyserman, Bybee, & MacFarlane, 2002).
Children and families exist in a context; environmental factors such as poverty, threats to safety, residential instability, and exposure to environmental toxins also pose significant risks to child development (Morello-Frosch et al., 2011). Phase I NEST items covered a variety of environmental factors, thus expanding the concept of adversity from individual and caregiver factors to also include the potential impact of community and public policy factors on the child (Cronholm et al., 2015). We used the U.S. Household Food Security Screener–Short Form (Blumberg, Bialotosky, Hamilton, & Briefel, 1999) to screen for food security, modifying it slightly to fit the online format. Our background research did not find any screening instrument for children that asks about residential instability—a key component of understanding children's development. To create items for this construct, we drew on surveys used in prior research studies on homeless families (Hayes et al., 2013) and from the Vulnerability Index—Service Prioritization Decision Assistance Tool (OrgCode, 2013), a survey used by various homelessness program. We also included the ACE scale for the child to assess the degree of exposure to early adversity. Identifying ACEs among young children early in their development can predict health outcomes (Finkelhor, Turner, Shattuck, & Hamby, 2013; Sacks, Murphey, & Moore, 2014). For low-income children in our target age range (3–5 years), preliminary research suggests when these children have a high ACE, it is a significant predictor of developmental problems (Blodgett, 2014; Egger & Angold, 2004). Finally, we included one item on places to play; access and safety are often issues for children living in poverty (Moore, Diez Roux, Evenson, McGinn, & Brines, 2008).
Expert consensus is a common method used to develop new measures (Grant & Davis, 1997). We vetted both the constructs and original item pool with an expert panel that consisted of four psychologists (clinical child and developmental) and one psychiatrist. The panel brought expertise in child development, assessment, parenting, maternal depression and trauma, poverty, and homelessness. Three had previous experience in instrument development. We asked the panel to evaluate our proposed item list for each domain/subconstruct using the following criteria: (1) Does the item capture the most important aspect of the construct we want to target for this population? (2) Is it a good predictive indicator of developmental risk for this population? and (3) Is it a marker of resiliency and should be included? After discussion and compilation of their feedback, we narrowed the item pool to 120 questions.
Item refinement: Cognitive interviewing
Our next step was to conduct cognitive interview testing with both caregivers and providers from family homeless shelters (caregivers: n = 15; program staff: n = 10). Cognitive interviewing is a process whereby item developers seek to understand how the respondent is interpreting and answering the question. Interviewers ask individuals to answer the item and a set of questions to understand how the respondent arrived at his or her answer (Sparrow et al., 2005). We used two or three probes per question; examples included are as follows:
- How hard or easy was it to answer this question?
- Is the word “[insert word from item]” the right word here? Is there a better word to use?
- Do you think other parents will understand what is meant by this action/activity?
- Please repeat this question in your own words.
- Talk me through how you came up with your answer.
- How difficult was this to answer?
Cognitive interviewing is used to improve wording, reduce ambiguity, increase cultural relevance, and ensure conceptual coverage of the constructs (Willis, 2005). We selected 72 questions for inclusion in this process; the remaining 48 items on our initial list were excluded from cognitive testing due to prior use with similar populations (e.g., PHQ-9 questions, U.S. Department of Agriculture food insecurity questions) or agreement and confidence between the principal investigators and the research team in the phrasing. Because of the number of questions, we developed four protocols: Parent—Protocol A, Parent—Protocol B, Provider—Protocol A, Provider—Protocol B. Each protocol had approximately 14 questions. When conducting cognitive testing, field testers were looking to reach saturation; that is, they were interviewing people until the point where their comments became repetitive. Field testers were researchers who had not been involved with item development to ensure unbiased results. They interviewed 10 providers. All were female. Forty percent were Black/African American, 60% were White, and 10% identified as Hispanic or Latino. Among the 15 caregivers who were interviewed, 93% were female and 7% male. Fifty-three percent identified as Black/African American, 33% as White, 7% as Alaskan Native/American Indian, and 7% as Other. Fourteen percent identified as Hispanic/Latino.
As a result of this process, we made a number of adjustments to the questions. Some changes were minor. For example, “Does your child adjust well to changes in routines?” became “Does your child adjust well to changes in daily routine?” This was in response to the need for clarity about the routines to which the question was referring (e.g., eating? sleeping?). We also choose to eliminate a few questions. For example, “Can your child put small items like macaroni pasta or beads onto a piece of string or shoelace?” did not resonate with parents or providers. Because the goal of the question was to understand fine motor skills, we replaced this question with a different one about fine motor skills. For other questions, particularly those designed to elicit information about self-regulation and behavior, respondents had more complicated responses. For example, the question “Can your child play well with a group of children during free time, for at least 10 min?” created confusion among respondents. Providers responded that children behave differently when with their parents versus in front of the staff, whereas parents took the question literally. Some parents focused on the “10-min” specification, whereas others wondered about the size of the peer group (two kids vs. 10 kids), or the type of activity being played. Thus, we changed the question to “Can your child play well with other children?” eliminating confusion, lowering the cognitive load of the item, and focusing the item on the issue of the quality of interpersonal relationships during play. At the end of this process, we had a final list of 72 items to take to the field for Phase I testing.
Developing the platform
Simultaneous to developing the items, we worked with a team of graphic designers and developers to focus on our second goal: creating a secure online platform from which providers could access the assessment (Figure 2). Knowing that NEST would be used by the paraprofessional staff in community settings, we wanted to ensure that the platform was easy to use and well designed. Many providers resist using computerized instruments due to difficulty with technology, a lack of usability, and not knowing how to fit them into their normal workflow (Garg et al., 2005). These issues are exacerbated in low-resource settings due to minimal resources for staff training and constraints on time. Understanding these dynamics, we created a simple, user-friendly interface and constructed NEST to be completed in under 30 min, so as to be integrated into an average case management meeting (see Figure 2 for a prototype item). Users (providers) are able to create a profile for each child and administer NEST, all while protecting confidentiality and secure transfer of data. The NEST platform is web based, requiring only an Internet connection and browser, makes use of a user-friendly navigation interface, and is accessible on standard computer desktops or mobile devices including desktops and tablets. Once finalized, it will be compliant with accessibility requirements of Section 508C of the Americans with Disabilities Act and with Web Compliance Accessibility Guidelines 2.0 (WCAG 2.0). Ultimately, this platform will contain an algorithm to link scores from each domain with recommended service interventions at the child, caregiver, and environmental levels.
Piloting the tool
Our third goal for the pilot phase was to field test the instrument. As the goal of the pilot was to determine feasibility, the sample size is necessarily small and located in close geographic proximity due to resource limitations. Based on the pilot results, we will conduct a Phase II nationwide study using a larger, more diverse sample. (See the “Discussion” section for more detail.) For the pilot, we partnered with several Boston area programs serving families who are homeless or recently housed. We trained more than three dozen case managers to administer NEST to parents. The 90-min training included a description of the tool and why it is important to the field; how to recruit families and administer the tool, including sample scripts and practice scenarios; and a discussion of implementation/data collection logistics. Caregivers of 60 children completed NEST with their case managers or program staff. Results are described in the following text.
Pilot data analysis process
In the Phase I of NEST development, we conducted preliminary psychometric evaluation of the tool on the small pilot sample (n = 60). To evaluate the items used in the pilot phase testing of the tool, we used a one-parameter Item Response Model, which is appropriate in low sample sizes, and limited the analysis to one subdomain at a time. Specifically, we used a Rasch-based Rating Scale Model (RSM) to evaluate the fit of the items. Rasch-based models are called one-parameter models because all items are constrained to provide the same amount of information about the construct. Conceptually, this is similar to a factor analysis model where all factor loadings are determined to be the same. The single parameter of the RSM model is the item location or item difficulty parameter that provides information about the location of the item on construct. For example, all the depression items are assumed to have the same relationship to measuring depression, but some items are indicative of higher levels of depression and some items are indicative of lower levels of depression.
A key benefit of this Rasch-based measurement approach is that the models allow for testing the fit of items using infit statistics, even considering the low sample size. In situations such as this pilot, where the goal is to select the best items, infit statistics allow us to select items that meet the restrictive fit standard of the RSM model and reject items that do not fit the model well. This limited parameterization of the model trades off sample size for constraints and allows us to be conservative with our item selection. We used traditional internal consistency reliability statistics (Cronbach α) to measure the internal coherence of the items to the measured constructs. Finally, we fit confirmatory factor analytical (CFA) models to test for unidimensional and construct validity.
A sample of 60 individuals (caregivers) were recruited from five programs. The principal investigators trained the program staff to recruit participants and supplied them with institutional review board-approved flyers. Participants received gift cards for participating in the study. Recruitment criteria were as follows: (1) The caregiver must have a child who must be between 3.0 and 5.11 years of age; (2) the child must be homeless or housed within last 30 days; and (3) the caregiver completing NEST must have working knowledge of English.
The decision to limit our sample to children who were homeless or recently housed was intentional; children living in poverty, from racially diverse groups, and who are residentially unstable are some of the most vulnerable to suboptimal developmental outcomes—with homeless children at the far end of the continuum for risk. In designing an ecological screener, we wanted to create a tool that could identify a variety of risks across a population with complex needs. Children experiencing homelessness fit that description. For both children and caregivers, 40% identified as White, 45% as African American, 25% as Hispanic, and 15% as another race category. Caregiver age ranged between 19 and 70 years, with the average age of caregivers being 30 years. By sampling strategy, the children were all between 3.0 and 5.11 years old, with an average age of 4.4 years; child cultural demographics matched their caregivers.
Domain and item analysis
Using the constructs and subconstructs identified in Figure 1, we estimated RSM models, reliability statistics, and CFA models separately for each of the subconstructs. In the process of our initial models, we further subdivided the physical and executive function subdomains. It should be noted that CFA models were underpowered but used in conjunction with other indicators to provide support or identify problems with the reliability and validity of the constructs. Four constructs comprised the child domain (physical, communication, executive function, and social-emotional), three constructs comprised the caregiver domain (depression, trauma/PTSD, and parenting), and three constructs comprised the environment domain (food security, child ACEs, and housing/homelessness). Table 1 shows the decision matrix we used to identify constructs within each overall domain that yielded good evidence in the pilot and would be targeted for reworking in Phase II. We used a detailed version of Table 1 to identify constructs and items that needed revision.
Table 1. -
||RSM Item Fit (Infit Range)
|Physical—Activities of daily living
|Physical—Gross and fine motor
Note. CFA RMSEA = confirmatory factor analytical root mean square error of approximation; EF = executive function; PTSD = posttraumatic stress disorder; RSM = Rasch-based Rating Scale Model.
Overall, of the 72 items we tested, data indicated that 58 items had strong psychometrics (based on Cronbach α scores). We made minor revisions to three items, eliminated 13, and added a total of 47 items to NEST across all domains. Of these, 22 underwent a second round of cognitive interviewing with a small sample of parents and paraprofessionals (n = 6). New items not included in cognitive testing were taken from existing screeners that were already psychometrically validated or well established (e.g., lead screening survey, health care access questions).
Psychometric data enabled us to refine some constructs, eliminate irrelevant factors, and focus sharply on subconstruct components that contribute most to child development. For example, we eliminated all items that corresponded to ADL in the child domain. The psychometric data indicated weaknesses in this subscale, with very low reliability (Cronbach α = 0.51). The ADL are age dependent with high variability in normative range; without floor and ceiling rates available in a general screener such as NEST, it would be difficult to use them as a general developmental marker and specificity would likely be very low (Sparrow et al., 2016). Daily living skills are also culturally and context specific; families living in unstable situations or with extraordinary threats to their day-to-day well-being (e.g., food insecurity, community violence) may not be focused on teaching young children these skills in the same way that more well-resourced parents may be. Furthermore, cultural mores may dictate a variety of ways to care for children, some stressing independence from an early age and others not. As such, we concluded that although ADL may be important on some measures, they were not essential for our tool.
In addition to psychometric evaluation, we looked at each item's frequency of responses to identify items that had floor or ceiling effects. For example, the item “Thoughts that you would be better off dead or of hurting yourself in some way” was not endorsed by any of the 60 respondents; 55 answered “not at all,” and five respondents selected the “prefer not to answer” option. This item was a poor fit for NEST as it failed to discriminate among respondents. However, because of the importance of screening for suicidality, we chose to include it in Phase II to further test the item with a larger sample. Other items, such as those related to housing/homelessness, were not psychometric scales but individual items that assess manifest constructs such has length of time homeless or residential instability (e.g., number of times a child has moved). For these items, we checked the frequency of responses but did not conduct further analyses.
Data also indicated two additional subconstructs that required further research: parenting and play. The parenting subscale in the caregiver domain demonstrated overall low reliability across items (Cronbach α = 0.41). As a group, the items did not show good internal consistency and held very little ability to predict potential for developmental risk.
Parenting is multidimensional, influenced by context, resources, and child and cultural factors. In addition, self-report by parents about their own parenting is subject to social desirability bias (Morsbach & Prinz, 2006). For the pilot, NEST's initial parenting subscale was compromised of items related to general sense of parental competence and closeness to the child. However, as these items did not perform well, we revisited the literature on parenting. Further research indicated that certain parenting behaviors predict child behavior more robustly than a parent's subjective sense of competence or attachment. Although variations across culture, class, and context exist, laxness or overreactivity in parenting behaviors are most strongly correlated with child behavior problems (Arnold, O'Leary, Wolff, & Acker, 1993; Baumrind, 1967,1971; Rhoades & O'Leary, 2007). To ensure NEST is theoretically and empirically grounded, we discarded the original parenting items and reconceptualized the Parenting Scale to focus on these two factors. To develop the items, we reviewed 13 existing parent assessment tools for their focus on laxness and overreactivity and for low social desirability. We chose to include the Parenting Scale (Arnold et al., 1993) in NEST. The Parenting Scale is a publicly available tool based on Baumrind's (1967,1971) two-factor model of parenting practices. It demonstrates a good fit with culturally and racially diverse populations and identifies concrete parenting skills as appropriate targets of service delivery (Lorber, Xu, Smith Slep, Bulling, & O'Leary, 2014; Reitman et al., 2010; Steele, Nesbitt-Daly, Daniel, & Forehand, 2005).
Items on the social-emotional development scale of the child domain also needed refinement. Initially, we crafted items that focused on peer relationships and play. However, the pilot data revealed that these items did not form a unidimensional construct that had good internal consistency or item fit to the latent construct. These results led us to deeper reflection and research about exactly what areas needed to be screened. In particular, we recognized the need to assess play more comprehensively, because it is core to so many skills developed in early childhood (Center on the Developing Child at Harvard University, 2010). Because the research on play is vast, we conducted an extensive literature search to determine what types of play are predictive of risk and resilience. Symbolic (imaginary) play indicates the development of cognitive flexibility, which relates to resilience and, if delayed, may indicate developmental risk (Ryan, 1999). It also is a key way that children develop self-regulation skills (Bergen, 2002; Lifter, Foster-Sanda, Arzamarski, Briesch, & McClure, 2011; Luckey & Fabes, 2006). The affect or tone of a young child's play can offer insight into the child's needs, feelings, and experiences (Findling, Bratton, & Henson, 2006; Lifter et al., 2011; Luckey & Fabes, 2006; Ryan, 1999). On the basis of this research, for Phase II we developed an 11-item subscale that focuses on symbolic/imaginary play, play with others, and the child's affect during play.
The reconceptualization of the parenting and play scales further narrowed the item pool for these scales to the components with the most predictive power. Finally, to support the ecological framework for evaluating children in poverty, we broadened the scope of the environment domain to better capture known risks for children in low-resource settings, adding 20 questions on health care access, continuity of health care, and lead exposure. The revised domain and construct list to be used in Phase II is outlined in Figure 3. The final version of NEST being used for Phase II data collection includes 95 questions across the three domains: 48 items in the Child domain; 18 items in the Caregiver domain; and 29 items in the Environment domain. During Phase II, we will test these items with a larger, more geographically diverse sample; these data will be used to generate the final algorithm that will generate real-time results and recommendations for caregivers.
The science of child development is rich and dates back several centuries. The earliest U.S. settlers recognized that for society to evolve and prosper they had to ensure the healthy development of the next generation (Mintz, 2004). However, despite this keen insight, society did not view all children equally. Gender, race, and class created great divides; while some children benefited from going to school, receiving better nutrition, living in cleaner and safer housing, or being allowed to play, others did not. These divides persist today; children are the ones most likely to be poor in America, and children of color even more so. One in five children younger than 6 years live in poverty, and half of these live in extreme poverty. Seventy percent of children living in poverty are also children of color, with White families holding seven times more wealth than Black families and five times more than Hispanic families (Children's Defense Fund, 2017; U.S. Census Bureau, 2017). These inequalities manifest themselves in myriad ways, including education, housing, nutrition, community violence, and access to health care (Mintz, 2004).
The developmental sciences are not immune (Mintz, 2004). Racial and economic bias has influenced test construction and how results are used. Although there is no shortage of tools and instruments on the market to assess children's development (Moodie et al., 2014), most assessment instruments have not been developed, normed, or psychometrically validated on low-income, culturally diverse populations.
Similar to the early U.S. settlers, today's policy makers should be morally and scientifically compelled to act to prevent adverse developmental outcomes for children to ensure the social, cultural, and economic advancement of society. To accomplish this goal, paraprofessionals who work with families and children affected by poverty need the resources to do their jobs effectively. The gap between what we know and what we do in meeting children's needs is great. We know that screening and early detection, followed by targeted intervention, work to thwart developmental risk and that an array of early childhood development services, including universal developmental screening, are necessary to address the urgent needs of children in poverty. However, the road from research to practice and policy is long, and implementation of standards slow, leaving far too many children at risk.
In low-resource settings where providers often lack resources, time, and training, the use of standardized measures in child assessment is rare. For example, a review of homeless service providers reported that only 18% adequately assess children's development and only 4% report using any type of standardized instrument (DeCandia, Bassuk, & Richards, 2017). In addition, providers' judgment is subjective. Research indicates that the use of standardized instruments that provide immediate feedback improves outcomes (Kazdin, Whitley, & Marciano, 2006; Kelley & Bickman, 2009). Despite this evidence, most providers do not routinely use standardized assessments or screeners as a regular practice. Even in primary care settings staffed by medical providers trained to use diagnostic measures, only 28% of children are routinely screened for developmental monitoring (Guerro et al., 2010; Marks et al., 2011).
The most potent solutions to pressing social issues often come from a combination of on-the-ground clinical experience and a robust literature base. The lack of child screening and assessment for children served by low-resource settings, and the missed opportunities for developmental support of vulnerable children, was identified nearly 20 years ago while one of the authors was working with children affected by a variety of ACEs in a community-based shelter setting. The lack of economic and clinical resources made routine screening of children in shelter impossible. As a result, NEST was designed to put the right tools in the hands of the people working on the front lines with low-income children and families.
Our goal for this pilot study was to determine the feasibility of developing and implementing an ecological tool with parents of young children in low-resource settings. Throughout the pilot study, we were guided by the very people whose lives NEST might impact. Parents and caregivers expressed gratitude for having been asked to participate in the project and give their perspective on what worked, something they rarely got to do. Providers reported that NEST fit well into their program structure and daily workflow, taking only 15 min to complete, and provided them with a structured way to talk about potentially difficult topics with caregivers. The input of both groups was pivotal to NEST's purpose, to give providers and families the power to implement routine developmental screening of young children—no matter what their life circumstances.
In Phase II, we will test NEST with a larger, nationwide sample to further refine the tool and test its psychometric validity. We will use Phase II to develop the scoring, programming, and user-friendly interface. As a screening tool, NEST is meant to sort, identify level of need, and provide matched recommendations. To accomplish this, in Phase II, we will develop a scoring algorithm that will generate a composite score to broadly classify a child to be at low, moderate, or high risk for developmental delay. Composite scores on their own are not psychometrically meaningful or practically useful. To ensure that NEST has practical value to practitioners and caregivers alike, NEST will also provide domain-level scores and specific scores for each construct. We will develop recommendations for each construct. These recommendations will provide caregivers (and the practitioners who support them) with actionable next steps based on the child's level of developmental risk. For example, if NEST results show that a child is classified as at “high risk” for a language delay and the caregiver is also suffering from a likely depressive disorder, the recommendations might indicate that further child evaluation by a pediatrician or developmental specialist (e.g.., early intervention referral) be pursued and a referral be made for clinical support for the parent. As community resources are not always readily available, we will also provide practical, concrete strategies to support the identified need in the home (e.g., singing songs, reading books for language development).
Based on provider reports to date, we anticipate the final version of NEST to take 15–20 min to complete, at the end of which a printable PDF report is generated. NEST will use a computerized scoring algorithm that will assign a weighted value to the risk factors assessed under each domain, combining them into a single score composite for the purpose of broad classification. This classification will then guide the recommendations generated. After caregivers complete NEST, they will click “submit” and recommendations tailored to their child will appear on the screen and be available for print.
Phase II development will also include the creation of a program/organization-level report function. This will allow administrators to quickly determine type and severity of need, in aggregate, at their particular organization to inform program development and resource allocation. It will also aid programs in advocating for needed community recourses based on actual quantitative data. Ultimately, NEST can help child-serving programs identify children at highest risk for developmental challenges, guide them to the appropriate level of service, and use limited resources more efficiently.
Like all pilot data, this study has several limitations. The obvious limitation is with a sample size of 60 participants, and all parameter estimates are subject to sampling bias. Anticipating this limitation, we used current estimates only to eliminate items and scales that did not fit basic assumptions. We also used Item Response Theory (IRT) models and other techniques that have lower sample size requirements. However, there are still questions about the generalizability of these findings to other populations, given the sample's limited geographic distribution and potential sampling bias. In addition, given the small pilot sample size, CFA models should be viewed with caution; specifically, the parameters of the CFA model are unreliable at low sample size, although gross misfit can be identified.
A more systematic limitation of this study is the use of parent self-report measures to evaluate children. Parent self-assessment and parent reporting about their child may be biased in important ways. The current study could not address this limitation. However, in the Phase II study, we will include gold standard clinician-administered assessments to better understand the limits of both the self-report and parental assessment used by NEST. While acknowledging these limitations, it is important to highlight that pilot studies are not designed to be full assessments of the object of investigation but rather an attempt to identify potential problems and solutions that can be used in the larger Phase II study.
Childhood is a time of opportunity and risk; how adults respond to children during the earliest years will influence their development and life course. NEST combines decades of research in child development and assessment with the latest advances in computerized clinical decision support systems (Garg et al., 2005) to close the gaps in community-based child evaluation, providing an avenue to universal screening. NEST is a child development screener whose purpose is to determine a child's level of risk for developmental challenges; its design allows for identification of the sources of that risk across a child's ecology. Conceived from real-life circumstances, and developed in partnership with intended users, NEST may offer a practical and promising solution to the need for neurodevelopmental screening of young children in low-resource environments
Abidin R. R. (2012). Parenting Stress Index, Fourth Edition. Lutz, FL: PAR Inc.
American Educational Research Association, American Psychological Association, & the National Council on Measurement in Education (NCME). (2014). Standards for educational and psychological testing. Washington, DC: Authors.
Arnold D. S., O'Leary S. G., Wolff L. S., Acker M. M. (1993). The Parenting Scale: A measure of dysfunctional parenting and discipline situations. Psychological Assessment, 5, 137–144.
Bassuk E. L., Beardslee W. (2014). Depression in homeless mothers: Addressing an unrecognized public health issue. American Journal of Orthopsychiatry, 84(1), 73–81.
Bassuk E. L., Dawson R., Huntington N. (2006). Intimate partner violence in extremely poor women: Longitudinal patterns and risk markers. Journal of Family Violence, 21(6), 387–399.
Bassuk E. L., Richard M., Tsertsvadze A. (2015). The prevalence of mental illness in homeless children: A systematic review and meta-analysis. Journal of the American Academy of Child
and Adolescent Psychiatry, 54(2), 86–96.
Baumrind D. (1967). Childcare practices anteceding three patterns of preschool behavior. Genetic Psychology Monographs, 75(1), 43–88.
Baumrind D. (1971). Current patterns of parental authority. Developmental
Psychology, 4(1, Pt. 2), 1–103.
Bavolek S., Keene R. G. (1999). Adult–Adolescent Parenting Inventory—AAPI-2: Administration and development handbook. Park City, UT: Family Development Resources Inc.
Beardslee W. R., Gladstone T. R., Wright E. J., Cooper A. B. (2003). A family-based approach to the prevention of depressive symptoms in children at risk: Evidence of parental and child
change. Pediatrics, 112(2), e119–e131.
Beck A. T., Steer R. A., Brown G. K. (1996). Beck Depression Inventory–II. San Antonio, TX: Psychological Corporation.
Bendell D., Field T., Yando R., Lang C., Martinez A., Pickens J. (1994). “Depressed” mothers' perceptions of their preschool children's vulnerability. Child
Psychiatry and Human Development, 24(3), 183–190.
Bendell D., Stone W., Field T., Goldstein S. (1989). Children's effects on parenting stress in a low income
, minority population. Topics in Early Childhood Special Education, 8(4), 58–71.
Bergen D. (2002). The role of pretend play in children's cognitive development. Early Childhood Research and Practice, 4(1). Retrieved May 31, 2017, from http://ecrp.uiuc.edu/v4n1/bergen.html
Bierman K. L., Nix R. L., Greenberg M. T., Blair C., Domitrovich C. E. (2008). Executive functions and school readiness intervention: Impact, moderation, and mediation in the Head Start REDI program. Developmental
Psychopathology, 20, 821–843.
Blevins C. A., Weathers F. W., Davis M. T., Witte T. K., Domino J. L. (2015). The Posttraumatic Stress Disorder Checklist for DSM-5 (PCL-5): Development and initial psychometric evaluation. Journal of Traumatic Stress, 28(6), 489–498.
Blodgett C. (2014). ACEs in Head Start children and impact on development. Pullman, WA: CLEAR Trauma Center, Washington State University.
Blumberg S. J., Bialotosky K., Hamilton W. L., Briefel R. (1999). The effectiveness of a short form of the Household Food Security Scale. American Journal of Public Health, 89(8), 1231–1234.
Bradley R. H., Corwyn R. F. (2002). Socioeconomic status and child
development. Annual Review of Psychology, 53(1), 371–399.
Briere J. (2011). Trauma Symptom Inventory–2 (TSI-2). Odessa, FL: Psychological Assessment Resources.
Brigance A. H., French B. F. (2013). Brigance Early Childhood Screen III, 0-35 months. North Billerica, MA: Curriculum Associates.
Bronfenbrenner U., Morris P. (1998). The ecology of developmental
processes. In Damon W. (Ed.), Handbook of child
psychology, 1: Theoretical models of human development (5th ed., pp. 993–1023). New York, NY: Wiley.
Buckner J. C., Bassuk E. L., Weinreb L. F. (2001). Predictors of academic achievement among homeless and low-income housed children. Journal of School Psychology, 39, 45–69.
Buckner J. C., Mezzacappa E., Beardslee W. R. (2009). Self-regulation and its relations to adaptive functioning in low income
youths. American Journal of Orthopsychiatry, 79(1), 19–30.
Bureau of Labor Statistics, U.S. Department of Labor. (2012). Table 1.2: Employment by detailed occupation, 2012 and projected 2022. Employment Projections program. Retrieved from http://www.bls.gov/emp/ep_table_102.htm
Bureau of Labor Statistics, U.S. Department of Labor. (2014a). Childcare workers. In Occupational outlook handbook (2014-15 ed.). Retrieved from http://www.bls.gov/ooh/personal-care-and-service/childcareworkers.htm
Bureau of Labor Statistics, U.S. Department of Labor. (2014b). Preschool teachers. In Occupational outlook handbook (2014-15 ed.). Retrieved from http://www.bls.gov/ooh/education-training-andlibrary/preschool-teachers.htm
Bureau of Labor Statistics, U.S. Department of Labor. (2014c). Social and human service assistants. In Occupational outlook handbook (2014-15 ed.). Retrieved from http://www.bls.gov/ooh/community-andsocial-service/social-and-human-service-assistants.htm
Cairney J., Boyle M., Offord D. R., Racine Y. (2003). Stress, social support and depression in single and married mothers. Social Psychiatry and Psychiatric Epidemiology, 38(3), 442–449.
Center on the Developing Child
at Harvard University. (2007). The science of early childhood development. Cambridge, MA: National Scientific Council on the Developing Child
. Retrieved from http://developingchild.harvard.edu/resources/reports_and_working_papers/science_of_early_childhood_development
Center on the Developing Child
at Harvard University. (2009). Maternal depression can undermine the development of young children (Working Paper No. 8). Cambridge, MA: Harvard University. Retrieved from http://www.developingchild.harvard.edu
Center on the Developing Child
at Harvard University. (2010). The Foundations of lifelong health are built in early childhood. Retrieved from http://www.developingchild.harvard.edu
Center on the Developing Child
at Harvard University. (2016). From best practices to breakthrough impacts: A science-based approach to building a more promising future for young children and families. Cambridge, MA: Author.
Chapman D. P., Whitfield C. L., Felitti V. J., Dube S. R., Edwards V. J., Anda R. F. (2004). Adverse childhood experiences and the risk of depressive disorders in adulthood. Journal of Affective Disorders, 82(2), 217–225.
Children's Defense Fund. (2017). The state of America's children 2017. Washington, DC: Author.
Cicchetti D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6(4), 284–290.
Council on Children with Disabilities, Section on Developmental
Behavioral Pediatrics, Bright Futures Steering Committee, Medical Home Initiatives for Children with Special Needs Project Advisory Committee. (2006). Identifying infants and young children with developmental
disorders in the medical home: An algorithm for developmental
surveillance and screening. Pediatrics, 118, 405–420.
Cronholm P. F., Forke C. M., Wade R., Bair-Merritt M. H., Davis M., Harkins-Schwarz M., Fein J. A. (2015). Adverse childhood experiences: Expanding the concept of adversity. American Journal of Preventive Medicine, 49(3), 354–361.
Cutuli J. J., Herbers J. E. (2014). Promoting resilience for children experience family homelessness: Opportunities to encourage developmental
competence. Cityscape, 16, 113–139.
Cutuli J. J., Hebers J. E. (2019). Housing interventions and the chronic and acute risks of family homelessness: Experimental evidence for education. Child
Development, 90(5), 1664–1683.
De Jong T. (2009). Cognitive load theory, educational research, and instructional design: Some food for thought. Instructional Science, 38(2), 105–134.
DeCandia C. J., Bassuk E. L., Richard M. (2017). Assessment of families experiencing homelessness: Analysis of current practice. In Haskett M. E. (Ed.), Child
and family well-being and homelessness (pp. 49–63). Cham, Switzerland: Springer.
Dozois D. J., Covin R. (2004). The Beck Depression Inventory-II (BDI-II), Beck Hopelessness Scale (BHS), and Beck Scale for Suicide Ideation (BSS). In Hersen M. (Series Ed.), Segal D. L., Hilsenroth M. (Vol. Eds.), Comprehensive handbook of psychological assessment: Vol 2. Personality assessment and psychopathology (pp. 50–69). Hoboken, NJ: Wiley.
Dube S. R., Anda R. F., Felitti V. J., Chapman D. P., Williamson D. F., Giles W. H. (2001). Childhood abuse, household dysfunction, and the risk of attempted suicide throughout the life span: Findings from the Adverse Childhood Experiences Study. JAMA, 286(24), 3089–3096.
Eamon M. K. (2001). The effects of poverty on children's socioemotional development: An ecological
systems analysis. Social Work, 46(3), 256–266.
Egger H., Angold A. (2004). Stressful life events and PTSD in preschool children. Paper presented at the annual meeting of the American Academy of Child
and Adolescent Psychiatry, Washington, DC.
Essex M. J., Klein M. H., Cho E., Kalin N. H. (2002). Maternal stress beginning in infancy may sensitize children to later stress exposure: Effects on cortisol and behavior. Biological Psychiatry, 52(8), 776–784.
Fantuzzo J. W., LeBoeuf W. A., Chen C. C., Rouse H. L., Culhane D. P. (2012). The unique and combined effects of homelessness and school mobility on the educational outcomes of young children. Educational Researcher, 41(9), 393–402.
Fantuzzo J., LeBoeuf W., Brumely B., Perlman S. (2013). A population-based inquiry of homeless episode characteristics and early education wellbeing. Children and Youth Services Review, 35, 966–972.
Fantuzzo J., Perlman S. (2007). The unique impact of out-of-home placement and the mediating effects of child
maltreatment and homelessness on early school success. Children and Youth Services Review, 29(7), 941–960. https://doi.org/10.1016/j.childyouth.2006.11.003
Felitti M. D., Anda R. F., Nordenberg D., Williamson D. F., Spitz A. M., Edwards V., Marks J. S. (1998). Relationship of childhood abuse and household dysfunction to many of the leading causes of death in adults. American Journal of Preventive Medicine, 14(4), 245–258.
Findling J. H., Bratton S. C., Henson R. K. (2006). Development of the Trauma Play Scale: An observation-based assessment of the impact of trauma on play therapy behaviors of young children. International Journal of Play Therapy, 15(1), 7–36.
Finkelhor D., Turner H. A., Ormrod R. K., Hamby S. L. (2009). Violence, crime, and exposure in a national sample of children and youth. Pediatrics, 124(5), 1–13.
Finkelhor D., Turner H. A., Shattuck A., Hamby S. L. (2013). Violence, crime, and abuse exposure in a national sample of children and youth: An update. JAMA Pediatrics, 167(7), 614–621.
Foy J. M., Kelleher K. J., Laraque D., & American Academy of Pediatrics Task Force on Mental Health. (2010). Enhancing pediatric mental health care: Strategies for preparing a primary care practice. Pediatrics, 125(3), S87–S108.
Friedman D. H. (2000). Parenting in public: Family shelter and public assistance. New York, NY: Columbia University Press.
Garg A. X., Adhikari N. K., McDonald H., Rosas-Arellano M. P., Devereaux P. J., Beyene J., Haynes R. B. (2005). Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: A systematic review. JAMA, 293(10), 1223–1238.
Gerhardt S. (2004). Why love matters: How affection shapes a baby's brain. New York, NY: Routledge.
Gewirtz A., Taylor T. (2009). Participation of homeless and abused women in a parent training program: Science and practice converge in a battered women's shelter. In Community Participation and Empowerment (pp. 97–114). Hauppage, NY: Nova Science Publishers.
Ginsburg K. R. (2007). The importance of play in promoting healthy child
development and maintaining strong parent–child
bonds. Pediatrics, 119(1), 182–191.
Glascoe F. P. (1997). Parents' concerns about children's development: Prescreening technique or screening test? Pediatrics, 99(4), 522–528.
Glascoe F. P. (2008). Parents' Evaluation of Developmental
Status: Guide to using PEDS. Nolensville, TN: PEDStest.com/LLC.
Glascoe F. P. (2010). Technical report for the BRIGANCE® Screens. North Billerica, MA: Curriculum Associates.
Grant J. S., Davis L. L. (1997). Selection and use of content experts for instrument development. Research in Nursing & Health, 20(3), 269–274.
Grant R., Brito A. (2010). Chronic illness and school performance: A literature review focusing on asthma and mental health conditions. New York, NY: Children's Health Fund.
Guerro A. D., Garro N., Chang J. T., Kuo A. A. (2010). An update on assessing development in the pediatric office: Has anything changed after two policy statements? American Pediatrics, 10(6), 400–404.
Halfon N., Larson K., Son J., Lu M., Bethell C. (2017). Income inequality and the differential effect of adverse childhood experiences in US children. Academic Pediatrics, 17(7), S70–S78.
Haskett M. E., Armstrong J., Tisdale J. (2015). Developmental
status and social-emotional functioning of young children experiencing homelessness. Early Childhood Education Journal, 44, 119–125.
Hayes M., Zonneville M., Bassuk E. (2013). The SHIFT Study final report: Service and housing interventions for families in transition. Newton, MA: National Center on Family Homelessness.
Helms J. E. (2006) Fairness is not validity or cultural bias in racial-group assessment: A quantitative perspective. American Psychologist, 61(8), 845–859.
Hollender N., Hofmann C., Deneke M., Schmitz B. (2010). Integrating cognitive load theory and concepts of human–computer interaction. Computers in Human Behavior, 26(6), 1278–1288.
Hostinar C. E., Gunnar M. R. (2015). Social support can buffer against stress and brain activity. AJOB Neuroscience, 6(3), 34–42.
Hoswarth J. (Ed.). (2010). The child
's world: The comprehensive guide to assessing children in need. Philadelphia, PA: Jessica Kingsley Publishers.
Institute for Children, Poverty & Homelessness. (2017). On the map: The Atlas of student homelessness in New York City 2017. Retrieved from http://www.icphusa.org/new_york_city/map-atlas-student-homelessness-new-york-city-2017
Jiang Y., Ekono M., Skinner C. (2016). Basic facts about low-income children: Children under 18 years, 2014. New York, NY: National Center for Children in Poverty, Columbia University Mailman School of Public Health.
Jiang Y., Koball H. (2018). Basic facts about low-income children: Children under 18 years, 2016. New York, NY: National Center for Children in Poverty, Columbia University Mailman School of Public Health.
Johnson C. P., Myers S. M., & American Academy of Pediatrics Council on Children With Disabilities. (2007). Identification and evaluation of children with autism spectrum disorders. Pediatrics, 120, 1183–1215.
Kamphaus R. W., Reynolds C. R. (2015). Behavior Assessment System for Children—Third Edition (BASC-3): Behavioral and Emotional Screening System (BESS). Bloomington, MN: Pearson.
Kazdin A. E. (2010). Problem-solving skills training and parent management training for oppositional defiant disorder and conduct disorder. In Weisz J. R., Kazdin A. E. (Eds.), Evidence-based psychotherapies for children and adolescents (2nd ed., pp. 211–226). New York, NY: Guilford.
Kazdin A. E., Whitley M., Marciano P. L. (2006). Child
-therapist and parent-therapist alliance and therapeutic change in the treatment of children referred for oppositional, aggressive, and antisocial behavior. Journal of Child
Psychology and Psychiatry, 47(5), 436–445.
Kelley S. D., Bickman L. (2009). Beyond outcomes monitoring: Measurement feedback systems in child
and adolescent clinical practice. Current Opinion in Psychiatry, 22(4), 363–368.
Kroenke K., Spitzer R. L. (2002). The PHQ-9: A new depression diagnostic and severity measure. Psychiatric Annals, 32(9), 509–515.
Kroenke K., Spitzer R. L., Williams J. B. (2001). The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine, 16(9), 606–613.
Kwok M. A., Eugene Arnold L., Barterian J., Pan X., Scahill L., Aman M., McDougle C. (2017). Evaluating the relationship between the Vineland Adaptive Behavior Scale and the assessment of basic language and learning skills-modified. Journal of the American Academy of Child
& Adolescent Psychiatry, 56(10), S255–S256.
Lifter K., Foster-Sanda S., Arzamarski C., Briesch J., McClure E. (2011). Overview of play: Its uses and importance in early intervention/early childhood special education. Infants & Young Children, 24(3), 225–245.
Lopresti A. L., Hood S. D., Drummond P. D. (2013). A review of lifestyle factors that contribute to important pathways associated with major depression: Diet, sleep and exercise. Journal of Affective Disorders, 148(1), 12–27.
Lorber M. F., Xu S., Smith Slep A. M., Bulling L., O'Leary S. G. (2014). A new look at the psychometrics of the Parenting Scale through the lens of item response theory. Journal of Clinical Child
and Adolescent Psychology, 43(4), 613–626.
Luckey A. J., Fabes R. A. (2006). Understanding nonsocial play in early childhood. Early Childhood Education Journal, 33(2), 67–72.
Lupien S. J., McEwen B. S., Gunnar M. R., Heim C. (2009). Effects of stress throughout the lifespan on the brain, behaviour and cognition: Nature reviews. Neuroscience, 10(6), 434–445.
Mackrides P. S., Ryherd S. J. (2011). Screening for developmental
delay. American Family Physician, 84(5), 544–549.
Marks K. P., Glascoe F. P., Macias M. M. (2011). Enhancing the algorithm for developmental
–behavioral surveillance and screening in children 0 to 5 years. Clinical Pediatrics, 50(9), 853–868.
Mash E. J., Johnston C. (1983). Parental perceptions of child
behavior problems, parenting self-esteem, and mothers' reported stress in younger and older hyperactive and normal children. Journal of Consulting and Clinical Psychology, 51(1), 86–99.
Masten A. S. (2001). Ordinary magic: Resilience processes in development. American Psychologist, 56(3), 227–238.
McLean D. E., Bowen S., Drezner K., Rowe A., Sherman P., Schroeder S., Redlener I. (2004). Asthma among homeless children: Undercounting and undertreating the underserved. Archives of Pediatrics & Adolescent Medicine, 158(3), 244–249.
Meyer G. J., Finn S. E., Eyde L. D., Kay G. G., Moreland K. L., Dies R. R., Reed G. M. (2001). Psychological testing and psychological assessment: A review of evidence and issues. American Psychologist, 56(2), 128–165.
Mintz S. (2004). Huck's raft: A history of American childhood. Cambridge, MA: Belknap.
Moodie S., Daneri P., Goldhagen S., Halle T., Green K., LaMonte L. (2014). Early childhood developmental
screening: A compendium of measures for children ages birth to five (OPRE Report 201411). Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
Moore J. (2013). Research Summary: Resilience and at-risk children and youth. Browns Summit, NC: National Center for Homeless Education at SERVE. Retrieved from www.serve.org/nche
Moore L. V., Diez Roux A. V., Evenson K. R., McGinn A. P., Brines S. J. (2008). Availability of recreational resources in minority and low socioeconomic status areas. American Journal of Preventive Medicine, 34(1), 16–22.
Morello-Frosch R., Zuk M., Jerrett M., Shamasunder B., Kyle A. D. (2011). Understanding the cumulative impacts of inequalities in environmental health: Implications for policy. Health Affairs, 30(5), 879–887.
Morsbach S. K., Prinz R. J. (2006). Understanding and improving the validity of self-report of parenting. Clinical Child
and Family Psychology Review, 9(1), 1–21.
Mowbray C., Oyserman D., Bybee D., MacFarlane P. (2002). Parenting of mothers with a serious mental illness: Differential effects of diagnosis, clinical history, and other mental health variables. Social Work Research, 26(4), 225–240.
Mullen J., Leginski W. (2010). Building the capacity of the homeless service workforce. Open Health Services and Policy Journal, 3, 101–110.
Nandi A., Sweet E., Kawachi I., Galea S. (2014). Associations between macro level economic factors and weight distribution in low- and middle-income countries: A multi-level analysis of 200,000 adults in 40 countries. American Journal of Public Health, 104(2), 162–171.
National Research Council & Institute of Medicine. (2009). Preventing mental, emotional, and behavioral disorders among young people: Progress and possibilities. In O'Connell M. E., Boat T., Warner K. E. (Eds.); Committee on Prevention of Mental Disorders and Substance Abuse Among Children, Youth, and Young Adults. Research advances and promising interventions. Washington, DC: National Academies Press.
National Scientific Council on the Developing Child
. (2004). Young children develop in an environment of relationships (Working Paper No. 1, pp. 1–8). Cambridge, MA: National Scientific Council on the Developing Child
, Center on the Developing Child
at Harvard University.
National Scientific Council on the Developing Child
. (2012). The science of neglect: The persistent absence of responsive care disrupts the developing brain (Working Paper No. 12). Cambridge, MA: National Scientific Council on the Developing Child
, Center on the Developing Child
at Harvard University.
National Scientific Council on the Developing Child
. (2015). Supportive relationships and active skill-building strengthen the foundations of resilience (Working Paper No. 13). Cambridge, MA: National Scientific Council on the Developing Child
, Center on the Developing Child
at Harvard University.
Obradović J., Long J. D., Cutuli J. J., Chan C. K., Hinz E., Heistad D., Masten A. S. (2009). Academic achievement of homeless and highly mobile children in an urban school district: Longitudinal evidence on risk, growth, and resilience. Development and Psychopathology, 21(2), 493–518.
Ohio Department of Education. (2010). The catalog of screening and assessment instruments for young children birth through age 5, second edition. Columbus, OH: Ohio Department of Education, Office of Early Learning and School Readiness.
OrgCode. (2013). Service prioritization decision assistance tool for families (F-SPDAT). Retrieved from http://org-beehivegroupcadev.nationbuilder.com/spdat
Pachter L. M., Auinger P., Palmer R., Weitzman M. (2006). Do parenting and the home environment, maternal depression, neighborhood, and chronic poverty affect child
behavioral problems differently in different racial-ethnic groups? Pediatrics, 117(4), 1329–1338.
Peña E. D., Halle T. G. (2011). Assessing preschool dual language learners: Traveling a multiforked road. Child
Development Perspectives, 5(1), 28–32.
Perlman S., Cowan B., Gewirtz A., Haskett M., Stokes L. (2012). Promoting positive parenting in the context of homelessness. American Journal of Orthopsychiatry, 82(3), 402–412.
Perlman S., Fantuzzo J. (2010). Timing and influence of early experiences of child
maltreatment and homelessness on children's educational wellbeing. Children and Youth Services Review, 32, 874–883.
Reitman D., Currier R. O., Hupp S. D. A., Rhode P. C., Murphy M. A., O'Callaghan P. M. (2010). Psychometric characteristics of the Parenting Scale in a Head Start population. Journal of Clinical Child
& Adolescent Psychology, 30(4), 514–524.
Rhoades K. A., O'Leary S. G. (2007). Factor structure and validity of the Parenting Scale. Journal of Clinical Child
and Adolescent Psychology, 36(2), 137–146.
Ringwalt S. (2008). Developmental
screening and assessment instruments with an emphasis on social and emotional development for young children ages birth through five. Chapel Hill, NC: The University of North Carolina, FPG Child
Development Institute, National Early Childhood Technical Assistance Center.
Rog D. J., Buckner J. C. (2007). Homeless families and children. In Dennis D., Locke G., Khadduri J. (Eds.), Toward understanding homelessness: The 2007 National Symposium on Homelessness Research (Section 5.1). Cambridge, MA: Abt. Associates.
Rouse H. L., Fantuzzo J. W. (2009). Multiple risks and educational wellbeing: A population-based investigation of threats to early school success. Early Childhood Research Quarterly, 24(1), 1–14.
Ryan V. (1999). Developmental
delay, symbolic play, and non-directive play therapy. Clinical Child
Psychology and Psychiatry, 4(2), 167–185.
Sacks V., Murphey D., Moore K. (2014). Adverse childhood experiences: National and state level prevalence. Child
Trends Research Brief. Retrieved from https://www.childtrends.org/wp-content/uploads/2014/07/Brief-adverse-childhood-experiences_FINAL.pdf
Schore J. R., Schore A. N. (2008). Modern attachment theory: The central role of affect regulation in development and treatment. Clinical Social Work Journal, 36, 9–20.
Shinn M., Schteingart J. S., Williams N. C., Carlin-Mathis J., Bialo-Karagis N., Becker-Klein R., Weitzman B. C. (2008). Long-term associations of homelessness with children's well-being. American Behavioral Scientist, 51(6), 789–809.
Shonkoff J. P. (2010). Building a new biodevelopmental framework to guide the future of early childhood policy. Child
Development, 81(1), 357–367.
Shonkoff J. P., Phillips D. A. (Eds.); Committee on Integrating the Science of Early Childhood Development, Board on Children, Youth, and Families, Commission on Behavioral and Social Sciences and Education. (2000). From neurons to neighborhoods: The science of early childhood development. Washington, DC: National Academies Press.
Sparrow S. S., Balla D. A., Cicchetti D. V. (2005). Vineland-II: Vineland Adaptive Behavior Scales: Survey forms manual. Circle Pines, MN: American Guidance Service.
Sparrow S. S., Cicchetti D. V., Saulnier C. A. (2016). Vineland-3: Vineland Adaptive Behavior Scales, 3rd ed. Minneapolis, MN: Pearson Assessments.
Spiker D., Hebbeler K. M., Barton L. R. (2011). Measuring quality of ECE programs for children with disabilities. In Measuring Quality in Early Childhood Settings (pp. 229–256). Baltimore MD: Brookes Publishing.
Squires J., Bricker D. (2009). Ages & Stages Questionnaires, Third edition (ASQ-3). Baltimore, MD: Brookes Publishing.
Steele R. G., Nesbitt-Daly J. S., Daniel R. C., Forehand R. (2005). Factor structure of the Parenting Scale in a low-income African American sample. Journal of Child
and Family Studies, 14(4), 535–549.
Sweller J., Van Merriënboer J., Paas F. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10(3), 251–296.
The United States Conference of Mayors. (2016). A status report on hunger and homelessness in America's cities, December 2016. Washington, DC: Author.
Tucker E. (2010). Towards a more rigorous scientific approach to social measurement: Considering a grounded indicator approach. In The Sage Handbook of Measurement (p. 313). London, UK: Sage Publications.
U.S. Census Bureau. (2017). 2016 Annual social and economic supplement, Tables POV01, POV03, POV13, POV21, POV40, and 3. Retrieved from https://www.census.gov/programs-surveys/cps.html
U.S. Department of Health and Human Services. (2014). Birth to 5: Watch me thrive! A compendium of screening measures for young children. Washington, DC: Author. Retrieved from https://www.acf.hhs.gov/sites/default/files/ecd/screening_compendium_march2014.pdf
Vogt D. S., Shepherd J. C., Resick P. A. (2012). Posttraumatic Maladaptive Beliefs Scale: Evolution of the personal beliefs and reactions scale. Assessment, 19, 308–317.
Walsh R. (2011). Lifestyle and mental health. American Psychologist, 66(7), 579–592.
Weathers F. W., Litz B. T., Keane T. M., Palmieri P. A., Marx B. P., Schnurr P. P. (2013). The PTSD Checklist for DSM-5 (PCL-5). Retrieved from www.ptsd.va.gov
Weinreb L. F., Buckner J. C., Williams V., Nicholson J. (2006). A comparison of the health and mental health status of homeless mothers in Worcester, Massachusetts: 1993 and 2003. American Journal of Public Health, 96(8), 1444–1448.
WestEd Center for Prevention & Early Intervention. (2015). Developmental
and behavioral screening guide for early care and education providers. San Francisco, CA: WestEd.
Williams D. R., Neighbors H. W., Jackson J. S. (2003). Racial/ethnic discrimination and health: Findings from community studies. American Journal of Public Health, 93(2), 200–208.
Willis G. B. (2005). Cognitive interviewing: A tool for improving questionnaire design. Thousand Oaks, CA: Sage.
Yoshikawa H., Aber J. L., Beardslee W. R. (2012). The effects of poverty on the mental, emotional, and behavioral health of children and youth: Implications for prevention. American Psychologist, 67(4), 272–284.