Early intervention (EI) is a comprehensive service provided for children with developmental delay or for those at a high risk of developmental delay, to improve their cognition, emotion, behaviour, and ability to adapt to the society. It implies the active participation of families in the intervention process,[2–4] although the family has generally been considered as a resource rather than as part of a global intervention objective. However, the current perspective revolves around consideration of the family as a fundamental axis in child health care, and specifically in EI services, advocating that the family is the expert on its children and makes decisions according to their needs and characteristics, supported by the professional in the process of attending to the child.[6,7] This change is reflected in the objectives of new research, where professionals are the means of ensuring that children and families continue to benefit from the progress made in this area. According to McCollum and Shonkoff, EI consists of multidisciplinary services to promote child health and wellbeing, enhance emerging competencies, minimize developmental delays, remediate existing or emerging disabilities, prevent functional deterioration, and promote adaptive parenting and overall family functioning.
Currently in Andalusia, the geographical area in the South of Spain where the study sample was obtained, EI programs are provided in EI centers (EIC), where a multidisciplinary team provides services aimed at serving children with developmental disorders, or at risk, as early as possible, from birth to 6 years of age.
Quality assessment is an essential aspect of health organizations at present, from the point of view of both providers and service users. Incorporation of the opinion of the user into the functioning of a health system is not something new.[12,13] From the users’ perspective, satisfaction analysis addresses their cognitive assessment of and emotional reaction toward their experience with health services. Most health service satisfaction assessment proposals are based on marketing theories, identifying the differences between expectations and perception of service.[15,16]
Parasuraman et al define perceived quality as the difference between expectations and consumer perceptions with regard to a service. The authors defined 5 aspects of any service that influence the quality, and popularized SERVQUAL. Other authors suggest that, depending on the type of business, the dimensions of service quality may be considered to be different.[18,19]
However, organizational systems that manage quality often operate simply on the basis of rational measurement, without regard for intangible or relational dimensions. Management models must take into account experiential aspects to provide more stable theoretical building blocks that help to understand and manage quality in daily clinical practice.[20,21] These aspects are particularly relevant in EI, given the characteristics of children with a developmental disorder, since their progress can be seriously compromised. In this sense, families, as external customers of the EIC, are considered an indispensable element of care and quality improvement programs.[22–26]
From this perspective, specific tools have been developed to measure different aspects of EI, including the Early Intervention Services Assessment Scale (EISAS), the 2 scales proposed by Fisher et al, or the Systematic Exploration and Process Inventory for health professionals in early childhood intervention services.
After a review of the scientific literature, a lack of service quality studies in EI is detected. It is necessary to propose improvements in services based on measurements performed using validated instruments.[28,29] In this context, and given that work on perceived quality in EI is scarce, an evaluation tool that allows collection of relevant aspects about the service quality provided in the EIC is missing. Therefore, the IQEIC was proposed in order to gather families’ perceptions as users of these centers. The gathered data can be a key element in evaluation and subsequent decision-making regarding quality improvement activities. Therefore, the main objective of this research is the validation of a short scale to measure the quality perceived by users/families at the EIC.
The population was composed of families of children between 0 and 6 years old who received EI for at least 1 month in an EIC in Andalusia (southern Spain). A margin of error of 5% was used and a confidence level of 99% (Z = 1.96) was assumed for the selection of the sample. Finally, a total of 887 families participated in the study, 73.1% (648) being female and 23.7% (210) male, with ages ranging from 20 to over than 70 years, most often in the age range 30 to 39 years (n = 498; 56.1%). In terms of kinship, mothers represented 70.7% (n = 628) and fathers 21.7% (n = 193) (Table 1).
We used the Inventory of Quality in Early Intervention Centers, constructed from the Questionnaire of Perceived Quality in Early Intervention Physiotherapy and the Satisfaction of the Client Family Questionnaire. This instrument is made up initially of 48 items and 6 dimensions: center facilities (13 items), treatment rooms and materials (9 items), user services (7 items), qualified staff (8 items), general information (5 items), and technical or specific information (6 items). The response format for all items is a 5-point Likert scale rated from 1 (strongly disagree) to 5 (strongly agree). In the present study, the general information dimension was removed due to the collection of anachronistic information in the current process of referral to de EIC.
In the original study, the dimensions showed good reliability (α values higher than 0.70, with the exception of the general information dimension), the lower value justified by the small number of items of the scale. Exploratory factor analysis was performed independently for each dimension and offered 10 factors, showing a Kaiser–Meyer–Olkin (KMO) index >0.80 and a total variance explained above 50%, with the exceptions of the user services and general information dimensions.
The researchers contacted 21 EICs located in the province of Malaga (Andalusia, Spain), offering one meeting with managers who were interested, with the purpose of explaining the objectives and the methodology of the study, the questionnaire and the data collection. Data were collected using the IQEIC, including a demographic characteristics section. The evaluators attended 2 training sessions prior to the start of the assessment period and visited the EIC every day over one month to deliver the questionnaires to participants, and they would read the different items so that the participants could mark the answer in the questionnaire, thus avoiding unanswered questions. Participants provided informed consent after reading the description of the study in accordance with the Declaration of Helsinki, and the anonymity of confidentiality of their responses was assured. Each participant took ten to fifteen minutes to complete the questionnaire. Permission to undertake this study was gained from the Research Ethics Committee at Malaga University (code 32-2017-H).
2.4 Data analysis
A cross-validation analysis was performed, a usual procedure in psychometric studies. To do this, the dimensionality of the ICCAIT was analyzed, splitting the sample into 2 random subsamples.[32–34] With the first half (n 1 = 440; 310 female: 70.5%; 104 male: 23.6%; 26 did not disclose: 5.9%), an exploratory factor analysis (EFA) was conducted utilizing principal components extraction and oblique rotation was applied to interpret the obtained factors. In order to analyze the factor loadings of every item, 0.40 was taken as the recommended cut-off point.[35,36] We had previously tested the factorization conditions using the Bartlett and KMO tests. The second subsample (n 2 = 447; 338 female: 75.6%; 106 male: 23.7%; 3 did not disclose: 0.7%) was analyzed using confirmatory factor analysis (CFA) to test the adjustment of the structure obtained in the first half, using the maximum likelihood estimation (ML) method. The fit indices utilized were as follows: the comparative fit index (CFI), Tucker–Lewis index (TLI), and incremental fit index (IFI),[37,38] where values above 0.90 indicate good fit. With root mean square error of approximation (RMSEA), values equal or lower to 0.06 indicate a well-fitting model, while with the χ2/degree of freedom ratio, values below 3.0 are considered a good fit. Factor loadings of 0.50 and above, with significant p-values, and modification indices (MI), were used to locate any problematic items that contributed to misfit to the data.
In addition, the internal consistency of the constructs was measured through composite reliability (CR). Convergent validity was evaluated through the average variance extracted (AVE), while discriminant validity was established when AVE for each construct exceeded the squared correlations between that construct and any other.
Finally, a multi-group analysis was performed to test the difference between 2 invariance models.We do not rely on the χ2, as it is judged to be too restrictive; instead we rely on the change in CFI value (ΔCFI), which has to be lower than 0.01 so that the hypothesis of invariance is not rejected. The Chi-square difference test was also reported for the comparison. Data were analyzed with Statistical Product and Service Solutions 20.0 (SPSS 20.0) and Amos Graphics statistical software 20.0.
3.1 Exploratory factor analysis
The results for the EFA were adequate in the first subsample (n 1 = 440). KMO measure (KMO = 0.94) and significance of Bartlett test (χ2 = 11477.23; df = 903; P < .001) revealed that the items had adequate common variance for factor analysis. Then, principal components analysis using oblimin oblique rotation yielded 8 factors explaining 64.08% of common variance across items. Reliability coefficients for all the obtained factors were higher than 0.70, with values between 0.77 and 0.91, the global scale being at 0.94.
3.2 Confirmatory factor analysis
We used the other half of the sample (n2 = 447) to conduct CFA and cross-validate model obtained in the EFA. The adjustment showed a poor fit to the data in the 8-factor model with 43 items and 5 dimensions [χ2(794) = 2052.59; χ2/gl = 2.58; RMSEA = 0.060 (CI = 0.057–0.063); CFI = 0.89; TLI = 0.87; IFI = 0.89]. Although the RMSEA value was indicative of a good fit, the CFI, TLI and IFI values were below the threshold of 0.90. These results indicated that the model should be adjusted. For this purpose, a revision was made based on the factor loadings and modification indices. After selecting the items whose factor loading was greater than 0.50, items 3, 4, 9, and 13 (center facilities) were deleted, as well as items 24 to 29 (user services). Item 23 (“the attention that the user receives in the center is adequate”) of the customer care dimension moved to the qualified staff dimension. Finally, several items showed high modification indexes (MI) with different factors, so we proceeded to eliminate items 7 (MI = 70.60), 8 (MI = 134.82), 10 (MI = 17.45), 16 (MI = 44.55), 17 (MI = 23.01), 19 (MI = 24.29), and 34 (MI = 46.80). The result was a scale composed of 4 dimensions and 26 items with an acceptable fit to the data in confirmatory factor analysis [χ2(263) = 667.49; χ2/gl = 2.53; RMSEA = 0.059 (CI = 0.053–0.064); CFI = 0.94; TLI = 0.92; IFI = 0.94]. According to these results, the modified model had an excellent fit. All items showed high factor loadings, ranging from 0.53 to 0.93, providing evidence that each item appropriately captured its respective factor (Table 2).
Internal consistency was assumed for all constructs, with composite reliability values greater than value 0.80, specifically ranging from 0.84 to 0.90. The AVE and squared correlation tests for discriminant validity are reported in Table 3. The AVE values were adequate and the squared correlation values ranged from 0.12 to 0.50, indicating discriminant validity in all constructs.
3.3 Multigroup validation
A multi-group CFA analysis was conducted with the testing sample (n 1 = 440) and the validation sample (n 2 = 447). The fit of the unconstrained model was acceptable [χ2(526) = 1330.08; χ2/gl = 2.53; RMSEA = 0.04; CFI = 0.94; TLI = 0.92; IFI = 0.94], and for the model with constrained factor loadings [χ2(543) = 1385.05; χ2/gl = 2.55; RMSEA = 0.04; CFI = 0.93; TLI = 0.92; IFI = 0.94]. As the CFI change between the models was lower than 0.01, we could assume factor loading invariance between the 2 groups.
To measure quality of service is not an easy task, and sociocultural variables surrounding participants should be taken into account during its measurement, since these subjects may have different interpretations of what quality means. As a consequence, it should be remembered that assessing the quality of the service in different areas and taking a unique scale or universal questionnaire as a reference is not possible.
The purpose of the current study was to validate the reduced version of the Inventory of Quality in Early Intervention Centers, a questionnaire specifically designed to measure the quality perceived by families served in EICs. This questionnaire can be used in any EIC, regardless of the number of professionals or disciplines that make up the model, this feature being one of the advantages of the instrument above others, as proposed by Medina et al. Similarly, the Systematic Exploration and Process Inventory for health professionals in early childhood intervention services only collects information from the EIC professionals and evaluates not the quality of service but the needs and resources of families attending EI services. Also, the EISAS, presented as an instrument that measures the quality of the practice of general EI services, has not yet been validated. On the other hand, Fisher et al proposed 2 scales which collect information from parents and which evaluate only intervention programs in EI, one of them with 135 items. Even so, these did not collect other essential aspects of the service, such as the intangible elements.
The relevance of measuring perceived quality in EI services is that it allows identification of opportunities for improvement. Overestimation of the quality of care in an EIC can perpetuate a suboptimal performance, while exact measurements of the current action and a realistic comparison with other more successful centers could provide the motivation to improve the quality, as stated by Palazón-Cabanes,et al.
According to the results obtained, it can be concluded that the IQEIC-R is a pragmatic and reliable tool that measures the perceived quality of service in EICs, encompassing both the healthcare and nonhealthcare processes provided in the EIC, since it covers all aspects of the service from the family arriving at the EIC to the child leaving. Moreover, the tool has the advantage of a small number of items: it can therefore be answered in a short time, estimated to be around 10 minutes.
Assessment of perceived quality is a complex, multifaceted area of study, and collecting information about satisfaction with services is routine within service systems. Since researchers usually do not report or share the study’ s results with their participants, in future research we propose to learn what the collected information means, how it is used by providers, and how families might use it to advocate changes to existing services and support, which could be empowering. This could benefit both the consumer and the provider agency, as stated by Copeland et al, Jemes-Campaña et al.
One of the possible limitations of the study is the lack of evaluation of sensitivity to changing the instrument, although this was not the aim and the design does not allow it. It is considered that the administration of this questionnaire on several occasions is required in order to assess the response to changes in the centers to improve the quality of service offered. Also, the factor structure should be checked in other health care contexts where EI is organized within different parameters for example, in other European countries or even in the United States of America. An analysis of factorial invariability is not conducted according to sociodemographic characteristics such as age or gender, which could cause changes in perception of the perceived quality of the construct. In this sense, and regarding future lines of research, a factor invariance analysis could be of interest, given potentially different interpretations in different cultural and social contexts. As it stands, this type of study is in its beginning stages, affording numerous investigative opportunities.
In conclusion, this study demonstrates that the short version of the IQEIC is a reliable, reproducible, and valid tool to evaluate the quality of the service perceived by the users of the Spanish Early Intervention Centers.
Conceptualization: Rita Pilar Romero-Galisteo, Ángel Blanco-Villaseñor, Noelia Moreno-Morales, Pablo Gálvez-Ruiz.
Data curation: Rita Pilar Romero-Galisteo, Noelia Moreno-Morales.
Formal analysis: Ángel Blanco-Villaseñor, Pablo Gálvez-Ruiz.
Methodology: Ángel Blanco-Villaseñor, Pablo Gálvez-Ruiz.
Resources: Rita Pilar Romero-Galisteo.
Software: Pablo Gálvez-Ruiz.
Supervision: Pablo Gálvez-Ruiz.
Validation: Ángel Blanco-Villaseñor, Noelia Moreno-Morales, Pablo Gálvez-Ruiz.
Visualization: Pablo Gálvez-Ruiz.
Writing – original draft: Rita Pilar Romero-Galisteo, Ángel Blanco-Villaseñor, Noelia Moreno-Morales, Pablo Gálvez-Ruiz.
Writing – review & editing: Rita Pilar Romero-Galisteo, Ángel Blanco-Villaseñor, Noelia Moreno-Morales, Pablo Gálvez-Ruiz.
. Liu X, Wang XM, Ge JJ, et al. Effects of the portage early education program on Chinese children with global developmental delay. Medicine 2018;97:e12202.
. Iversen MD, Shimmel JP, Ciacera SL. Creating a family
-centered approach to Early Intervention
Services: perceptions of parents and professionals. Pediatr Phys Ther 2003;15:23–31.
. Pinto AI, Grande C, Aguiar C, et al. Early childhood intervention in Portugal. An overview based on the Developmental Systems Model. Infants Young Child 2012;25:310–22.
. Soriano V. Early childhood intervention analysis of situations in Europe. Key aspects and recommendations. Brussels: European Agency for Development in Special Needs Education. Summary Report 2005.
. Cunha CA, Paz-Lourido B. El enfoque de la calidad de vida familiar en la atención en salud: el caso de la Fisioterapia en Atención Temprana. Saúde Transformaçao Social (in Spanish) 2010;1:20–7.
. Guralnick MJ. Why early intervention
works: a systems perspective. Infants Young Child 2011;24:6–28.
. Kyzar KB, Chiu C, Kemp P, et al. Feasibility of an online professional development program for early intervention
practitioners. Infants Young Child 2014;27:174–91.
. Hadders-Algra M, Boxum AG, Hielkema T, et al. Effect of early intervention
in infants at very high risk of cerebral palsy: a systematic review. Dev Med Child Neurol 2017;59:246–58.
. McCollum JA. Shonkoff JP, Meisels SJ. Topics in early childhood special education. Book Review: Handbook of Early Childhood Intervention 2nd ednCambridge, MA: Cambridge University Press; 2001. 176–8.
. Shonkoff JP. From neurons to neighborhoods: old and new challenges for developmental and behavioral pediatrics. J Dev Behav Pediatr 2003;24:70–6.
. Bradley J, Ingras S. Improving the quality of child health services: participatory action by providers. Int J Qual Health Care 2005;17:391–9.
. Pilgrim NA, Cardona KM, Pinder E, et al. Clients’ perceptions of service quality and satisfaction at their initial Title X family
planning visit. Health Commun 2014;29:505–15.
. Martínez MT, Martínez L. Promoción del desarrollo infantil y Atención Temprana: calidad de los servicios. Azarbe: Revista internacional de trabajo social y bienestar (in Spanish) 2013;2:49–67.
. Shirley ED, Sanders JO. Patient satisfaction: implications and predictors of success. J Bone Joint Surg Am 2013;95:E69.
. Batbaatar E, Dorjdagva J, Luvsannyam A, et al. Conceptualisation of patient satisfaction: a systematic narrative literature review. Perspect Public Health 2015;135:243–50.
. Vázquez FL, Torres A, Otero P, et al. Psychometric properties of the castilian Spanish version of the client satisfaction Questionnaire (CSQ-8). Curr Psychol 2017;doi.org/10.1007/s12144-017-9659-8.
. Parasuraman A, Berry LL, Zeithalm VA. Refinement and reassessment of the SERVQUAL scale. J Retail 1991;67:420–50.
. Andaleeb SS. Service quality perceptions and patient satisfaction: a study of hospitals in a developing country. Soc Sci Med 2001;52:1359–70.
. Batista-Foguet JM, Coenders G. Cuadernos de Estadística: Modelos de Ecuaciones Eestructurales (in Spanish). 2nd ed.Madrid: La Muralla; 2012.
. Farr M, Cressey P. Understanding staff perspectives of quality in practice in healthcare. BMC Health Serv Res 2015;15:123.
. Guesalaga R, Pitta D. The importance and formalization of service quality dimensions: a comparison of Chile and the USA. J Consum Mark 2014;3:145–51.
. Aytch L, Castro DC, Selz-Campbell L. Early Intervention
Services Assessment Scale (EISAS)—conceptualization and development of a program quality self-assessment instrument. Infants Young Child 2004;17:236–46.
. Cerqueira S, Dessen MA, Pérez-López J. Evaluación de los servicios de atención a familias de niños con deficiencia. Perspectiva de familiares y profesionales brasileños. Anal Psicol (in Spanish) 2012;28:866–74.
. Romero-Galisteo R, Morales V, Hernández A. Development of a tool for assessing the perceived quality
in Early Intervention
Centers. Anales de Psicología 2015;31:127–36.
. Ziviani J, Cuskelly M, Feeney R. Parent satisfaction with early intervention
services for children with physical disabilities: internal consistency of the European Parent Satisfaction Scale about Early Intervention
. Infants Young Child 2010;23:184–94.
. Fisher VP Jr, Elbaum B, Coulter WA. Construction and validation of two parent-report scales for the evaluation of early intervention
programs. J Appl Meas 2012;13:57–76.
. Scharmanski S, Renner I. The assessment of family
resources and need for help: construct validity
and reliability of the Systematic Exploration and Process Inventory for health professionals in early childhood intervention services (SEVG). Z Evid Fortbild Qual Gesundhwes 2016;118-119:1–9.
. Romero-Galisteo RP, Morales-Sánchez V, Hernández-Mendo A. Genralizability and optimization for quality assessment in early intervention
centers (CAIT). Univ Psychol 2015;14:1009–20.
. Jemes-Campaña I, Romero-Galisteo RP, Labajos-Manzanares MT, et al. Evaluación de la Calidad de servicio en Atención Temprana: revisión sistemática. An Pediatr (in Spanish) 2018;doi: 10.1016/j.anpedi.2018.04.014. In press.
. Medina F, De Oliveira SL, Lillo MC, et al. Percepción de los padres de la atención de fisioterapia recibida en centros de atención temprana. Revista de Atención Temprana (in Spanish) 2007;10:21–8.
. García F, Mirete A, Marín C, et al. Satisfacción del cliente familia en atención temprana: valoración de la importancia que otorgan a distintos aspectos del servicio. Siglo Cero (in Spanish) 2008;39:35–54.
. Anderson JC, Gerbing DW. Structural equation modeling in practice: a review and recommended two-step approach. Psychol Bull 1988;103:411–23.
. Brown TA. Confirmatory Factor Analysis for Applied Research. New York, NY: The Guilford Press; 2006.
. Lloret-Segura S, Ferreres-Traver A, Hernández-Baeza A, et al. El análisis factorial exploratorio de los ítems: una guía práctica, revisada y actualizada. Anales de Psicología (in Spanish) 2014;30:1151–69.
. Matsunaga M. How to factor-analyze your data right: do's, don’ts, and how-to's. Int J Psychol Res 2010;3:97–110.
. Worthington R, Whittaker T. Scale development research. A content analysis and recommendations for best practices. Counseling Psychol 2006;34:806–38.
. Browne M, Cudeck R. Bollen KA, Long JS, et al. Alternative ways of assessing model fit. Testing Structural Equation Models. Beverly Hills, CA: Sage; 1993. 136–62.
. Hu L, Bentler P. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Eq Model 1999;6:1–55.
. Hair JF, Black WC, Babin B, et al. Multivariate Data Analyses. 7th ed.New York, NY: Pearson Prentice-Hall; 2010.
. Kline RB. Principles and Practice of Structural Equation Modeling. 2nd ed.New York: The Guilford Press; 2005.
. Wang J, Wang X. Structural Equation Modeling Applications Using Mplus. Chichester: John Wiley & Sons Ltd; 2012.
. Fornell C, Larcker DF. Evaluating structural equation models with unobservable variables and measurement error. J Mark Res 1981;18:39–50.
. Byrne BM. Structural Equation Modeling with AMOS: Basic Concepts, Applications, and Programming. 2nd ed.Mahwah: Lawrence Erlbaum Associates; 2009.
. Cheung GW, Rensvold RB. Evaluating goodness-of-fit indexes for testing measurement invariance. Struct Eq Model 2002;9:233–55.
. Marôco J, Marques T. Qual a fiabilidade do alfa de Cronbach? Questões antigas e soluções modernas? Laboratório de Psicología (in Portuguese) 2006;3:65–90.
. Wu ML. Structural Equation Modeling: Operation and Application of Amos. 2nd ed.Chongqing: Chongqing University Press; 2010.
. Marôco J. Análise de Equações Estructurais. Fundamentos teóricos, Software & Aplicações (in Portuguese). Pero Pinheiro: Rolo y Filhos II; 2010.
. Chowdhary N, Prakash M. Prioritizing service quality dimensions. Manag Service Quality 2007;17:493–509.
. Scharmanski S, Renner I. The assessment of family
resources and need for help: Construct validity
and reliability of the Systematic Exploration and Process Inventory for health professionals in early childhood intervention services (SEVG). J Evid Qual Health Care 2016;118-119:1–9.
. Palazón-Cabanes B, López-Picazo JJ, Morales-Ortiz A, et al. Identification of the factors conditioning times and indicators of quality in the intrahospital care of acute stroke. Rev Neurol 2016;62:157–64.
. Copeland SR, Luckasson R, Shauger R. Eliciting perceptions of satisfaction with services and supports from persons with intellectual disability and developmental disabilities: a review of the literature. J Intellec Disabil Res 2014;58:1141–55.
. Mas JM, Basqués N, Balcells-Balcells A, et al. Family
quality of life for families in early intervention
in Spain. J Early Interv 2016;38:59–74.