Secondary Logo

Internal Consistency and Convergent Validity of the Portuguese Versions of the Global Appraisal of Individual Needs–Initial and Short Screener: Validity of the Portuguese GAIN-I and SS

Garcia Claro, Heloísa BSN, MSc, PhD; Ferreira de Oliveira, Márcia Aparecida BSN, MSc, PhD; de Almeida Lopes Fernandes, Ivan Filipe MSc, PhD; Titus, Janet C. PhD; Ribeiro Tarifa, Rosana BSN, MSc; Fernandes Rojas, Thais BSN; Hayasi Pinho, Paula PhD

doi: 10.1097/JAN.0000000000000143
Original Article
Open

The goal of this article is to present evidence on the internal consistency and convergent validity of the Brazilian Portuguese versions of the Global Appraisal of Individual Needs–“Initial” and “Short Screener” versions.

Methods One hundred sixty-eight individuals from an inpatient service and/or a community-based outpatient service located in São Paulo were interviewed using the Brazilian Portuguese versions of the instruments. The internal consistency of the instruments scales was computed, along with evidence for the convergent validity between corresponding subscales of the Initial and Short Screener instruments.

Results Cronbach’s alpha values for both instruments’ total scale scores were greater than .7. The Short Screener scales showed strong-to-moderate correlations with corresponding subscales of the Initial. The General Individual Severity Scale from the Initial and Total Disorder Screener from the Short Screener have convergent validity with each other (ρ = 0.801).

Conclusions The Brazilian Portuguese instrument scales showed evidence for internal consistency and convergent validity performing similarly to the American English versions.

Heloísa Garcia Claro, BSN, MSc, PhD, Márcia Aparecida Ferreira de Oliveira, BSN, MSc, PhD, Rosana Ribeiro Tarifa, BSN, MSc, Thais Fernandes Rojas, BSN, and Paula Hayasi Pinho, PhD, School of Nursing, University of São Paulo, Brazil.

Ivan Filipe de Almeida Lopes Fernandes, MSc, PhD, Federal University of ABC, São Bernardo do Campo, São Paulo, Brazil.

Janet C. Titus, PhD, Chestnut Health Systems, Normal, Illinois.

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.

Correspondence related to content to: Heloísa Garcia Claro, BSN, MSc, PhD, School of Nursing, University of São Paulo, Rua Doutor Enéas de Carvalho Aguiar, 419, Departamento ENP, CEP: 05403-000, São Paulo, Brazil. E-mail: heloisa.claro@usp.br

This is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial-No Derivatives License 4.0 (CCBY-NC-ND), where it is permissible to download and share the work provided it is properly cited. The work cannot be changed in any way or used commercially without permission from the journal.

Back to Top | Article Outline

INTRODUCTION

In 2012, 243 million people worldwide reported using illicit drugs (UNODC, 2014). The most recent epidemiological studies about illegal drugs showed that, in 2012, 3.9% of the population reported lifetime use of inhaled cocaine and 2.2% reported lifetime use of crack. Cocaine addiction was identified in 41.4% of individuals who reported using the drug in the past 12 months (Abdalla et al., 2014).

Each year, 2.5 million people worldwide die from about 60 different types of diseases caused by alcohol use (World Health Organization [WHO], 2011). Recently, in Brazil, 12.7% of the population reported an episode of heavy drinking during the past 30 days, and 2.8% of the population met the criteria for alcohol dependence (WHO, 2014).

Mental disorders are responsible for more than 10% of years of life lost and more than 30% of years lived with disability. These disorders are highly prevalent and are often chronic in nature. Low rates of screening and ineffective treatment aggravate the problem, especially in poor or developing countries such as Brazil (Chisholm, Saxena, Ommeren, & Department of Mental Health and Substance Abuse, WHO, 2006; Funk, Drew, Freeman, Faydi, & WHO, 2010).

The expansion of the drug market and an increase in crime in the Brazilian states are associated with an increase in drug use. When users are identified and referred for treatment at an early stage of use, the overall costs related to their use are reduced, including costs related to criminal justice (Pereira Filho, Tannuri-Pianto, & Sousa, 2010; Santos & Kassouf, 2007).

Although there is clearly an association between mental health, use of alcohol and other drugs (AODs), and social behavior, the regular screening tools for diagnosis and treatment planning are limited to AOD use and do not assess the individuals’ needs and problems in associated areas (Claro, 2010, 2013; Claro & Oliveira, 2010).

The Global Appraisal of Individual Needs (GAIN) is a family of evidence-based assessments for use with individuals who may be in need of identification, referral, and/treatment for substance abuse and other mental health disorders. The spectrum of GAIN measures ranges from a screening instrument (GAIN–Short Screener [SS]), which takes approximately 10–15 minutes to conduct; a targeted assessment (GAIN–Quick 3), which takes 30–45 minutes to conduct; and a full biopsychosocial evaluation (GAIN–Initial [I]), which takes 90–120 minutes to conduct (Dennis, White, Titus, & Unsicker, 2003).

The GAIN instruments are designed to guide clinical decisions about the diagnosis, treatment, planning, placement, and monitoring of treatment results. They are widely used to support research in the substance use field, mainly in the United States and Canada. The assessments can also be integrated into electronic health records and information systems (Dennis, Chan, & Funk, 2006; Dennis et al., 2003; Dennis, Funk, Godley, Godley, & Waldron, 2004).

The GAIN-I1 instrument covers information in eight main topic areas (background, substance use, physical health, risk behaviors, mental health, environment, legal aspects, and vocational aspects; Dennis et al., 2003). Because it evaluates withdrawal and other physiological symptoms, it is also used in more intensive and prolonged care settings such as intensive outpatient and inpatient services. The instrument is also used as an important research tool for the evaluation of outcomes (Dennis et al., 2003, 2004, 2006).

The GAIN-SS2 was developed to meet the demands of a practical instrument to easily and quickly detect individuals’ potential needs in mental health, AOD, and crime and violence. Scores from the assessment place individuals along a spectrum of risk, providing guidance for further evaluation or referral. It also may be used as part of a brief intervention and prevention tool as well as a means for referral to specialized treatment (Dennis et al., 2006).

All of the GAIN tools are housed within an online application called the GAIN Assessment Building System (ABS). Because the GAIN instruments are computer-administered assessments, participant data are entered directly into the ABS at the time of assessment administration. After assessment administration, clinical reports and diagnoses based on the American Society of Addiction Medicine; patient placement criteria for the treatment of substance-related disorders; the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition; and the International Classification of Diseases criteria are generated for clinical use (Dennis et al., 2003).

Recently, two of the GAIN assessments—the GAIN-I and the GAIN-SS—were translated to Portuguese and adapted to the Brazilian culture as well as the ABS system in which they reside. These projects were conducted by the Study Group on Alcohol and Other Drugs, a research group in the School of Nursing at the University of São Paulo, Brazil.

The purpose of this study is to evaluate the internal consistency and convergent validity of the Brazilian Portuguese versions of the GAIN-I and GAIN-SS, which acronyms and names in Portuguese became AGNI-I (Avaliação Global das Necessidades Individuais–Inicial) and AGNI-RR (Avaliação Global das Necessidades Individuais–Rastreio Rápido).

Back to Top | Article Outline

METHODS

Data were collected using the Portuguese versions of the GAIN-I (AGNI-I) and GAIN-SS (AGNI-RR) instruments. The interviews were conducted with inpatient and outpatient users being treated for substance abuse at two programs in São Paulo, Brazil. The sample consisted of 168 individuals (70 who completed both the AGNI-I and the AGNI-RR, 40 who completed only the AGNI-I, and 58 who completed only the AGNI-RR).

Users who entered each program were approached for participation. After explaining the project, showing them the instruments, and conducting the informed consent, those individuals who agreed to participate were interviewed. Interviews were conducted in a comfortable environment, and users were offered snacks, water, and breaks as needed. In cases of fatigue, users were given the option to finish the assessment on another day. The study methods were approved by the ethics committees of the School of Nursing at the University of São Paulo, the Municipal Health Secretariat of São Paulo, and Chestnut Health Systems in Bloomington, Illinois.

Data from the interviews were entered into the GAIN ABS and were exported to the Statistical Package for Social Sciences Version 20 for Microsoft Windows 7. Descriptive statistics estimates of internal consistency for each instrument scale and Pearson correlations between corresponding AGNI-I and AGNI-RR scales were calculated.

  • AGNI-I (15 scales to be measured: Substance Frequency Scale, Current Withdrawal Scale [CWS], Self-Efficacy Scale, Substance Problem Scale-Lifetime, Health Distress Scale, Health Problem Scale, Internal Mental Distress Scale [IMDS], General Mental Distress Scale [GMDS], Behavior Complexity Scale [BCS], Emotional Problem Scale, Environmental Risk Scale, General Victimization Scale, Illegal Activities Scale [IASS], Employment Activity Scale, General and Individual Severity Scale [GISS]);
  • AGNI-RR (five scales to be measured: Internal Disorder Screener [IDScr], External Disorder Screener [EDScr], Substance Disorder Screener [SDScr], Total Disorder Screener [TDScr], Crime/Violence Screener [CVScr]);
  • IDScr and IMDS (AGNI-I);
  • EDScr (AGNI-RR) and BCS (AGNI-I);
  • SDScr (AGNI-RR) and SPS (AGNI-I);
  • CVScr (AGNI-RR) and CVS (AGNI-I); and
  • TDScr (AGNI-RR) and GISS (AGNI-I).
Back to Top | Article Outline

RESULTS

Demographics

Users interviewed with the AGNI-RR were predominantly men (85.2%), with a mean age of 36.79 years. The average time to complete the interview was 12 minutes. Users reported an average of three internalizing disorder symptoms (out of 6), 3.73 alcohol and drug use disorder symptoms (out of 5), 2.10 externalizing disorder symptoms (out of 7), and 1.01 crime and violence problems (out of 5) during their lifetime. Limited demographics were available on this group given the screening nature of the AGNI-RR.

Users interviewed with the AGNI-I were also predominantly men (70%), with a mean age of 33.35 years. The average time to complete the interview was 123 minutes. The age of first use of drugs or alcohol to intoxication was 15.82 years (range = 7–42 years). In addition, 41.8% identified themselves as White, 67.3% had never been homeless, 38.2% were tobacco smokers, 91.8% were sexually active during the year preceding the interview, and 73.6% reported never being involved in criminal activity. Nearly one third of the users (30.9%) reported having suffered mentally or psychologically in a significant way during the 2 days before the interview, 44.5% were attending treatment for the first time, and 80.9% reported being drunk or high on at least 1 of the 90 days before the interview.

Back to Top | Article Outline

Internal Consistency

Internal consistency of the 15 AGNI-I scales and the five AGNI-RR scales was analyzed using Cronbach’s alpha.

Because data were collected in both inpatient and outpatient services, a comparative analysis of the confidence intervals (CIs) of the alphas for both services was conducted to check for possible statistically significant differences between the samples.

An overlap between the CIs for the alphas shows that there is no statistically significant difference in the alphas between the inpatient and outpatient samples.

Back to Top | Article Outline

Evidence for Convergent Validity

Using data from 70 users who completed both the AGNI-I and AGNI-RR, Pearson correlations were computed between corresponding scales on the AGNI-I (long versions) and AGNI-RR (short versions). The analysis was conducted to determine the degree of evidence in support of convergent validity (see Table 1).

TABLE 1

TABLE 1

Results showed acceptable correlation indices between the GISS and TDScr scales (.80) and between the IDScr and IMDS scales (.72), with values above a minimally accepted standard of .70 (see Table 1; Dancey & Reidy, 2007, 2008).

The remaining comparisons yielded indices between .40 and .60, indicating a moderate relationship (Dancey & Reidy, 2007, 2008).

Back to Top | Article Outline

DISCUSSION

A previous study conducted in the United States showed that the English GAIN-SS had an internal consistency estimate (using Cronbach’s alpha) of .87, a value similar to that found in the Brazilian Portuguese version (.828) and above .70, an acceptable cutoff specified in the scientific literature (Dennis et al., 2006). This result shows that the internal consistency of the AGNI-RR is compatible with that of the original version.

The internal consistency of the AGNI-RR subscales on the Brazilian Portuguese version is lower when compared with those in the original English instrument. However, the size of the U.S. sample was at least 80 times greater than the sample in this study. The IDScr Cronbach’s alpha was above .70 (α = .762, 95% CI [0.692, 0.82]); the EDScr and SDScr scales yielded similarly valued coefficients (α = .655, 95% CI [0.554, 0.740]; α = .673, 95% CI [0.575, 0.755], respectively). The CIs for the internal consistency estimates include the value .70 for all but one scale—the CVScr—which estimate is below .70 and which CI does not include this value (α = .446, 95% CI [0.279, 0.585]). This scale also performed less well in English, as seen in Table 2.

TABLE 2

TABLE 2

The measurement of behaviors related to crime and violence is extremely complex and diverse and may interfere with the ability of the CVScr to measure such behaviors (Dennis, Feeney, & Titus, 2013). There is reference in the literature to a self-report measurement obstacle called “social convenience blindness” in which respondents deny behaviors associated with social stigma, immorality, and other socially condemnable behaviors (Pérula De Torres et al., 2009; Fabbri, Furtado, & Laprega, 2007; Gan, Sanz, Valladolid, & Clavo, 2006).

Of the 15 subscales on the AGNI-I instrument, nine had an α coefficient greater than .70 (see Table 3), whereas 14 had 95% CIs that included or exceeded the value .70 (CWS, SPS, IMDS, GMDS, BCS, Environmental Risk Scale, General Victimization Scale, IASSp, and GISS).

TABLE 3

TABLE 3

Compared with the U.S. data in 7 of the 15 subscales studied, the U.S. alpha value is contained within the 95% CI of the Brazilian data, showing compatibility of the Portuguese subscales with the original English versions.

The internal consistency of a quantitative measure refers to the consistency with which an instrument measures an attribute: the smaller the variation produced by the instrument—the measurement of an attribute—the higher the internal consistency (Polit, Beck, & Hungler, 1995). The internal consistency analysis examines the correlations among the items and the full scale score. The Cronbach’s alpha is a statistic indicating the degree of covariance among the items that should be intercorrelated when measuring the same phenomena (Polit et al., 1995). There is no consensus on the interpretation of the internal consistency of a questionnaire. The value of .70 is often used as the minimum acceptable value. This parameter was achieved by both the AGNI-I and AGNI-RR in their most comprehensive forms (GISS and TDScr).

For four AGNI-I scales, the alpha coefficients were greater than .90 (CWS, IMDS, GMDS, and IASSp), which is a very high degree of correlation. This result may indicate redundancy among the items within the scales (Tavakol & Dennick, 2011).

Some items were shown to be redundant by the statistical analysis in representing clinical information that is important. For example, Items m2h and m2n on the AGNI-I (“You used alcohol or other drugs to help yourself sleep or forget about things that happened in the past” and “You felt guilty about things that happened because you felt like you should have done something to prevent them”), although both items deal with problems and difficult situations from the past, the information in each of these items is unique and useful for the clinician.

Regarding the evidence in favor of convergent validity for the subscales of the AGNI-RR and AGNI-I, the GISS and TDScr scales (ρ = 0.801) and the IDScr and IMDS scales (ρ = 0.721) showed acceptable correlations (≥.70; Dancey & Reidy, 2007, 2008). Correlation coefficients between .40 and .60 are considered positive and moderate in size. Moderate positive correlations were also found between the other two combinations of scales in this study (EDScr and BCS, ρ = 0.553; SDScr and Substance Problem Scale-Lifetime, ρ = 0.581; CVScr and CVS, ρ = 0.652).

In summary, the full scales on both AGNI-I and AGNI-RR are positively correlated, exhibiting evidence for convergent validity. The corresponding subscales between both instruments have moderate-to-high positive correlations, also exhibiting evidence for convergent validity (Dancey & Reidy, 2007, 2008).

Back to Top | Article Outline

Study Limitations

The sample size in this study was over 80 times smaller than the sample studied in the United States. Even so, there were compatibility between the American English and Brazilian Portuguese versions of the instruments, acceptable levels of internal consistency, and evidence for convergent validity between the AGNI-I and AGNI-RR scales and subscales.

In addition, data were collected in only two AOD programs, so the generalization of results is limited. Most respondents were between 18 and 60 years old. In future studies, the elderly and adolescent population should be included, and the instruments should be tested in a larger variety of municipalities and services with larger samples.

Back to Top | Article Outline

CONCLUSIONS

The 14 subscales of the AGNI-I have good (above 0.6) to excellent (above 0.9) estimates of internal consistency. The AGNI-RR instrument has internal consistency compatible with the original American English version and meets the recommendations found in the literature.

Analyses of the two instruments support evidence for their convergent validity. The four subscales of AGNI-RR show high-to-moderate correlations with the AGNI-I scales. The total disorder scale (TDScr) of AGNI-RR and the general severity scale of AGNI-I (GISS) are highly correlated.

The quality of mental health, social relationships, and behavior are influenced by potential outcomes of social inequality (crime and violence) and have a direct impact on the severity of problems regarding the use of AOD. These results emphasize the need for interdisciplinary measures when assessing users of AODs. The standardized and validated instruments that collect data on a range of areas of interest are important tools for the practitioner.

Back to Top | Article Outline

Acknowledgments

We thank Fundação de Amparo à Pesquisa do Estado de São Paulo for the funding of research projects related to this article: Grant nos. 2012/23873-7, 2011/00226-3, and 2010/20741-7.

We also thank the Chestnut Health Systems for their help and support throughout this research.

Back to Top | Article Outline

REFERENCES

Abdalla R. R., Madruga C. S., Ribeiro M., Pinsky I., Caetano R., & Laranjeira R. (2014). Prevalence of cocaine use in Brazil: Data from the II Brazilian National Alcohol and Drugs Survey (BNADS). Addictive Behaviors, 39(1), 297–301.
Chisholm D., Saxena S., Ommeren M. V., & Department of Mental Health and Substance Abuse, World Health Organization. (2006). Dollars, DALYs and decisions: Economic aspects of the mental health system. Geneva, Switzerland: World Health Organization.
Claro H. G. (2010). Tradução e adaptação cultural do instrumento Global Appraisal of Individual Needs-Initial. Retrieved from http://www.teses.usp.br/teses/disponiveis/7/7141/tde-13012011-151858/pt-br.php
Claro H. G. (2013). Desafios na adaptação cultural e validação dos instrumentos “avaliação global das necessidades individuais” para uso no Brasil. Paper presented at the 17o Seminário Nacional de Pesquisa em Enfermagem, Natal, RN, Brasil.
Claro H. G., & Oliveira M. A. F. d. (2010). Adaptação cultural do instrumento “Global Appraisal of Individual Needs” para uso com adolescentes no Brasil. Projeto de Mestrado, Escola de Enfermagem da Universidade de São Paulo, São Paulo, Brazil.
Dancey C., & Reidy J. (2008). Statistics without Maths for psychology: Using SPSS for Windows. Harlow, England: Pearson Education.
Dancey C. P., & Reidy J. (2007). Statistics without Maths for psychology. Harlow, England: Pearson Education.
Dennis M., Feeney T., & Titus J. (2013). Global Appraisal of Individual Needs-Short Screener (GAIN-SS): Administration and scoring manual. Retrieved from http://gaincc.org/_data/files/Instruments%20and%20Reports/Instruments%20Manuals/GAIN-SS_Manual_3_0.pdf
Dennis M., White M., Titus J., & Unsicker J. (2003). Global Appraisal of Individual Needs: Administration guide for the GAIN and related measures. Bloomington, IL: Chestnut Health Systems.
Dennis M. L., Chan Y. F., & Funk R. R. (2006). Development and validation of the GAIN Short Screener (GSS) for internalizing, externalizing and substance use disorders and crime/violence problems among adolescents and adults. The American Journal on Addictions, 15(Suppl. 1), 80–91. doi:G007886P85267424 [pii] 10.1080/10550490601006055
Dennis M. L., Funk R., Godley S. H., Godley M. D., & Waldron H. (2004). Cross-validation of the alcohol and cannabis use measures in the Global Appraisal of Individual Needs (GAIN) and Timeline Followback (TLFB; Form 90) among adolescents in substance abuse treatment. Addiction, 99(Suppl. 2), 120–128. doi:ADD859 [pii] 10.1111/j.1360-0443.2004.00859.x
Fabbri C. E., Furtado E. F., & Laprega M. R. (2007). Alcohol consumption in pregnancy: Performance of the Brazilian version of the questionnaire T-ACE. Revista de Saude Publica, 41(6), 979–984. doi:S0034-89102006005000044 [pii]
Funk M., Drew N., Freeman M., Faydi E., & World Health Organization. (2010). Mental health and development: Targeting people with mental health conditions as a vulnerable group. Geneva, Switzerland: World Health Organization.
Gan V., Sanz B., Valladolid G., & Clavo F. (2006). Validación de la Escala sobre el Deseo de Beber-EDB. Clinica y Salud, 17(2), 20.
Modisette K., Hunter B., Ives M., Funk R., & Dennis M. (2012). NORMS including alpha, mean, N, SD, ICC for adolescents (by demographics) and overall for young adults (18–25) and Adults (18+) using the SAMHSA/CSAT 2012 SA dataset [electronic version]. Normal, IL: Chestnut Health Systems.
    Pereira Filho O. A., Tannuri-Pianto M. E., & Sousa M. d. C. S. d. (2010). Medidas de custo-eficiência dos serviços subnacionais de segurança pública no Brasil: 2001–2006. Economia Aplicada, 14, 313–338.
    Pérula De Torres L. A., Carmona-Tamajón V., Montero Pérez-Barquero M., Ruiz-Moral R., Fernández-García J. A., & Muriel-Palomino M. (2009). Validation of the MALT-S questionnaire for the diagnosis of alcoholism among the adult population. Atencion Primaria, 41(8), 446–452.
    Polit D., Beck C., & Hungler B. (1995). Fundamentos de pesquisa em enfermagem (3rd ed.). Porto Alegre, Brazil: Artes Médicas.
    Santos M. J. d., & Kassouf A. L. (2007). Uma investigação econômica da influência do mercado de drogas ilícitas sobre a criminalidade Brasileira. Economia, 8(2), 187–210.
    Tavakol M., & Dennick R. (2011). Making sense of Cronbach’s alpha. International Journal of Medical Education, 2, 53–55.
    UNODC. (2014). World drug report 2014. New York: United Nations.
    World Health Organization. (2011). Global status report on alcohol and health. Geneva, Switzerland: Author.
    World Health Organization. (2014). Global status report on alcohol and health—2014. Geneva, Switzerland: Author.

    1Portuguese version (Avaliação Global das Necessidades Individuais – Inicial - AGNI-I) available for consultation: http://goo.gl/yJ19pt
    Cited Here...

    2Portuguese version (Avaliação Global das Necessidades Individuais – Rastreio Rápido AGNI-RR) available for consultation: http://goo.gl/28BP2c
    Cited Here...

    Keywords:

    alcohol-related disorders; substance-use-related disorders; validation studies

    © 2016International Nurses Society on Addictions