Secondary Logo

Share this article on:

Incremental Effect of Academic Predictors on Nursing Admission Assessment

Liu, Xin, PhD; Codd, Casey, PhD; Mills, Christine, PhD

doi: 10.1097/NNE.0000000000000502
Feature Articles

This study examined the incremental values of academic predictors that may be adopted as nursing school admission criteria. The findings revealed that using additional content areas of mathematics, reading, and English in conjunction with science contributes significantly to the prediction of early nursing school success. This study demonstrates an incremental validity approach that provides valuable guidelines to inform the selection of effective, accurate, and cost-effective admission tools for screening qualified nursing candidates.

Author Affiliations: Senior Psychometrician (Dr Liu), Lead Psychometrician (Dr Codd), and Director (Dr Mills), Research and Applied Psychometrics, Ascend Learning, Leawood, Kansas.

The authors are employed by Ascend Learning, whose subsidiary (Assessment Technologies Institute) produces the assessments discussed in this manuscript. The authors’ compensation is not dependent on research findings. Research design, analysis, and results reporting are conducted independently from the business unit.

Correspondence: Dr Liu, 11161 Overbrook Rd, Leawood, KS 66211 (Xin.Liu@AscendLearning.com).

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (www.nurseeducatoronline.com).

Accepted for publication: November 16, 2017

Published ahead of print: December 29, 2017

The demand for nursing jobs is expected to increase across the United States over the next decade. According to the Bureau of Labor Statistics, the total number of job openings for RNs is expected to grow 16% from 2014 to 2024, much faster than the average for all other occupations.1 The promise of greater opportunity to enter the nursing profession has successfully drawn an increasing number of applicants for admission to nursing programs. However, nursing student attrition rates are too high and have been reported as much as 50% in some baccalaureate nursing programs.2 Although nursing schools cannot mitigate all risk factors leading to attrition of nursing students (eg, unforeseen changes in students’ personal circumstances after they are admitted), nursing faculty can use effective admission criteria (eg, standardized admission tests) to help select applicants who are the most likely to succeed in the program.3,4

Back to Top | Article Outline

Literature Review

Commonly identified predictors of academic success in the literature about nursing program admission include a student’s high school grade point average (GPA), standardized general academic test scores such as the SAT and ACT, or standardized examinations specifically designed for students applying to nursing programs, such as the Assessment Technologies Institute (ATI) Test of Essential Academic Skills (TEAS), the National League for Nursing Preadmissions Exam, the Nurse Entrance Test (NET), and the HESI A2.5-9 Previously published studies indicate that the use of standardized entrance examinations is significantly related to early nursing school success, nursing program completion, and the NCLEX pass rate.10-12

Standardized academic examinations typically assess several content areas such as mathematics, science, reading, and English, reporting subtest scores (or subscores) and some form of a composite score. Several studies in nursing education have indicated that the science component of a standardized academic assessment is a strong predictor of nursing program success.13-18 Wolkowitz and Kelley17 examined whether students’ TEAS subtest scores in reading, mathematics, science, and English predict early nursing school success. The authors concluded that nursing program success was best predicted by TEAS subscores in the order of science, reading, English, and mathematics. Potolsky et al13 found that prerequisite science course performance (eg, science GPA) reliably predicts academic performance in nursing.

In addition to science, researchers have demonstrated that other admission test content areas were predictive of program success. Sayles et al19 found that NET mathematics skills, reading comprehension, and composite scores were predictive of success on the NCLEX-RN. In research by Gallagher et al,20 the NET subscore of mathematics had a significant relationship to success of students in earning a grade of C or better in the first nursing course.20 Other studies21,22 found that verbal SAT scores were highly correlated with NCLEX-RN performance. Hopkins23 reported that the reading section of the NET was predictive of success. In an exploration of predictive factors to assist associate degree in nursing (ADN) programs in determining admission criteria, the ACT English subscore was statistically significant to program completion.24 Hope25 found that the first-semester GPA was positively related to the TEAS English score and the TEAS science scores for ADN students. Walker et al26 reported that reading comprehension was the best predictor of nursing program completion.

On the other hand, several research findings are not supportive of the predictive impact of reading, mathematics, or English content areas on nursing school success. The authors reviewed a number of them and listed a few as examples here. Gallagher et al20 found that the NET subscore of reading had no significant relationship to success of students in earning a grade of C or better in the first nursing course. Sayles et al19 also found that the ACT reading section was not a good predictor of success. Hope25 found no significant correlation for TEAS mathematics and reading scores with students’ GPA for ADN students.

The identification of effective admission criteria that predict success in a nursing program is important to the students, institution, and community.27 Nursing schools are often faced with multiple admission criteria or factors related to success in the program, and admission committees need to choose the assessment that provides the best predictive model for the school. Seeking balance through nursing admission criteria, a question that an admission committee might ask is whether the best predictive variable can be good enough itself such that there is no need to look at the other variables. This question probes into the increase in validity gained from adding extra measures to the current assessment, which is referred to as incremental validity.28 Incremental validity helps determine whether a particular additional measure could provide a significant improvement in the evaluation outcome.

In the current study, the authors look into the presence of incremental validity issues related to the utility of academic predictors in the nursing admission assessment. Specifically, the authors probe into the utility of various content areas covered in a nurse admission test. The research questions were as follows: (1) Is there significant incremental validity in improving the accuracy of predicting early school success by adding the other content areas in addition to science in the admission criteria? (2) Are there significant gains in the prediction accuracy using the composite scores (that are based on all content areas) instead of using the science subscores only?

Previously published studies have focused mainly on the predictive validity of each predictor as a separate unit. It might be the case that, for example, 2 variables are both significant predictors of school success, but using 1 instead of 2 might be good enough because the other adds no additional predictive contribution. Few research studies have examined the incremental values of the academic measures in predicting nursing school success. The incremental validity research on nursing admission predictors provides empirical evidence in situations where 1 predictor is consistently cited by the published literature as a significant predictor of success more often than the others. In this study, the authors examine the incremental validity of adding reading, mathematics, and English in addition to science predictor. This study demonstrates a procedure for nursing program admission officers and committees to evaluate the incremental benefit of additional variables considered in admission criteria.

Back to Top | Article Outline

Methods

Instrumentation

The predictor variables in this study were the performance data on reading, mathematics, science, and English tests of the TEAS V test. The assessment contains 150 items covering the 4 content areas of reading, mathematics, science, and English and language usage.

The outcome variable in this study was an indicator of early nursing school success measured by ATI’s RN Fundamentals 2013 assessment. The accuracy of an academic preparedness test in predicting school success is likely to be best measured when the criterion is early program success.29 Although this course might not be one of the most challenging early nursing courses,30 there are at least 2 reasons why the Fundamentals examination is used as the surrogate for early school success in this study. First, because it is usually given during a student’s first semester in a nursing program, success in a fundamentals course is critical to accomplish the objectives necessary to pass at subsequent levels. Second, RN course examinations may vary greatly from 1 program to another. The RN Fundamentals assessments are standardized, allowing for direct comparisons among students from different institutions.

Back to Top | Article Outline

Participants

The participants were students who took both the TEAS V examination and the RN Fundamentals 2013 assessment between April 2013 and November 2016 across the United States. Although examinees have the option to retake TEAS V multiple times, to maintain research design integrity, examinees who retook the assessment were excluded from the study. Similarly, examinees who took the RN Fundamentals assessment more than once were not included in the study. Previous surveys showed that some of the institutions used the TEAS score in their postadmission decisions only, whereas the majority used it in their preadmission decision only.17 To achieve a common level of the stakes, only preadmission use of the TEAS V scores was considered in the study. As such, only those examinees who took RN Fundamentals at least 2 months after TEAS V were included in the study. As the focus is on the incremental values of each content area on predicting RN Fundamentals, only those examinees who completed all 4 content areas of TEAS V and the RN Fundamentals were included. The final number of participants remaining for the study was 6402, with an average age of 29 years. They came from 204 institutions across 35 states. Among them, 3149 were from an ADN, and 3253 were from a BSN program. Demographic information on sex, ethnicity, primary languages, and regions are reported in Supplemental Digital Content 1, http://links.lww.com/NE/A449, Table 1.

Table 1

Table 1

Back to Top | Article Outline

Statistical Analyses

This study used a predictive design to evaluate the utility of the 4 common sources of information (reading, mathematics, science, and English subscores on the TEAS V assessment) in predicting early nursing school success on the RN Fundamentals course. Hierarchical regression was used to examine the relationship between the predictor sets and predictive outcome. The results answered the first research question mentioned in the earlier Literature Review section. The 4 predictor variables were TEAS V scores on each of the 4 content areas of reading, mathematics, science, and English and language usage. The dependent variable was the overall RN Fundamentals score.

Prior to conducting the hierarchical regression, a pairwise correlation coefficient between the RN Fundamentals score and each content area score was computed. Predictor variables were entered into the hierarchical regression model one at a time. The entry for predictor variables into the hierarchical regression followed the magnitude of the correlation coefficients from high to low based on the pairwise correlation coefficient results obtained at the earlier step. Specifically, the predictor with the highest correlation coefficient was entered at the first step, followed by the predictor with the second highest correlation at the second step, the third highest correlation at the third step, and the lowest correlation at the fourth step. The model at each subsequent step had 1 more predictor added. The model at the first step had only 1 predictor, and the model at the last step contained 4 predictors. At each subsequent step, the change in R 2 indicated the unique variance accounted for by the additional predictors (ie, incremental effect). The standardized regression coefficients showed the incremental importance of each predictor. As a final analysis step to the hierarchical regression results, a general regression model that regressed the TEAS composite score on RN Fundamentals scores was conducted to determine the predictive power of the 4 subscores combined, as an answer to the second research question mentioned in the earlier Literature Review section. Statistical analysis was conducted on the ADN subsample only (n = 3149, denoted as ADN), BSN subsample only (n = 3253, denoted as BSN), and the ADN-BSN combined total sample (N = 6402, denoted as total hereafter).

Back to Top | Article Outline

Results

Prior to the regression analyses, the independent variables were examined for collinearity. The low variance inflation factor (<1.9 for science, <1.7 for reading, <1.8 for English, and <1.6 for mathematics) and high collinearity tolerance (>0.5 for science, >0.6 for reading, >0.5 for English, and >0.6 for mathematics) suggested that collinearity was not an issue in the data.

Correlations between Fundamentals scores and each of the 4 TEAS V subscores are displayed in Table 1. All correlations are statistically significant (P < .001). For the total sample, the science subtest was the predictor with the highest correlation followed by reading, English, and mathematics. This pattern holds true for both the ADN and BSN subsamples.

Table 2 presents the standardized regression coefficients (β), R 2, and R 2 change (ΔR 2) for hierarchical regression analyses at every step. Table 2 shows some consistent patterns across models for ADN, BSN, and the total samples. First, at step 1, the science predictor accounted for a substantial and statistically significant amount of variance for total (14.7%), ADN (12.2%), and BSN (16.2%) programs. Science was the statistically significant independent variable (β significant at .05 level). Second, at step 2, as indicated by ΔR 2s, the reading predictors accounted for a significant amount of additional variance for total (3.5%), ADN (4.5%), and BSN (2.7%) over the step 1 models. The regression coefficients for both science and reading were significant for 3 samples (P < .05). Third, at step 3, the English predictor added a substantial and significant amount of the variance explained for total (0.7%), ADN (0.5%), and BSN (0.9%) samples; all 3 predictors were significant at P < .05 in all models. Finally, at step 4, the mathematics predictor added statistically significant variance for each sample, although the additional variance explained by the step 4 model is the least (by 0.1%), compared with the steps 1 to 3 models.

Table 2

Table 2

A notable finding is evident when comparing the regression results between ADN and BSN samples. At each and every step of the regression models, the model R 2 for BSN students is higher than that for ADN students. A possible implication from this finding is that performances on the academic measures might have a stronger impact on predicting success for BSN students than for ADN students.

The results of the linear regression of the TEAS composite scores are reported in Supplemental Digital Content 2, http://links.lww.com/NE/A450, Table 2. The TEAS composite score significantly predicted success on RN Fundamentals for the total sample of BSN and ADN students combined, β = .431, t 6,400 = 38.254, P < .001. The TEAS composite score also explained a significant portion of variance in the RN Fundamentals scores, R 2 = .431, F 6,400 = 1463.381, P < .001. The results of the linear regression of the TEAS composite scores showed the similar patterns for the ADN and BSN subsamples, respectively.

Back to Top | Article Outline

Discussion

Faced with the challenge to admit an increased number of qualified applicants, nursing programs require evidence-based admission criteria that effectively predict student success. The existing research findings provide useful but overwhelming resources for nursing faculty and admission staff in choosing effective tools for making accurate admission decisions. This poses challenges to institutions that are searching for the most helpful yet cost-effective assessment tool. Principles for selecting the most appropriate assessment include tools that are proven to be useful, reliable, valid, and cost-effective.31 The focus of the vast majority of existing research studies on the predictors for nursing school success is on finding out whether a particular predictor (or a set of predictors) is significant for student success. Few studies, as far as the authors were able to determine, examined which predictor is more predictive of success in nursing programs than the others. Because cost may be a factor in the selection process, research questions regarding financial implications may be added to inform future studies. In other words, factors related to time and money may be strong predeterminants in selecting the most appropriate admission tools and therefore included within a prediction model.

Recommendations on how to sort through the significant predictors are rare in the published literature. The present study addresses the gap by demonstrating an incremental validity analysis using ATI standardized testing data in predicting students’ success in the nursing program.

This study found that the correlation of TEAS V science with RN Fundamentals was higher than that of the other content areas of reading, English, and mathematics, in that order. This finding is in accordance with previous research.17 Based on the incremental validity analysis using hierarchical regression models, the results indicated that science cannot effectively substitute other content areas in nursing admission without the cost of losing predictive validity. The incremental validity is achieved from combining additional content areas with science; the additions provide information not captured by science alone. The incremental effect is significant for both BSN and ADN students. The significant model R 2 change at each step of the models indicates that each content area contributes significantly unique predictive information on early nursing school success.

The findings of this study provide insightful suggestions and recommendations to the assessment of nursing school admission criteria. Specifically, nursing school admission criteria should place higher values on candidates who demonstrate a broad range of skill and knowledge areas. All 4 contents areas of reading, mathematics, science, and English should be considered when using TEAS V as one of the admission criteria; that is, a composite score is recommended instead of a subscore. A successful candidate should possess a composite set of abilities (eg, verbal, quantitative, qualitative, scientific knowledge, etc) to be able to successfully face the multiple challenges of nursing practice.

The relationship of the TEAS assessment score to early nursing program success/persistence was demonstrated in numerous prior studies.17,32,33 This study has gone beyond and proved the incremental effects of TEAS subscores in predicting success on RN Fundamentals examination. A newer version of the TEAS assessment has been released in the summer of 2016. Future research should investigate the incremental predictive effect of the newer version of the TEAS.

Back to Top | Article Outline

Limitations and Future Research

It should be noted that a restriction of range, which is typical of a retrospective research study, exists in the sample of this study. The sample is composed of students who passed TEAS V and completed RN Fundamentals course; students who failed the TEAS V or dropped out before taking the RN Fundamentals are not included because of the retrospective post hoc study design. Thus, findings on the relationship between the TEAS V and RN Fundamentals are inevitably underestimated.

In recent years, holistic admission process has been adopted by nearly 47% of the nursing schools mainly to increase the diversity of student body.34 Under a holistic admission review process, the admission committee considers the student’s life experiences and personal qualities alongside the traditional academic measures. Future research should include more variables of interest including pertinent nonacademic variables that are available to nursing faculty in evaluating the incremental effect in nursing admission decision. It would be interesting to see how multiple academic predictors and nonacademic predictors interact in an incremental manner in the prediction of nursing program success under a holistic review process.

The measures of success used in this study (performances on a standardized admission test) are academic and do not address further measures of nursing success such as communication, empathy, caring, and professional collaboration. These qualities are equally important and contribute to a nursing student’s ultimate success in the workforce.35 But these factors are far more challenging to assess or measure.

Back to Top | Article Outline

Conclusion

Successful job performance in the nursing profession requires a broad range of general academic traits in addition to nursing specific knowledge. Collectively, the findings of this set of studies results in the conclusion that each and every TEAS V subtest contributes incremental validity in the prediction of early nursing school success. Practically speaking, this means that all content areas help to predict performance in early nursing school. Considering time-saving and cost-effectiveness, we hope researchers will continue to investigate the incremental effect of a broader range of holistic admission criteria variables with regard to a broader range of desired outcomes.

Back to Top | Article Outline

References

1. Bureau of Labor Statistics. Registered nurses: job outlook. Available at http://www.bls.gov/ooh/healthcare/registered-nurses.htm#tab-6. Accessed January 2017.
2. Harris RC, Rosenberg L, O’Rourke ME. Addressing the challenges of nursing student attrition. J Nurs Educ. 2014;53(1):31–37.
3. Grossbach A, Kuncel NR. The predictive validity of nursing admission measures for performance on the National Council Licensure Examination: a meta-analysis. J Prof Nurs. 2011;27(2):124–128.
4. McGahee TW, Gramling L, Reid TF. NCLEX-RN success: are there predictors. South Online J Nurs Res. 2010;10(4):208–221.
5. Alameida MD, Prive A, Davis HC, Landry L, Renwanz-Boyle A, Dunham M. Predicting NCLEX-RN success in diverse student population. J Nur Educ. 2011;50(5):261–267.
6. Dunham ML, Alameida M. ATI TEAS V cut score for the California Community College nursing programs. Teach Learn Nurs. 2017;12(2):89–94.
7. Newton SE, Smith LH, Moore G, Magnan M. Predicting early academic achievement in a baccalaureate nursing program. J Prof Nurs. 2007;23(3):144–149.
8. Noble JP, Sawyer RL. Is high school GPA better than admission test scores for predicting academic success in college? Coll Univ. 2004;79(4):17–22.
9. Timer JE, Clauson MI. The use of selective admissions tools to predict students’ success in an advanced standing baccalaureate nursing program. Nurse Educ Today. 2011;31(6):601–606.
10. ATI Nursing Education. Predicting Early Nursing School Performance in an ADN Program: How Do TEAS V Preparedness Levels Predict Fundamentals Performance? Available at http://www.atitesting.com/Libraries/pdf/Research_Brief-TEASVFund_PredEarlyADNProgramSuccess_V1.sflb.ashx. Accessed July 28, 2017.
11. Crow C, Handley M, Morrison R, Shelton M. Requirements and interventions used by BSN programs to promote and predict NCLEX-RN success: a national study. J Prof Nurs. 2004;20(3):174–186.
12. Sicard K. Test of Essential Academic Skills as a Predictor of Bachelor of Science in Nursing Program Completion [dissertation]. Hattiesburg, MS: William Carey University; 2014.
13. Potolsky A, Cohen J, Saylor C. Academic performance of nursing students: do prerequisite grades and tutoring make a difference? Nurs Educ Perspect. 2003;24(5):246–250.
14. Byrd G, Garza C, Nieswiadomy R. Predictors of successful completion of a baccalaureate-nursing program. Nurse Educ. 1999;24(6):33–36.
15. Lewis C, Lewis J. Predicting academic success of transfer nursing students. J Nurs Educ. 2000;39(5):234–236.
16. Wong J, Wong S. Contribution of basic sciences to academic success in nursing education. Int J Nurs Stud. 1999;36(4):345–354.
17. Wolkowitz AA, Kelley JA. Academic predictors of success in a nursing program. J Nurs Educ. 2010;49(9):498–503.
18. Ukpabi CV. Predictors of successful nursing education outcomes: a study of the North Carolina Central University’s nursing program. Educ Res Quart. 2008;32(2):30–40.
19. Sayles S, Shelton D, Powell H. Predictors of success in nursing education. ABNF J. 2003;14(6):116–120.
20. Gallagher P, Bomba L, Crane L. Using an admissions exam to predict student success in an ADN program. Nurse Educ. 2001;26(3):132–135.
21. Haas RH, Nugent KE, Rule RA. The use of discriminant function analysis to predict student success on the NCLEX-RN. J Nurs Educ. 2004;43(10):440–446.
22. Kline TM. Identifying Predictors of Success for Computerized NCLEX-RN in Associate Degree Nursing Graduates [dissertation]. Highland Heights, KY: Northern Kentucky University; 2010.
23. Hopkins TH. Early identification of at-risk nursing students: a student support model. J Nurs Educ. 2008;47(6):254–259.
24. Gilmore M. Predictors of success in associate degree nursing programs. Teach Learn Nurs. 2008;3(4):121–125.
25. Hope LJ. Predicting Success in First-Year Associate Degree Nursing Students [dissertation]. Minneapolis, MN: Walden University; 2015.
26. Walker GC, Klotz L, Martin P, et al. A regional academic partnership for the early identification and retention of at-risk nursing students. J Prof Nurs. 2011;27(6): e8–e13.
27. Seago JA, Keane D, Spetz J, Grumbach K. Predictors of students’ success in community college nursing programs. J Nurs Educ. 2012;51(9):489–495.
28. Sackett PR, Lievens F. Personnel selection. Ann Rev Psychol. 2008;59:419–450.
29. Zwick R. Higher education admission testing. In: Brennan R, ed. Educational Measurement. 4th ed. Westport, CT: American Council on Education/Praeger; 2006:647–680.
30. Abele C, Penprase B, Ternes R. A closer look at academic probation and attrition: what courses are predictive of nursing student success? Nurse Educ Today. 2013;33(3): 258–261.
31. American Psychological Association, National Council on Measurement in Education. Standards for Educational and Psychological Testing. Washington, DC: AERA, APA, NCME; 2014.
32. Cooper SE. Examining Student Achievement and Curriculum in a Nursing Program at a Midwestern Community College [dissertation]. Ames, IA: Iowa State University; 2012.
33. Rogers T. Preadmission Academic Achievement Criteria as Predictors of Nursing Program Completion and NCLEX-RN Success [dissertation]. Morgantown, WV: West Virginia University; 2009.
34. Urban Universities for Health. Holistic Admissions in the Health Professions. Available at http://urbanuniversitiesforhealth.org/media/documents/holisticadmissionsinthehealthprofessions.pdf. Accessed October 20, 2017.
35. Wambuguh O, Eckfield M, Van Hofwegen L. Examining the importance of admissions criteria in predicting nursing program success. Int J Nurs Educ Scholarsh. 2016;13(1):87–96.
Keywords:

academic success; admission criteria; admission tests; nursing education; nursing programs

Supplemental Digital Content

Back to Top | Article Outline
Copyright © 2018 Wolters Kluwer Health, Inc. All rights reserved