Health care facilities should be places of caring and healing, not places of danger. Despite advancements in systems-based technology, approximately 100 000 patients a year continue to be harmed in health care facilities as a result of preventable medication administration errors (MAEs).1 Nurses are often afraid to report mistakes made because health care organizations fail to develop a culture of safety; therefore, identifying and resolving MAEs continue to be an unaddressed part of patient safety.1 The specific problem is that despite efforts to minimize medications errors or potentially eliminate them by using systems-related solutions, nurses are not consistently mindful enough when administering medications and this contributes to errors. In most instances, nurses are sincere in their efforts to provide excellent care and not to expose patients to unsafe health care systems.
Intravenous catheters are generally flushed with a mild solution of heparin to prevent harmful coagulation. In November 2007, while hospitalized in a prestigious California hospital, newborn twins received intravenous flush heparin solution at approximately 1000 times the recommended dose. This MAE caused no long-lasting harm; however, similar cases in Indiana in September 2006 caused the deaths of 3 infants. Medication errors are so commonplace that patients can be subjected to at least one medication error every day of hospitalization.2 Systems approaches, bar codes, and patient identification are examples of effective proactive correction that work to prevent medical errors. Dartmouth scholars determined that a systems approach combined with a range of technological solutions drastically can reduce errors in hospitals.3
Mindfulness and concentration are specialized types of consciousness that could be part of a systems approach to error reduction.3 Concentration occurs when a person stops extraneous thoughts from interfering and obscuring ideas, thinking, questioning, and experiencing personal situations. Taking time to pause, breathe, and dwell on the immediate moment is pure concentration. The Buddhist philosophy emphasizes concentration and mindfulness in 4 steps: (1) taking a long breath; (2) taking a short breath; (3) being aware of breathing in and out; and then (4) calming the sense of breath dwelling within the body. Buddha never detached mindfulness and concentration. The ability to think and stay in the moment, or mindfulness thinking, has the potential to enhance workplace performance through improved communications, enhanced efficiency during meetings, better decision-making capacity, and a greater understanding of problems within an organization. Results of early mindfulness research indicate that interventions using Buddhist-like meditation show promise to improve performance.3
Medication errors occur in many of the delivery steps in providing patient care. These steps include prescribing, documenting, transcribing, dispensing, administering, and monitoring.4 While nurses strive to contribute positively to patient care, mistakes may occur as a result of staff shortages, overwork, distraction, fatigue, and inadequate training. Medication errors occur in health care facilities at an alarming rate, a problem that has created a nationwide concern.4
Nightingale was first recognized as a nurse theorist and an integralist.5 Nightingale was focused on the most basic needs of each patient. Nightingale revolutionized the health care philosophy, curriculum, and nursing processes, and she expanded visionary capacities well beyond the day. The focus on transforming health care and improving patient outcomes through integral nursing began 150 years before the 21st-century theory of integral nursing became the goal for creating the healing environment.5
Three eras of medicine can be identified in the Western world. Era I existed in the mid-1800s when illness was considered strictly physical in nature and the patient's mind was not integrally involved in the disease process or healing. Era II, from the mid-1900s, raised the awareness that the patient's mind had some relationship to the disease state. Era III, the most current era, is entrenched in science and is recognized to be linked to both the body and the mind. Era III medicine requires a willingness to be in the moment, aware, and focused.5 Integral or holistic nursing encourages each nurse to consider self-care, self-development, and a reflection on mind-body-spirit when providing care to patients.6
Research performed by the US Agency for Healthcare Research and Quality summarized that few studies exist regarding MAEs and effective interventions.7,8 Based on randomized clinical trials, good evidence exists that mindfulness training programs are effective for physical conditions, such as mood disorders, depression, pain, and anxiety, and more serious chronic disorders such as multiple sclerosis and cancer.8 More specific studies emerged on cognitive thinking and emotion regulation that focus on whole-body attention and thinking in the moment.9 Mindfulness training translates into positive clinical outcomes.9,10 Miller et al11 call health care professionals who have been involved in a medical error a second victim. Second victims endure stress, burnout, and powerlessness. These symptoms may affect the second victims on professional and personal levels. In 2019, a study conducted by Miller et al11 indicated that mindfulness interventions positively impact stress and burnout and improve self-compassion. Future research is needed to establish mindfulness-based interventions' effectiveness on second victims.
A powerful tool, corporate mindfulness, is not a new concept as it is found in the Buddhist tradition. Mindfulness training has become a common activity in business leadership and management in companies such as Toyota, Disney, and Google.12 Mindfulness in business focuses primarily on 2 concepts: (1) self-awareness; and (2) a focus on the present moment. As little as 10 minutes of practicing mindfulness a day is said to make a difference.10
A study conducted at the National Institutes of Health explored mind and body research for chronic pain. Research showed modest pain relief received from mindfulness.13 The mind and body connection also considered yoga, acupuncture, and biofeedback as remedial steps for pain relief.13
The holistic nursing paradigm supported the framework for teaching integral nursing, which enlarges a nurse's understanding of the mind-body-spirit concept and connection. This is patient-centered care and healing of the whole person by developing a “capacity of mindfulness.”14(p69) The term “integral” refers to a way to organize a phenomenon of human experience and reality into 4 areas (I, It, We, and Its): (1) the individual interior (I—personal/intentional); (2) individual exterior (It—physiology/behavioral); (3) collective interior (We—shared/cultural); and (4) collective exterior (Its—systems/structures).14 An example of the 4-quadrant perspective can be demonstrated in a case where a patient is taken to the surgical suite. The nurse uses the 4 perspectives as follows: (1) interior self—the nurse cares for one's self in the moment to engage in emotional maturity, beliefs, and values; (2) exterior self—the nurse uses a form of relaxation to change his or her physiology from brain knowledge to measureable, quantitative behavior; (3) self in relationship to others—the nurse shifts awareness to the patient and the surgical team for a shared vision and a leadership task; and (4) system structures—the nurse works with the surgical staff to improve patient quality care procedures within the hospital. These 4 quadrants create and recreate the whole paradigm and encourage creativity toward best practices14 (Figure 1).
Dossey and Keegan's work15 is a tapestry of views that embrace how we define, know, experience, and respond to our realities and how our new understandings of our realities can influence our actions. The Dossey framework is designed to complement principles of holistic nursing, human caring, and, most of all, healing. Dossey and Keegan16 provide a framework for the application of principles of direct care, education, and research.
Learning to be a mindful thinker requires the conscious removal of the autopilot button, an experience that will offer the awareness of what is here and now, a mindful compass.17 Mindfulness is a naturally occurring attribute that varies from person to person. Mindful thinking is one of the more successful strategies used to decrease nurse anxiety in the clinical setting.18 Several meta-analysis research studies on mindfulness were conducted for chronic pain, behavioral problems, and well-being in the adult population.19 The enhancement of mindfulness is associated with various well-being outcomes due to the growth in emotional intelligence.19
Traditional mindfulness thinking
Most nursing schools administer some type of cognitive thinking assessment to student nurses to test their thinking skills: evaluation, inference, and analysis.20 Cognitive psychologists suggested that most people use shortcuts in thinking that may perpetrate errors known as cognitive biases.20 Nursing students must be aware of this cognitive bias known as the Dunning-Kruger effect or inflated self-assessment of cognitive skills and be prepared properly to assess decision-making ability, reading limitations, ability to recognize errors, and not overestimate knowledge and skills in performing nursing duties.20 Since 1997, more than 240 programs offering mindfulness thinking training have been offered to various employees of organizations and are expanding exponentially.21
Controlled breathing skill allows the person to learn to concentrate on one function and become fully aware of that mental discipline. A philosopher, Patañjali, the author of Yoga-Sutras, realized that breathing had a close relationship with the mind.22 When the mind was agitated or excited, a shortness of breath or irregular breathing occurred.22 The author of Beyond Mindfulness in Plain English: An Introduction Guide to Deeper States of Meditation recommended to “start with focusing your undivided attention on your breathing to gain some degree of basic concentration.”22(p45) In 1992, Thich Nhat Hanh stated, “Mindfulness is the miracle by which we master and restore ourselves ... it is the miracle which can call back in a flash our dispersed mind and restore it to wholeness so that we can live each minute of life.”23(pp20-21) Training typically commenced with the breathing exercises to develop the mindful awareness of a basic concentration and control. In 2008, Dhiman stated, “Be where you are with all your mind.”24(p36)
In the Buddhist form of meditation, the theory of vipassana, the contemplation of mindful breathing, leads to the development of insight and analysis of the body.25 Mindfulness of the body and mind identifies enlightenment factors of tranquility, concentration, and equanimity. Harvey stated, “Mindfulness investigates hindrances such as ill will, drowsiness, restlessness, worry, and self-doubt.”25(p255)Vipassana meditation promotes viewing reality with focus and clarity. Harvey25 called this meditation a purifying process in which to work with a calm, centered, and quiet spirit ready to focus on the moment.
Contemporary mindfulness thinking
The consequences of negative stress among nurses are well known, and the ways to reduce this stress have yet to be resolved. Mindfulness-based stress reduction (MBSR) programs are becoming more accepted in the health care arena as a way to deal with a variety of clinical problems.26 An abbreviated mindfulness thinking training intervention program was developed and administered to 51 nurses in the test group of this research.
Gaps related to mindfulness thinking as a strategy to reduce medical errors
Health care leaders, nursing directors, and nursing staff continually attempt to improve patient safety. A recent study on nurse education programs illustrates that research based on classroom training, supplemented by high-fidelity human simulation, demonstrates improved training.27 Studies do not show in-depth training on mindfulness thinking.
A gap in literature exists in identifying the prevention of MAEs through the process of teaching mindfulness thinking to individual nursing staff and nursing leaders. An additional gap exists in the measurement of patient outcomes before and after the training of mindfulness thinking. Current methods for mindfulness thinking training resemble the training on cognitive thinking already found in nurse education programs; mindfulness thinking training is more in-depth, personalized to each nurse, and measurable.
This mindfulness research addressed whether the use of mindfulness thinking diminished the incidence of MAEs and the severity of those errors. The mindfulness thinking training (STOP—mindfulness training method) provided the test group subjects with an overarching holistic nursing theory. Permission to use the Four Quadrants Theoretical Model was granted. The control group, however, did not receive the mindfulness training; they were not educated on the holistic nursing paradigm. A total of 132 nurses agreed to participate in the study. Because of attrition, 111 nurses completed the study. The research proposal was evaluated and approved by an academic institutional review board prior to data collection. Senior management monitored that the project was being ethically administered. Identification of participants and the health care system remains anonymous.
Sample and setting
Three hundred forty nurses from a hospital in rural northern California were invited to participate in the study. Each nurse signed a consent form and volunteered to participate. The inclusion criteria were as follows: (a) nurses who were registered or certified by a nationally accredited agency; (b) nurses who had experience working with critically ill patients in a health care facility for a period of at least 3 months administering medications; and (c) nurses who self-reported adverse or near-miss incidences of MAEs. Exclusion criteria were nurses who had not worked at the venue hospital for at least 3 months because those nurses would not have historical MAE data.
The intervention program involved training in mindfulness thinking by providing a 2-hour CD training tape prepared by a psychologist on mindfulness for each nurse in the test group. A manual summarizing the key points of mindfulness thinking for daily practice was supplied. The control group did not receive mindfulness thinking training. Instead, they received a recently published article on the impact of emerging technologies in nursing care. The control group was asked to e-mail the researcher when finished reading the article. Each member of control group described one new emerging technology from the article. Participants in the 2 groups were purposefully chosen and alternately assigned. The 2 groups were considered comparable prior to the administration of the treatment based on similar educational backgrounds, medical certifications, and job experience. Any significant differences after the treatment were strong evidence that the intervention caused that difference. The 2 groups were monitored for 3 months for the self-reported frequency and severity of MAEs.
Fronsdal28 concluded that mind is not in a static state but is malleable, pliable, and shaped by cognitive thinking training. In English, this is known as insight meditation. This practice combines mindfulness with focused attention.28 For nurses, mindfulness may reveal mental conflicts with others and the world around that prevent total focus on the job. Buddhist-like meditation encourages quiet thinking when assuming responsible activities. Performing duties accurately is often negatively influenced by worry, distraction, suppressed anger, despair, or discouraging thoughts of external conflicts. Rather than trying to control conditions of the external world, nurses can create calm in whatever situation is happening. Mindfulness is learning to create mental stability and discipline. When mindfulness and focused attention work in harmony, full concentration is present in the moment.28
The intervention training used for this research was called the STOP program. The STOP program has 2 meanings. STOP stands for “stop and think of the patient.” STOP also means “Self-acceptance of becoming present in the moment,” “Teaching mind-mapping to accept working with the distractions that narrow thinking and actions,” “Opening the mind to accept and redirect thinking to deal with strong emotions and difficult experiences that keep the mind from totally focusing on the present,” and “Practicing meditation to deepen the habit of learning to accept inner peace.”29
Therapeutic training programs, such as the STOP program, used as an intervention that incorporates mindfulness are becoming more popular. The results suggested that the MBSR changes regions of the brain, tests participants use in learning and memory processes, emotional regulation, and self-perspective.30 Mindfulness-based interventions are used more readily in health care education.30 Mindfulness can be practiced without Buddhism. Buddhism cannot be practiced without some form of mindfulness activity.31 The intervention mindfulness thinking training program for this study had 3 overarching purposes: (a) knowing the mind; (b) training the mind; and (c) freeing the mind.28
After the pre- and posttraining data were checked for normal distribution, descriptive statistics summarized the demographic information. Of 111 participants, 19 were male and 92 were female. Ages ranged from 20 through 70 years, with the 20 to 30 years age category being the majority. Most nurses in the study were married. The majority of nurses' educational level was Associate of Arts (54.1%), followed by Bachelor of Science (37.8%). The majority of the participants worked the day shift and full-time. Approximately 35% of the nurses worked on the medical/surgical floor.
There were 51 nurses in the treatment group who received mindfulness training and 60 nurses in the control group who received no mindfulness training. Results analysis used paired t tests (within-group analysis) from 51 nurses to measure the pre- and post–mindfulness training on the number and severity of the error outcomes. In the treatment group, errors were reduced from 15 (pre) to 4 (post). This was a 73.3% decrease in errors. The control group showed a reduction from 7 errors to 6. Of the 9 demographic variables, only gender and education level proved to be significantly related to the errors and included as covariates in the t-test analysis to increase accuracy.
The instrument used for the study was based on the medication error index from the National Coordinating Council for Medication Error Reporting (NCC MERP). Characteristics of the NCC MERP index are shown in Figure 2. Permission to use the NCC MERP index chart was granted. The instrument is designed to categorize medication errors into 9 levels of severity. As discussed in the Definitions of Terms for the severity of medication errors, mistakes are categorized into the following groups that correspond to mild severity, moderate severity, significant severity, and severe severity: (a) there was a nonmedication error or a near-miss error; (b) a mistake occurred but was caught before it reached the patient; (c) a mistake occurred that reached the patient but caused no damage to the patient, only increased monitoring of vital signs; (d) a mistake happened and reached the patient, which changed the vital signs and increased monitoring of vital signs, but no patient harm occurred and no laboratory tests were needed for monitoring the patient; (e) a mistake occurred that resulted in the need for treatment with additional drugs; (f) a mistake occurred that resulted in an increased hospital stay and temporary patient harm was done; (g) a mistake occurred that caused the patient to receive permanent harm; (h) the incident required emergency intervention to sustain life; and (i) a mistake caused the death of the patient.32
The NCC MERP index assists health care providers to monitor medication errors in a consistent manner. The index reflects factors such as whether the medication error reached the patient, and if the patient was injured by the error, to what extent or degree. A study using the NCC MERP instrument showed substantial statistical agreement and interrater reliability.31 Chang et al32 reported that the interrater comparison was consistent; terminology of harm categories was clear and concise, permitting accurate reporting of data. (Figure 2).
The specific problem in this study was that despite efforts to minimize medication errors or potentially eliminate them by using systems-related solutions, nurses are not consistently mindful enough when administering medications; this contributes to errors. Nurse mindfulness thinking may lead to improved patient care. Effective training in mindfulness thinking to help nurses stop and think of the patient, and focus clearly on the task at hand, could prove to diminish medication errors.
Nursing behavior may be altered because of the attention subjects receive from the researchers, rather than because of any manipulation of dependent variables. This is known as the Hawthorne effect. There is no evidence that this phenomenon took place in this research; instead, a decrease in errors may be attributed to the educative effect of the intervention.
The SPSS v25 analyzed the data, generated statistics, and tested the hypothesis of the study.33 The level of significance was set at P < .05. All tests were generated and tested.
An independent t test was conducted in order to answer and test the following research questions and hypotheses:
- RQ1. What is the difference in the number of medication administration errors between group types (control, experimental) after controlling for the number of preintervention medication administration errors?
- H10: Group A (experimental group) will not have a statistically significantly lower number of medication errors than group B (control group) for the 3-month period after mindfulness thinking training, after controlling for the number of medication errors during the 3-month period before thinking training.
- H11: Group A (experimental group will have a statistically significantly lower number of medication errors than group B (control group) for the 3-month period after mindfulness thinking training, after controlling for the number of medication errors during the 3-month period before mindfulness thinking training.
- The experimental group showed a statistically significant difference in the mean number of errors from pre- to posttraining (t50 = 2.671, P = .010). The control group did not show any statistically significant difference in mean errors from pre- to posttraining (t59 = 0.00, P = 1.00). The null hypothesis for RQ1 was rejected.
Logistic regression conducted with SPSS answered and tested the following RQ2 and hypothesis:
- RQ2. What is the difference in the severity of medication errors between group type (control, experimental) after controlling for the level of preintervention medication error severity?
- H20: Group A (experimental group) will not have a statistically lower average of severity of medication errors than group B (control group) for the 3-month period after mindfulness thinking, after controlling for the severity of medication errors during the 3-month period before mindfulness thinking training.
- H21: Group A (experimental group) will have a statistically significantly lower average of severity of medication errors than group B (control group) for the 3-month period after mindfulness thinking training, after controlling for the average severity of medication errors during the 3-month period before mindfulness thinking training.
Because of the rarity of a severe event, there were only 3 severe events out of the 111 observations. Because of the small sample size, bootstrapping was performed. Bootstrap provides a way to draw a conclusion about a population from a small sample.34 After controlling for preintervention errors of gender and education levels, group type (P = .002) was found to be statistically significant. The control group was more likely to have severe errors (18.003). The null hypothesis for RQ2 was rejected.
Both null hypotheses for RQ1 and RQ2 were rejected concluding that the test group did show a relationship after the mindfulness training intervention. A decrease of 73.3% in the number of MAEs was observed with the test group. The control group did not demonstrate a significant decrease in errors or severity of errors.
Because of the nature of the study to monitor only nurses, and the fact that 50% of the errors originated in the pharmacy, further research needs to be conducted. This limitation did not adversely affect the nursing study as the errors were attributed equally between the treatment and control groups. To gain a clearer picture of patient errors, a future study should include training pharmacy personnel on mindfulness.
Teaching mindfulness thinking to 51 nurses showed a relationship in the decrease of MAEs. Mindfulness thinking programs help nurses think in the moment and minimize errors. Mindfulness-based programs enable nurses to increase overall attention to daily tasks, be present in the moment with patients, and decrease costly errors. Since mindfulness needs to be learned and practiced regularly, routine classes should be taught as part of orientation and ongoing education. The potential for nurse leaders to learn to use mindfulness has value in today's health care settings.
1. Carthon JM, Lasater KB, Sloane DM, Kutney-Lee A. The quality of hospital work environments and missed nursing care is linked to heart failure readmissions: a cross-sectional study of US hospitals. BMJ Qual Saf. 2015;24(4):255–263.
2. Roughead EE, Semple SJ, Rosenfeld E. The extent of medication errors and adverse drug reactions throughout the patient journey in acute care in Australia. Int J Evid Based Healthc. 2016;14(3):113–122.
3. Crane J, Crane FG. Preventing medication errors in hospitals through a systems approach and technological innovation: a prescription for 2010. Hosp Top. 2006;84(4):3–8.
4. da Silva BA, Krishnamurthy M. The alarming reality of medication error: a patient case and review of Pennsylvania and national data. J Community Hosp Intern Med Perspect. 2016;6(4):31758. doi:10.4502/jchimp.v6.31758.
5. Dossey BM. Holistic Nursing: A Handbook for Practice. 5th ed. Sudbury, MA: Jones & Bartlett; 2008.
6. Klebanott N, Hess D. Holistic nursing: focusing on the whole person. Am Nurs Today. 2013;8(10):9.
7. Raban M, Westbrook J. Are interventions to reduce interruptions and errors during medication administration effectives: a systematic review. BMJ Qual Saf. 2013;2(1):1–8.
8. Gaylord SA, Palsson OS, Garland EL, et al. Mindfulness
training reduces the severity of irritable bowel syndrome in women: results of a randomized controlled trial. Am J Gastroenterol. 2011;106(9):1678–1688.
9. Williams JMG, Duggan DS, Crane C, Fennell MJ. Mindfulness
-based cognitive therapy for prevention of recurrence of suicidal behavior. J Clin Psychol. 2006;62(2):201–210.
10. Kabat-Zinn J. Guided Mindfulness
Mediation [audio CD]. New York, NY: Sounds True; 2005.
11. Miller CS, Scott SD, Beck M. Second victims and mindfulness
: a systematic review. J Patient Saf Risk Manage. 2019;24(3):108–117. doi.org/10.1177/2516043519838176
12. Liker JK. The Toyota Way: 14 Management Principles From the World's Greatest Manufacturer. New York, NY: McGraw Hill; 2004.
13. National Institutes of Health. Mind and body approaches for chronic pain. https://nccih.nih.gov/health/providers/digest/ChronicPain-mindandbody
. Accessed February 3, 2000.
14. Dossey BM. Theory of integral nursing. ANS Adv Nurs Sci. 2008;31(1):E52–E73.
15. Dossey BM, Keegan L. Theory of integral nursing. In: Holistic Nursing: A Handbook for Practice. 5th ed. Burlington, MA: Jones & Bartlett; 2009:3–45.
16. Dossey BM, Keegan L. Theory of integral nursing. In: Holistic Nursing: A Handbook for Practice. 6th ed. Burlington, MA: Jones & Bartlett; 2012:25–53.
17. Kume J. Meditation and mindfulness
in the peer-reviewed literature: review. J Yoga Physiother. 2018;5(4):1–3.
18. Penque S. Mindfulness
to promote nurses' well-being. Nurs Manag. 2019;50(5):38–44. doi:10.1097/01.NUMA.0000557621.42684.c4.
19. Keng S, Smoski MJ, Robins CJ. Effects of mindfulness
on psychological health: a review of empirical studies. Clin Psychol Rev. 2011;31(6):1041–1056.
20. Dunning D. The Dunning-Kruger effect: on being ignorant of one's own ignorance. Adv Exp Soc Psychol. 2011;44:247–287.
21. Ott M, Norris RL, Bauer-Wu SM. Mindfulness
mediation for oncology patients: a discussion and critical review. Integr Cancer Ther. 2006;5(2):98–108.
22. Guarantano BH. Beyond Mindfulness
in Plain English: An Introduction Guide to Deeper States of Meditation. Somerville, MA: Wisdom; 2009.
23. Hanh TN. Touching Peace: Practicing the Art of Mindfulness
. Berkeley, CA: Parallax Press; 1992.
24. Dhiman S. Cultivation mindfulness
: the Buddhist art of paying attention to attention. Interbeng. 2008;2(2):35–52.
25. Harvey P. An Introduction to Buddhism: Teaching, History, and Practices. Cambridge, MA: Cambridge University Press; 1990.
26. Jensen CG, Vangkilde S, Frokjaer V, Hasselbalch SG. Mindfulness
training affects attention—or is it attentional effort? J Exp Psychol Gen. 2012;141(1):106–123.
27. Erlinger J, Redinger WO, Kim B. Decision-making and cognitive biases. Encyclopedia of Mental Health https://www.researchgate.net/publication/301662722_Decision-Making_Cognitive_Biases
. Published 2016. Accessed September 30, 2019.
28. Fronsdal G. Mindfulness
mediation as a Buddhist practice. http://www.insightmeditationcenter.org/books-articles
. Published 2006. Accessed September 30, 2019.
29. Hölzel BK, Carmody J, Vangel M, et al. Mindfulness
practice leads to increases in regional brain gray matter density. Psychiatry Res. 2011;191(1):36–43.
30. Hassed C, deLisle S, Sullivan G, Pier C. Enhancing the health of medical students: outcomes of an integrated mindfulness
lifestyle program. Adv Health Sci Educ Theory Pract. 2009;14(3):387–398.
31. deZoysa P. The use of Buddhist mindfulness
meditation in psychotherapy: a case report from Sri Lanka. Transcult Psychiatry. 2011;48(5):675–683.
32. Chang H, Mazzso J, Roussos L. Detecting DIF for polytomously scored items: an adaptation of the SIBTEST procedure. J Educ Meas. 1996;33(3):333–353.
33. Levesque R. Programming and data management for IBM statistics 20: a guide for IBM SPSS ad SAS users. https://www.ibm.com/products/spss-statistics
. Published 2011. Accessed September 30, 2019.
34. Streukens S, Leroi-Werelds S. Bootstrapping and PLS-SEM: a step-by-step guide to get more out of your bootstrap results. Euro Manag J. 2016;34(6):618–632.