Secondary Logo

Journal Logo

Original Article

Evaluation of skills acquisition using a new low-cost tool for CPR self-training

Sá-Couto, Carla PhDa,b,c,e,∗; Ferreira, Ana Margarida PhDb,d,e; Almeida, Diana MSce; Nicolau, Abel BSca,b,f; Vieira-Marques, Pedro PhDb,d,e

Author Information
Porto Biomedical Journal: August 2018 - Volume 3 - Issue 1 - p e8
doi: 10.1016/j.pbj.0000000000000008

Abstract

Introduction

In sudden cardiac arrest, early cardiopulmonary resuscitation (CPR), with emphasis on chest compressions, is a fundamental step in the Chain of Survival.1 According to the 2015 European Resuscitation Council CPR guidelines for the adult,1 when performing chest compressions, it is important to take into account parameters such as hands position (center of the chest), compression depth (5–6 cm), compression rate (100–120 per minutes), and chest wall recoil (allow complete recoil). The chest compressions should not be suspended for more than 10 seconds to provide ventilations.1

High-quality CPR remains essential to improve patient outcome. Several studies have shown that CPR is performed ineffectively, possibly due to irregular training and low skill retention.2–4

The traditional acquisition and retention of skills in CPR is based in a theoretical and practical training on a mannequin or task-trainer, given by 1 instructor according to the following sequence: theoretical background in CPR, chain of survival, correct CPR performance with emphasis on chest compressions, automated external defibrillator use, and correct positioning of the victim after recovery.5,6 The training includes a standardized assessment of performance (with feedback given by the instructor), either during the training in the form of continuous assessment, or at the end of the training during which the key learning outcomes have to be successfully demonstrated.5,6

Although accurate theoretical knowledge of guidelines and CPR procedures is associated with increased odds of correct performance of some aspects of CPR, overall performance remains poor7 suggesting that frequent training may be needed to ensure consistent, effective performance and therefore better outcomes after cardiopulmonary arrest.8

Training models based on short videos of self-learning may be used combined with practical training sessions with minimal intervention from instructors. Other alternative that seems to improve CPR skills of rescuers is the use of automated voice advisory mannequin that is able to improve skills through continuous verbal feedback during individual CPR training without an instructor.4

Feedback on performance is a crucial component of the learning processes associated with simulation and has been shown to improve CPR quality during simulated cardiac arrest on mannequins.9 Feedback technology supports the rescuer with vocal or visual information on CPR quality to improve guideline adherence.3

Educational feedback also appears to slow the decay of acquired skills and allows learners to self-assess and monitor their progress toward skill acquisition and maintenance. Sources of feedback may either be “built-in” on a simulator, given by an instructor in “real time” during educational sessions, or provided post hoc by viewing a videotape of the simulation-based educational activity.10

There is evidence that instructors provide poor feedback, including correction of skills, in performing chest compressions.11 Previous studies12–18 have compared the quality of CPR training with feedback devices with the traditional CPR training. The outcomes of these studies confirm that some feedback systems improve significantly the quality of chest compressions, in a simulated cardiac arrest scenario. These systems may provide complementary training19–21 or assessment22 models, overcoming constraints, such as time and logistical/financial aspects or instructor availability, that are pointed out as barriers to frequent training.

Considering the above, the general aim of this study is to evaluate skills acquisition using a new low-cost device for CPR self-training with automated feedback and compare it to the standard training method. The prototype is expected to facilitate regular training and improve long-term preservation of knowledge and skills.

Methods

Prototype description: CPR Personal Trainer

CPR Personal Trainer consists in a standard CPR training mannequin instrumented with off-the-shelf piezoresistive sensors connected to an electronic preprocessing unit and information system. The total cost of the add-on is around €150. The signal is analyzed by extracting relevant data of chest compressions performance and scoring them on 3 different factors associated with compression quality: hands position, compression rate, and chest recoil. The compressions depth is still under development and will be included in a near future.

The software is connected to a user-friendly online Graphical User Interface, which manages training workflow and provides visual and audio feedback. CPR Personal Trainer software gathers trainees’ performance metrics and provides a performance analysis with suggestions to improve the procedure. It also provides reports for each training session, the overall progression along the sessions and the performance evolution of the trainee regarding each CPR maneuver component. Figure 1 presents the CPR Personal Trainer interface representing 2 training sessions performed by a student.

F1
Figure 1:
Print-screen of the cardiopulmonary resuscitation personal trainer interface. Feedback to trainee is presented visually and numerically. Top: Feedback indicates a need for improvement in all parameters (red labels). Visual and audible aids are available to help the hands position and the compressions rate, respectively. Bottom: Feedback indicates all parameters were correctly performed (green labels).

Subjects

The target population of this study was pregraduated students of Medicine and Biomedical Engineering from the University of Porto, Portugal. The recruitment of medical students was restricted to basic years in order to avoid students with CPR training. Students were invited to participate in the study, through announcements and posters from the students’ associations, and were invited to voluntarily register for the study. Only registered students constituted the sample of this study. Demographic information of the students was collected at registration.

Study design

This longitudinal double-blinded randomized control study was carried out in May and October 2016, at the Biomedical Simulation Center of the Faculty of Medicine, University of Porto. The study was approved by the Ethical Commission of our institution and a written informed consent was obtained from all participants before the study.

On Day 1, registered students who were present for the study were given information and clarifications about the study and an informed consent to read and sign. Of the 55 registered students, only 42 attended Day 1. The 42 students were subjected to the pretest evaluation, including both theoretical and practical components. After the evaluation, students were randomized and allocated to the control (n = 23) or the intervention group (n = 19). The study was blinded, meaning that students have no knowledge of the other group type of training. On Day 1, students also received the 2015 CPR guidelines1 and algorithm and were advised to review them.

On Day 8, the control and intervention groups were divided into smaller subgroups of 4 or 5 students. The control group trained with a standard torso and received instructions and feedback from an experienced Basic Life Support (BLS) instructor (Fig. 2, top). The intervention group trained with the prototype, receiving feedback on CPR relevant parameters: hands position, rate, and recoil, from the CPR Personal Trainer interface (without instructor) (Fig. 2, bottom). Two aids were included in the CPR Personal Trainer to guide the student in the training: an audio sample of a metronome at 100 bpm to help in the compression rate and an image of the correct hands position on the chest. These aids were available to the student during the training exercise, together with the feedback of his/her performance. Both groups had the 2015 CPR algorithm displayed, near the training station. The training time was equivalent for both groups and lasted approximately 1 hour for each subgroup of students. Students from the intervention group also answered a questionnaire on the usability of the CPR Personal Trainer. There were 3 dropouts, 2 in the intervention group (n = 21), and 1 in the control group (n = 18).

F2
Figure 2:
Training sessions. Top: Control group—training with an instructor. Bottom: Intervention group—training with cardiopulmonary resuscitation personal trainer (no instructor).

On Day 15, all students who were present for the training on Day 8 (n = 39) were subjected to the post-test evaluation, which included similar theoretical and practical components to the pre-test.

The detailed study protocol is graphically represented in Figure 3.

F3
Figure 3:
Study flow diagram.

Participants’ assessment

The knowledge and technical skills of the participants were evaluated through a theoretical and practical test, before and after the CPR training course.

Theoretical tests

The theoretical tests consisted of 14 different questions about CPR, with emphasis on chest compressions. The type and number of questions were defined by an experienced CPR instructor, based on the 2015 CPR guidelines, guaranteeing that all relevant aspects of the BLS algorithm were covered. Theoretical tests were made available using a Moodle platform (http://www.moodle.org). Each student accessed the platform using their individual credentials and answered the test online, in an examination room, with an instructor present to avoid plagiarism. The test was only available for the period of the evaluation. The pre- and post-tests were similar in content but the questions and answers were randomly ordered. The maximum test score was 100%.

Practical tests

The practical tests were performed individually and consisted in executing 2 minutes of the BLS algorithm. All performances were video recorded. The evaluation of the correct application of the BLS algorithm and correct CPR performance of each student was made through visualization of each video by an independent expert, blinded to the study, with the use of a checklist.

The checklist consisted of 10 items related to the BLS algorithm, 4 of them referring to compression-related parameters: call for help, check normal breathing, compressions-to-ventilations ratio of 30:2, compression rate, hands position, chest recoil, compressions depth, 2 deep breaths, pause for breaths of <10 seconds, and head extension. Each item was rated as 0—incorrect/not applied, 1—insufficient/incomplete, and 2—correct. The total score of the practical test was calculated as the mean of the 10 items with a maximum value of 2.

Usability questionnaire

A preliminary assessment of the CPR Personal Trainer's usability was performed with the students from the intervention group (n = 21). For that, an European Portuguese translation23,24 of the System Usability Scale (SUS) questionnaire25 was applied.

The SUS questionnaire is a reliable tool for usability assessment, consisting of 10 questions, to be rated in a 5-point Likert scale (from strongly agree to strongly disagree), among which 5 of those questions have a positive height and the remaining a negative height. This questionnaire has been considered to be a remarkably robust measure of system usability,26–28 even with a small sample size.27

The SUS score23–25 is calculated by summing up all individual items. For items 1, 3, 5, 7, and 9 (positive statements) the score contribution is the scale position minus 1. For items 2, 4, 6, 8, and 10 (negative statements), the contribution is 5 minus the scale position. The overall value for each student is obtained by multiplying the sum of the scores by 2.5. The global SUS score has a range of 0 to 100 and represents the median of the sample. A global score higher than 70 is considered above the average and higher than 80 is considered good.29

The SUS questionnaire also included 2 open-questions on the “positive” and “need for improvement” aspects of the CPR Personal Trainer. The questionnaire was available on the Moodle platform (http://www.moodle.org) and, similarly to the theoretical tests, each student accessed the platform using their individual credentials and answered the test online, in a supervised examination room.

Statistical analysis

Statistical analysis was conducted using the IBM SPSS Statistics software, version 23.0. Both descriptive and inferential analyses were performed. To evaluate knowledge and skills differences, intra- and intergroups’ nonparametric tests were used, considering a significance level of 5%. Given the small sample size nonparametric tests were used. Specific tests are indicated in the tables with the results.

Total scores of the pre- and post-theoretical tests and total and partial scores of the pre- and postpractical tests were calculated and compared. For the practical tests, partial scores representing the compression-related parameters and other parameters of the CPR algorithm were calculated, analyzed, and are presented separately.

Results

A total of 39 individuals (25 females and 14 males) constituted the sample of this study. The mean age of participants was 20.6 ± 2.4 years. Both groups (control and intervention) were compared for age and gender and no statistical differences were found between groups, confirming its homogeneity. For a detailed description of the demographic data, we refer to Table S1 (Supplemental Digital Content, https://links.lww.com/PBJ/A0).

Assessment results

Theoretical and practical tests in both groups presented an increase in the total score between the pre- and the post-test, with statistical significant differences (Table 1). The intergroup comparison shows no statistically significant differences in all scores, with the exception of the practical post-test, with a higher score in the control group.

T1
Table 1:
Comparison between the pre- and post-test mean scores, for the theoretical and practical tests

The pre- and post-test comparison for the overall and specific compression-related parameters presents a statistical significant increase, in both groups. No statistical differences were observed in any comparison between groups. Table 2 presents the detailed results and comparisons. The intervention and control groups presented identical mean differences for the total score (0.72 vs 0.72), with no statistical difference (P = 0.754, Mann–Whitney U test).

T2
Table 2:
Comparison between pre- and post-test compressions related measurements mean scores

The pre- and post-test comparison for the overall and the CPR parameters not directly related to compressions present a statistical significant increase, in both groups. Comparison between groups, revealed statistical significant differences in the post-test scores for the “shout for help,” “check breathing,” “head extension,” and overall. Table 3 presents the detailed results and comparisons.

T3
Table 3:
Comparison between pre- and post-test mean scores of cardiopulmonary resuscitation parameters not directly related to compressions

Usability results

The intervention group (n = 21) answered the SUS questionnaire, including 2 open questions on the most positive and need improvement aspects. The individual scores were calculated for each student and, from those, the global score (median of the sample). The global SUS score of the CPR Personal Trainer was 78.

Aspects pointed by the students as most positive were “Immediate feedback,” “Rapid learning curve,” and “Feedback on compressions performance.” As for the aspects that need improvement, “No compressions depth assessment,” “No ventilation parameters assessment,” and “No information on the algorithm” were pointed as the most relevant.

Discussion

This study investigates and compares the acquisition of knowledge and technical skills related to chest compressions in CPR maneuvers, when using an automated CPR feedback tool and when receiving standard CPR training.

The theoretical tests showed an improvement in the scores from the pre-test to the post-test, both in the intervention and in the control groups. Although the intervention group had no theoretical support from an instructor, the content made available, revealed to be sufficient to increase their knowledge to a similar level as for the control group, as no statistical significant differences were observed between the groups.

The practical tests (total score) showed an improvement between the pre- and the post-tests for both groups, although the control group presented a higher mean difference (0.63 vs 0.97) with statistical significance. The higher improvement in the control group is related to the fact that this practical total score includes all the CPR parameters, including the parameters that the CPR Personal Trainer did not provide feedback about (such as ventilation-related parameters or compression depth). For each compression-related parameters (hands position, recoil, rate, and depth), significant increase in scores was observed, between the pre- and the post-tests, in both groups. The intervention and control groups presented identical mean differences for the total score (0.72 vs 0.72), with no statistical difference (P = 0.754). This result corroborates our initial hypothesis, indicating that the CPR Personal Trainer is as valuable as the traditional method in the acquisition of skills.

An interesting result was observed for the depth parameter, where similar improvement was observed in both groups, although the CPR personal trainer did not provide feedback on compressions depth. This improvement can be attributed to the relationship between depth and recoil, as providing feedback for the later may have an indirect impact on depth performance.

Considering the practical test results for the parameters not related to compressions, an overall improvement was observed in both groups, but with a greater difference in the control group (0.66 vs 1.12). Of mention is that, in the “shout for help” parameter, the control group scored in the post-test 2.0 with a standard deviation of 0.0, indicating that all students in this group correctly executed this step. Comparison between groups, revealed statistically significant differences in the post-test scores for the “shout for help,” “check breathing,” “head extension,” and overall, with higher scores in the control group. This notorious improvement in the control group in parameters not related to compression was expected and justified by the involvement of the instructor in the training. Nevertheless, in the intervention group, even without an instructor and with prototype-only feedback on these parameters, all mean scores increased from the pre-test to the post-test, with statistical significance. This improvement can be attributed to the knowledge acquired through the educational materials provided.

The usability of the CPR Personal Trainer scored 78, which, based on the literature,23,29 is considered above average.

It was observed that CPR Personal Trainer is effective when compared with the traditional method. The intervention group showed improvements regarding skills’ acquisition and the CPR Personal Trainer allowed to achieve a similar level of knowledge when compared with the control group. These results are in line with others that also showed that feedback systems are able to contribute to a better CPR performance.13,30

The proposed system can encourage frequent training sessions, as it may reduce the need for a permanent instructor, specifically for the training of chest compressions, allowing regular training outside formal training courses, not only in healthcare institutions but also in other places like firefighters departments or nursing homes.

Limitations

An important limitation was the reduced sample. Due to time constraints, students’ availability, and other logistic restrictions, the sample used was limited which may weaken the conclusions of this work.

Another potential bias can emerge from the practical evaluation, which was performed by a single evaluator. Although the evaluator was blind to the study and used the video recordings and a standard checklist, the intrinsic subjectivity of the human nature could be reduced if other evaluators were included.

Finally, an important limitation is related to the under developing features of the CPR Personal Trainer, namely the assessment of the compressions depth and ventilations. This leads to an incomplete assessment of the effectiveness of CPR Personal Trainer in the CPR training. Nevertheless, the present study can be seen as a proof of concept, confirming which future developments are more pressing.

Conclusions

Tools for self-guided CPR training are an alternative to traditional training, which requires an instructor, as they can offer the opportunity of learning and continuous training to a broader population, with an associated low cost.

CPR Personal Trainer is a low-cost simulation-based tool for CPR training with a feedback system that improves technical skills and reinforces knowledge. The repetitive self-training allows the trainees to improve skills at their own pace. The feedback feature provides objective, reliable, and standardized assessment of skills acquisition and, as opposed to the subjective evaluation of an instructor, potentially boosting the trainee's confidence in performing CPR correctly. Despite some limitations, this tool proved to be effective in the acquisition of compression-related skills, with similar outcomes as the traditional instructor-based method. The proposed system can also be considered cost-efficient as it reduces the permanent presence of an instructor for the chest compressions training, promoting regular training outside formal training courses.

In conclusion, this study corroborates the hypothesis that low-cost tools with feedback for CPR self-training can provide an alternative or a complementary extension to traditional methods for CPR skills acquisition and maintenance.

Acknowledgments

The authors would like to thank the students who voluntarily participated in this study. A special thanks to Joana Amaro for leading the instructor-based sessions, and to Luís Monteiro for his collaboration on data collection.

This article was supported by FEDER through the operation POCI-01-0145-FEDER-007746 funded by the Programa Operacional Competitividade e Internacionalização—COMPETE2020 and by National Funds through FCT—Fundação para a Ciência e a Tecnologia within CINTESIS, R&D Unit (reference UID/IC/4255/2013).

Author contributions

All authors have made substantial contributions to the manuscript. CS-C, DA, AMF, and PV-M contributed to the conception and design of the study. CS-C, DA, and AN implemented the study and supervised the conduct of the trial and data collection. CS-C, DA, and AN conducted the recruitment of students and managed the data, including quality control. CS-C, DA, AMF, and PVM analyzed and interpreted the data. CSC drafted the article, AMF and PV-M made important and critical contributions to the content, and all authors reviewed it critically for important intellectual content. All authors have approved the manuscript version submitted.

Declarations

The study was approved by the joint Ethical Commission of the Faculdade de Medicina da Universidade do Porto/Centro Hospitalar de São João. A written informed consent was obtained from all participants before the study.

Conflicts of interest

The authors declare no conflicts of interest.

References

1. Monsieurs KG, Nolan JP, Bossaert LL, et al. European Resuscitation Council Guidelines for Resuscitation 2015 Section 1. Executive summary. Resuscitation 2015; 95:1–80.
2. Hopstock LA. Cardiopulmonary resuscitation; use, training and self-confidence in skills. A self-report study among hospital personnel. Scand J Trauma Resusc Emerg Med 2008; 16:18.
3. Pozner CN, Almozlino A, Elmer J, et al. Cardiopulmonary resuscitation feedback improves the quality of chest compression provided by hospital health care professionals. Am J Emerg Med 2011; 29:618–625.
4. Perkins GD. Simulation in resuscitation training. Resuscitation 2007; 73:202–211.
5. Fernandes JMG, dos Santos Leite AL, de Sá Duarte Auto B, et al. Teaching basic life support to students of public and private high schools. Arq Bras Cardiol 2014; 102:593–601.
6. European Resuscitation Council. ERC Course strategy, 2008. Available at: http://genoplivning.dk/wp-content/uploads/2013/08/ERC-Course-strategy-2008.pdf. Accessed June, 2018.
7. Brown TB, Dias JA, Saini D, et al. Relationship between knowledge of cardiopulmonary resuscitation guidelines and performance. Resuscitation 2006; 69:253–261.
8. Burkhardt JN, Glick JE, Terndrup TE. Effect of prior cardiopulmonary resuscitation knowledge on compression performance by hospital providers. West J Emerg Med 2014; 15:404–408.
9. Abella BS, Edelson DP, Kim S, et al. CPR quality improvement during in-hospital cardiac arrest using a real-time audiovisual feedback system. Resuscitation 2007; 73:54–61.
10. Issenberg SB, McGaghie W, Petrusa E, et al. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach 2009; 27:10–28.
11. Wang J, Tang C, Zhang L, et al. Compressing with dominant hand improves quality of manual chest compressions for rescuers who performed suboptimal CPR in manikins. Am J Emerg Med 2015; 33:931–936.
12. Allan KS, Wong N, Aves T, et al. The benefits of a simplified method for CPR training of medical professionals: a randomized controlled study. Resuscitation 2013; 84:1119–1124.
13. Yeung J, Davies R, Gao F, et al. A randomised control trial of prompt and feedback devices and their impact on quality of chest compressions: a simulation study. Resuscitation 2014; 85:553–559.
14. Kurowski A, Szarpak Ł, Bogdański Ł, et al. Comparison of the effectiveness of cardiopulmonary resuscitation with standard manual chest compressions and the use of TrueCPR and PocketCPR feedback devices. Kardiol Pol 2015; 73:924–930.
15. Zapletal B, Greif R, Stumpf D, et al. Comparing three CPR feedback devices and standard BLS in a single rescuer scenario: a randomised simulation study. Resuscitation 2014; 85:560–566.
16. Krasteva V, Jekova I, Didon J-P. An audiovisual feedback device for compression depth, rate and complete chest recoil can improve the CPR performance of lay persons during self-training. IOP Sci J 2011; 32:687–699.
17. Semeraro F, Frisoli A, Loconsole C, et al. Motion detection technology as a tool for cardiopulmonary resuscitation (CPR) quality training: a randomised crossover mannequin pilot study. Resuscitation 2013; 84:501–507.
18. Skorning M, Beckers SK, Brokmann JC, et al. New visual feedback device improves performance of chest compressions by professionals in simulated cardiac arrest. Resuscitation 2010; 81:53–58.
19. Qayumi K, Pachev G, Zheng B, et al. Status of simulation in health care education: an international survey. Adv Med Educ Pract 2014; 5:457–467.
20. Dyson K, Bray JE, Smith K, et al. Paramedic resuscitation competency: a survey of Australian and New Zealand emergency medical services. Emerg Med Australas 2017; 29:217–222.
21. Johnson M, Peat A, Boyd L, et al. The impact of quantitative feedback on the performance of chest compression by basic life support trained clinical staff. Nurse Educ Today 2016; 45:163–166.
22. Mpotos N, De Wever B, Valcke MA, et al. Assessing basic life support skills without an instructor: is it possible? BMC Med Educ 2012; 12:58.
23. Simões AP, De Moraes A. The ergonomic evaluation of a virtual learning environment usability. Work A J Prev Assess Rehabil 2012; 41 (suppl 1):1140–1144.
24. Martins AI, Rosa AF, Queirós A, et al. European Portuguese validation of the system usability scale (SUS). Proc Comput Sci 2015; 67:293–300.
25. Brooke J. Jordan PW, Thomas B, Weerdmeester BA, et al. SUS: a quick and dirty usability scale. Usability Evaluation in Industry. London: Taylor & Francis; 1996. 189–194.
26. Lewis JR, Sauro J. The Factor Structure of the System Usability Scale. Presented at: Proceedings of the 1st International Conference on Human Centered Design: Held as Part of HCI International 2009. Berlin, Heidelberg: Springer-Verlag; 2009:94–103.
27. Tullis TS, Stetson JN. A comparison of questionnaires for assessing website usability. Paper presented at: Usability Professionals Association (UPA) 2004 Conference; 2004:7–11.
28. Bangor A, Kortum PT, Miller JT. An empirical evaluation of the system usability scale. Int J Hum Comput Interact 2008; 24:574–594.
29. Bangor A, Kortum P, Miller J. Determining what individual SUS scores mean: adding an adjective rating scale. J Usability Stud 2009; 4:114–123.
30. Wutzler A, Bannehr M, von Ulmenstein S, et al. Performance of chest compressions with the use of a new audio–visual feedback device: a randomized manikin study in health care professionals. Resuscitation 2015; 87:81–85.
Keywords:

BLS; CPR training; feedback; low-cost; skills acquisition; training devices

Supplemental Digital Content

Copyright © 2018 The Authors. Published by Wolters Kluwer Health, Inc. on behalf of PBJ-Associação Porto Biomedical/Porto Biomedical Society. All rights reserved.