The effectiveness of online courses on the learning of the participants has been a subject of discussion and research across the world. Various studies from western countries have evaluated the effectiveness of such courses in the short as well as long term regarding the knowledge, attitude, skills, satisfaction, practice, and outcomes gained from such online courses.[1,2,3,4] Fitzpatrick's levels of learning had been widely used in those studies.[5] However, the majority were restricted to level 1 and level 2, i.e. participant satisfaction and knowledge domains of the learning, while level 3 and level 4, i.e. the practice of the acquired knowledge/skills and outputs from the learning have not been studied, widely.[5]
We developed a virtual system, i.e. an e-learning platform for the International Course on Public Health Approaches to Noncommunicable Diseases organized from March 2 to 6, 2021. It was a 5-day, virtual course. This virtual system catered to all the requisites for the course, i.e. registration, course material, teaching, and certification. The course focused on the prevention, surveillance, and management of noncommunicable diseases (NCDs) including mental health in the Indian, regional, and global context. Among Kirkpatricks four levels of assessment, the first three levels were used to assess whether the participants have achieved the intended objectives of the course.[5,6] Kirkpatrick's levels include reaction (Level 1), which was studied by feedback from the participants and includes initial comments, overall experience, and satisfaction levels with the course, learning (Level 2), which involves knowledge and skills gained from the course, and transfer (Level 3), which involves changes in behavior and practices after the course.
While the data on pre-and post-test at the end of the course were collected within the course's online interface, data at 3 and 6 months were collected using Google Forms mailed to the registered mail IDs of the participants who completed the course and got certified. A maximum of three reminders were sent to them, to elicit the responses. Analysis was done using SPSS 26.0 (IBM Corp. Released 2019. IBM SPSS Statistics for Windows, Armonk, NY: IBM Corp). Normality of the knowledge scores was tested by Kolmogorov–Smirnov test and they were found to be normally distributed. Paired t-test was used to assess the change in the knowledge domain before and after the course. Repeated measures ANOVA was used to test the significance in the change of knowledge domain over the period of time till 6 months after the course. A P < 0.05 was considered statistically significant. The present study was exempted from the ethical review by the Institute Ethics Committee, Postgraduate Institute of Medical Education and Research, Chandigarh.
The online course was attended and completed by 64 participants (110 registrants), with a completion rate of 58.2% (64/110). The mean standard deviation of the age of the course participants was 38.6 (9.4) years. Majority were females (62.5%). Residents (35.9%) and NCD Program officers from across the country (34.4%) were the major groups among the participants. The study reported that 92.2% of the participants were satisfied with the online course. Most of the participants opined that the online course had a good-to-excellent improvement in their attitude toward public health (92.2%). Almost all the participants (98.4%) had placed high-to-very high importance for using the evidence in public health practice. Among the ones who completed the course, 28 (43.8%) participants and 17 (26.6%) participants responded back for the 3- and 6-month follow-up evaluation.
The knowledge of the participants significantly improved after attending the public health practice course online (P < 0.001). However, the knowledge levels did not change over the period of time after the online course (P = 0.988), as found in the 3- and 6-month follow-ups [Table 1]. Majority of the previous literature has also shown significant improvement in the knowledge levels of the online continuing medical education (CME) participants of health sector.[2,3,4] However, their follow-up period was limited to post-test at the end of the course. Whereas the present study assessed the retention of the knowledge acquired by the participants till 6 months.
;)
Table 1: The knowledge scores of the course participants
A large number of the respondents conveyed that they are applying the learnings from the course in their program management, albeit a fall in this proportion over the period of time between 3 months (89.3%) and 6 months (64.7%). The proportion of respondents who have started any new health promotion activity, rehabilitation, palliative program, and implementation research increased over the period of time from 3 to 6 months. Multisectoral action plan for NCD management was formulated by 29.4% of the respondents by the end of 6 months after the course. In a similar self-reported study, improvement in professional skills was found among the participants of web-based courses on public health nutrition.[7]
Half of the participants (50%) preferred the physical mode of the course. The major reason quoted was that leave was not granted to attend the virtual course, but they were attending it along with their routine duties. A similar issue was flagged in the previous study that fitting online CMEs into the busy schedule of the doctors was a major limitation.[8] Low completion rate (58.2%) among the registrants of the assessed course might also be due to the above reason. Existing evidence shows much lower rate of completion of online courses (10%).[9] The major strength of the study was that it assessed three levels of Kirkpatrick model, which is novel in Indian settings. Small sample size, high loss to follow-up, and no active comparison with offline mode of learning are the limitations.
In conclusion, online delivery of public health courses in India has shown to improve knowledge and maintain the acquired knowledge. It has also improved the behavior of the participants in adopting the learnings and practices acquired from the online course. Employers/supervisors must allow the registrants to concentrate on the course full time during the course period. Future research must be conducted by devising methods and objective tools to evaluate the level 3 and 4 learning outcomes of the online courses.
Ethical approval
The present study was exempted from the ethical review by the Institute Ethics Committee (IEC), Postgraduate Institute of Medical Education and Research, Chandigarh.
Financial support and sponsorship
Nil.
Conflicts of interest
There are no conflicts of interest.
Acknowledgment
The authors would like to acknowledge the World Health Organization, World NCD Federation, and Viatris for their technical support for the 6th International Course on Public Health Approaches to NCDs 2021. We would also like to thank the course Coordinators Prof JS Thakur, PGIMER, Chandigarh and Prof Anand Krishnan, AIIMS, New Delhi, and the assistant course coordinators from PGIMER Chandigarh and AIIMS, New Delhi, for the planning and coordinating for the course. We are really grateful to all the course faculty who were a part of this course and delivered their lectures through recordings. We acknowledge the efforts of the core organizing team to make this virtual course a success.
REFERENCES
1. Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Internet-based learning in the health professions: A meta-analysis JAMA. 2008;300:1181–96
2. Curran VR, Fleet L. A review of evaluation outcomes of web-based continuing medical education Med Educ. 2005;39:561–7
3. Curran VR, Fleet LJ, Kirby F. A comparative evaluation of the effect of Internet-based CME delivery format on satisfaction, knowledge and confidence BMC Med Educ. 2010;10:10.
4. Rouleau G, Gagnon MP, Côté J, Payne-Gagnon J, Hudson E, Dubois CA, et al Effects of E-learning in a continuing education context on nursing care: Systematic review of systematic qualitative, quantitative, and mixed-studies reviews J Med Internet Res. 2019;21:e15118.
5. Praharaj SK, Ameen S. The relevance of telemedicine in continuing medical education Indian J Psychol Med. 2020;42:97S–102S
7. Sigulem DM, Morais TB, Cuppari L, Franceschini SC, Priore SE, Camargo KG, et al A Web-based distance education course in nutrition in public health: Case study J Med Internet Res. 2001;3:E16.
8. . Delivering Pulmonary Continuing Medical Education Over the InternetLast accessed on 2022 Mar 29 Available from:
https://indexarticles.com/reference/chest/delivering-pulmonary-continuing-medical-education-over-the-internet/
