Secondary Logo

Journal Logo

Innovation

Immersive photoreal new-age innovative gameful pedagogy for e-ophthalmology with 3D augmented reality

Ramesh, Prasanna V,; Aji, K1; Joshua, Tensingh2; Ramesh, Shruthy V3; Ray, Prajnya1; Raj, Pragash M4; Ramesh, Meena K5; Rajasekaran, Ramesh6

Author Information
Indian Journal of Ophthalmology: January 2022 - Volume 70 - Issue 1 - p 275-280
doi: 10.4103/ijo.IJO_2133_21
  • Open
  • Associated Video

Abstract

The Inception of the Pedagogical Application “Eye MG AR”

Throughout the last decade, augmented reality (AR) has slowly and steadily become a substantial part of modern life, with increasing applications in the field of medicine and ophthalmology.[1] During COVID-19, the impetus of AR into ophthalmology is more stronger than ever. The COVID-19 pandemic has momentously changed the landscape in which we practice ophthalmic medicine. As we adapt to what is the new “norm” surrounding social distancing and interpersonal contact, innovative ways to teach and learn e-ophthalmology are in high demand.[2] Hence, we have innovated an AR program named “Eye MG AR” to simplify ophthalmic concept learning and to serve as a new-age immersive 3D pedagogical tool for gameful learning.

Gamification - 3D Gameful Learning Tool

Recently, an AR program named “Eye MG AR” was innovated by us to show different three-dimensional (3D) anatomical structures [Figs. 1 and 2] related to e-ophthalmic learning, with multiple customized angles of the viewer’s choice to simplify concept learning, using real-time TrueColor confocal images. Simple structures such as the eyeball [Fig. 3] and complex structures relevant to the eye (cerebral venous and dural sinuses) were also constructed in advanced real-time 3D photoreal visuals for immersive visual experiences.[34] This pedagogical transformation in e-ophthalmology aims to reinvent the approach to ophthalmic teaching through virtual platforms with Unreal Engine software [Figs. 4-6] [Video Clip 1 and 2]. This AR app was constructed with pioneering ways of cult teaching on an android platform, along with gameful 3D learning in ophthalmology. Gamification in e-learning with deep visualization and cinematic experiences has never been reported in the literature and can pave the way for a new age ophthalmic pedagogy.

Figure 1
Figure 1:
Image showing annotated three-dimensional (3D) model of the eyeball
Figure 2
Figure 2:
Image showing the annotated 3D side view of the cerebral venous and dural sinuses model
Figure 3
Figure 3:
Image showing the pedagogue viewing the eyeball model: (a) top view; (b) anterior segment; (c) posterior segment
Figure 4
Figure 4:
(a) Logo of the AR software (Unreal Engine). (b) Main-level workspace in the Unreal Engine software. (c) Cerebral venous and dural sinuses model in the viewport created by us. (d) Level blueprint created by us in the software. (e) Widget blueprint created by us in the software
Figure 5
Figure 5:
Image showing the eyeball model used inside Unreal Engine workspace
Figure 6
Figure 6:
Image showing the cerebral venous and dural sinuses model used inside unreal engine workspace

{"href":"Single Video Player","role":"media-player-id","content-type":"play-in-place","position":"float","orientation":"portrait","label":"Video Clip 1","caption":"","object-id":[{"pub-id-type":"doi","id":""},{"pub-id-type":"other","content-type":"media-stream-id","id":"1_f5un7ccr"},{"pub-id-type":"other","content-type":"media-source","id":"Kaltura"}]}

{"href":"Single Video Player","role":"media-player-id","content-type":"play-in-place","position":"float","orientation":"portrait","label":"Video Clip 2","caption":"","object-id":[{"pub-id-type":"doi","id":""},{"pub-id-type":"other","content-type":"media-stream-id","id":"1_0quz537m"},{"pub-id-type":"other","content-type":"media-source","id":"Kaltura"}]}

Setting Up the Framework for Accessing Eye MG AR

With the following steps [Video Clips 3 and 4] one can access this gamification tool for infinite e-learning:

{"href":"Single Video Player","role":"media-player-id","content-type":"play-in-place","position":"float","orientation":"portrait","label":"Video Clip 3","caption":"","object-id":[{"pub-id-type":"doi","id":""},{"pub-id-type":"other","content-type":"media-stream-id","id":"1_dm4ddr4u"},{"pub-id-type":"other","content-type":"media-source","id":"Kaltura"}]}

{"href":"Single Video Player","role":"media-player-id","content-type":"play-in-place","position":"float","orientation":"portrait","label":"Video Clip 4","caption":"","object-id":[{"pub-id-type":"doi","id":""},{"pub-id-type":"other","content-type":"media-stream-id","id":"1_q55221zw"},{"pub-id-type":"other","content-type":"media-source","id":"Kaltura"}]}

  1. Download the AR template and take a print of it and place it on your table.
  2. Install the Eye MG AR app on your android phone from the Google Play Store.
  3. Launch the app and focus your mobile camera [Fig. 7] on the AR template [Fig. 8], which should be fitted within the given borders shown on the display screen [Fig. 9].
  4. Anatomical structure of the eye/cerebral and dural venous sinuses [Figs. 10 and 11] appears.
  5. Rotate the paper to access customized degrees or angles for the user’s view of the various structures.
  6. Move the mobile accordingly to view the structure from the top [Fig. 12].

Figure 7
Figure 7:
Image showing the pedagogue beginning the augmented reality (AR) teaching session by focussing the mobile camera over the AR template
Figure 8
Figure 8:
Image showing the AR template to be downloaded and printed as the base, for the model to appear
Figure 9
Figure 9:
Image showing the borders within which the AR template should be fitted in the mobile screen
Figure 10
Figure 10:
The pedagogue viewing the eyeball (red arrow) in the mobile screen over the AR template
Figure 11
Figure 11:
The pedagogue viewing the cavernous sinus (red arrow) in the mobile screen over the AR template
Figure 12
Figure 12:
Top view of the cerebral venous and dural sinuses when mobile is adjusted accordingly

Pedagogical Transformation

To make ophthalmic concept learning better, we have innovated this app for studying ocular anatomy, built on an advanced interactive 3D touch interface. Concepts pertaining to ophthalmology that have lots of theoretical framework can be broken into small fragments, as no longer the neophytes have to mentally visualize them during training [Fig. 13] [Video Clip 5]. This is a powerful cognitive tool that serves as a 3D encyclopedia, where students can choose their optimal frame of choice, from the 360° angle for viewing [Video Clip 6-9], thus filling in their cognitive mental gaps.

Figure 13
Figure 13:
Image showing the pedagogue. (a) Training the students using the Eye MG AR app. (b) Demonstrating the parts of the eyeball. (c) Focusing the optic disc

{"href":"Single Video Player","role":"media-player-id","content-type":"play-in-place","position":"float","orientation":"portrait","label":"Video Clip 5","caption":"","object-id":[{"pub-id-type":"doi","id":""},{"pub-id-type":"other","content-type":"media-stream-id","id":"1_25rhiei7"},{"pub-id-type":"other","content-type":"media-source","id":"Kaltura"}]}

{"href":"Single Video Player","role":"media-player-id","content-type":"play-in-place","position":"float","orientation":"portrait","label":"Video Clip 6","caption":"","object-id":[{"pub-id-type":"doi","id":""},{"pub-id-type":"other","content-type":"media-stream-id","id":"1_b684qh5f"},{"pub-id-type":"other","content-type":"media-source","id":"Kaltura"}]}

{"href":"Single Video Player","role":"media-player-id","content-type":"play-in-place","position":"float","orientation":"portrait","label":"Video Clip 7","caption":"","object-id":[{"pub-id-type":"doi","id":""},{"pub-id-type":"other","content-type":"media-stream-id","id":"1_sqfd8o3l"},{"pub-id-type":"other","content-type":"media-source","id":"Kaltura"}]}

{"href":"Single Video Player","role":"media-player-id","content-type":"play-in-place","position":"float","orientation":"portrait","label":"Video Clip 8","caption":"","object-id":[{"pub-id-type":"doi","id":""},{"pub-id-type":"other","content-type":"media-stream-id","id":"1_a8s09yh7"},{"pub-id-type":"other","content-type":"media-source","id":"Kaltura"}]}

{"href":"Single Video Player","role":"media-player-id","content-type":"play-in-place","position":"float","orientation":"portrait","label":"Video Clip 9","caption":"","object-id":[{"pub-id-type":"doi","id":""},{"pub-id-type":"other","content-type":"media-stream-id","id":"1_tyv1qn3o"},{"pub-id-type":"other","content-type":"media-source","id":"Kaltura"}]}

Prerequisites of the App

This app is currently available free of cost from the Google Play Store for Android phones that support Google Play services for AR.[5] The users can either download the app from the Google Play Store or directly scan the QR code [Fig. 14] and then download it. More ocular structures related to ophthalmology are being added as updates for comprehensive ophthalmic pedagogy. To support this AR app, the android device must have a compass (magnetometer), a rear-facing camera, a gyroscope, a global positioning system (GPS), and an accelerometer.[6] These devices need to run Android 7.0 Nougat or later, and need to be installed with Google Play services for AR. In this era, there is no dearth of smartphone users. The majority of students have already been using their smartphones to take lecture notes, and for surfing the internet for studying. Carving a mobile application with 3D AR will allow them to not have any dark side regarding the anatomy of oculus uterque. The app is currently under development for iOS users.

Figure 14
Figure 14:
Image showing details of the Eye MG AR app on the Google Play Store, and the QR code for directly downloading the app by scanning it

Conclusion

AR enhances the real world with immersive experiences by providing a digital overlay over the real world. In contrast, virtual reality replaces the real world and shuts out the physical world. Ophthalmology is a field of medicine that is well suited for the application of AR. We believe that time will indeed foster further technological advances and lead to widespread use of AR in routine ophthalmic practice.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

References

1. Gopalakrishnan S, Chouhan Suwalal S, Bhaskaran G, Raman R Use of augmented reality technology for improving visual acuity of individuals with low vision Indian J Ophthalmol 2020 68 1136 42
2. Iskander M, Ogunsola T, Ramachandran R, McGowan R, Al-Aswad LA Virtual reality and augmented reality in ophthalmology:A contemporary prospective Asia Pac J Ophthalmol (Phila) 2021 10 244 52
3. Karakas AB, Govsa F, Ozer MA, Eraslan C 3D brain imaging in vascular segmentation of cerebral venous sinuses J Digit Imaging 2019 32 314 21
4. Cabrilo I, Bijlenga P, Schaller K Augmented reality in the surgery of cerebral arteriovenous malformations:Technique assessment and considerations Acta Neurochir (Wien) 2014 156 1769 74
5. Eye MG AR - - Apps on Google Play [Internet] [cited 2021 Nov 26] Available from: https://play.google.com/store/apps/details? id=com. EyeMG_AR&hl=en&gl=IN
6. Aydındoğan G, Kavaklı K, Şahin A, Artal P, Ürey H Applications of augmented reality in ophthalmology Biomed Opt Express 2020 12 511 38
Keywords:

3D; Augmented Reality; 3D Eye Models; Cerebral Venous System; e-Ophthalmology; Pedagogy

Copyright: © 2021 Indian Journal of Ophthalmology