In the health sector, interactive virtual visualization technologies such as 3D screen navigation and Virtual Reality (VR) and Augmented Reality (AR) technologies have an important differential that conveys credibility to the user, which is the fact that the three-dimensional model files used come from non-invasive medical imaging exams such as computed tomography (CT), magnetic resonance imaging (MRI) and 3D ultrasound (US).
When inserted into a virtual environment, the medical exams, which are cases of real patients, allow for interactive visualization where, depending on the technology used, various actions are possible, such as three-dimensional (3D) segmentation, simulation of procedures and information for non-specialized audiences (such as patients).
One of the first projects developed for virtual 3D visualization of the human body using fetal MRI files was carried out in 2011 by Brazilian researchers Heron Werner, Jorge Lopes and Ricardo Fontes and published by the British magazine “New Scientist”. In the video, it is possible to “visualize the internal anatomical structure of a 36-week-old fetus in an MRI scan.
The various projects that followed this research, also published in international scientific journals, motivated the authors to expand the field of possibilities in the use of non-invasive medical images to visualize clinical cases.
With the advent of the COVID-19 pandemic, video and image communication technologies have been boosted. The team at PUC-Rio’s BIODESIGN laboratory, in partnership with the Dasa group, has developed personal protective equipment for the health area (projects supported by FAPERJ and CAPES), including a face shield with a digital interface where it is possible to view medical records without physical contact.
Also during the pandemic, due to the limited physical contact between researchers, work meetings were started in the virtual environment of IMPA’s “Pi Center”, using VR through the “SPATIAL” application, a collaborative workspace in a virtual environment created in 2017.
The meetings held in the SPATIAL software broadened our knowledge of the then newly widespread concept of the metaverse, which has a relevant application in medicine, with growing and satisfactory applications.
The main difference between the successful use of the metaverse in medicine compared to other areas can be understood, as explained above, in the fact that in medicine the 3D model files used come from real, non-invasive medical imaging exams such as CT, MRI and 3D US, i.e. they are not 3D simulations modeled in software but files that are imported and opened directly on the platform. When inserted into a virtual environment, real medical exams can be checked in the metaverse simultaneously by several participants in real time.
The concept of Extended Reality (ER), which encompasses VR, AR and Mixed Reality (MR), is revolutionizing healthcare by improving diagnosis, treatment, medical education and patient engagement. These immersive technologies enable precise 3D visualizations, helping doctors to identify various diseases.
In surgical planning, XR allows 3D models to be created from US, CT and MRI scans, making it easier to navigate complex anatomies and optimize surgical approaches. During operations, real-time navigation with CR can increase the accuracy and safety of procedures. In addition, RE offers interactive training environments where surgeons and students practice complex procedures, such as robotic surgery.
Medical education benefits greatly, providing immersive and detailed experiences of human anatomy and various clinical situations.
XR can also promote remote collaboration and telemedicine, connecting specialists from different parts of the world without the need to travel. In mental health, virtual therapy sessions could open up new ways of interacting with patients.
