The term “haptics” emerged to designate the science of touch, dedicated to studying and simulating pressure, texture, vibration and other related biological sensations. From this science arose multiple innovations aimed at providing more realism and depth in video game control, airplane yokes that vibrate or get harder to pull in case of imbalance during flight, and in cellphones that vibrate in certain situations. The study of haptic interfaces is a relatively new field of research that is quickly growing.
The acquired knowledge in this field was obtained through the multidisciplinary collaboration, where numerous technologies, such as virtual reality, 3D printers, and more recently 3D haptic devices.
The initial stage involved segmenting ultrasound (US) images from real cases. This process enabled the creation of detailed three-dimensional (3D) models, ensuring the fidelity of the simulated tissues and organs. To achieve this, segmentation and 3D reconstruction software were used to convert 2D images into models compatible with Unity.
To enable realistic movement of tissues and organs, rigging was applied to the 3D models. This process involved creating internal structures for articulation and deformation, using:
- Bones: Define articulation points and internal movements of the model.
- Influence Weights: Control the degree of deformation in each area in response to articulations.
This technique was crucial for ensuring precise tactile interactions, allowing tissues to be manipulated and repositioned without unnatural behavior.
Unity’s Configurable Joint and Hinge Joint components were used to configure the degrees of freedom of the model’s movable parts, ensuring greater anatomical realism. The main settings included:
- Rotation Constraints: Limiting angles on the X, Y, and Z axes to prevent unnatural postures.
- Controlled Movements: Preventing unrealistic displacements, such as the overextension of the umbilical cord.
- Physical Interactions: Adjusting touch responses to simulate natural reactions, such as tissue displacement when pressed.
The simulation was integrated with the Touch device from 3D Systems using the OpenHaptics library, enabling tactile feedback for the user. The device provides linear forces on the X, Y, and Z axes but does not support torque (rotational forces). To compensate for this limitation:
- Rotations were converted into adjusted linear forces.
- The system was configured to simulate resistance and weight when handling tissues and organs, creating a more realistic tactile experience.
Several technical challenges arose during the development of the haptic simulation, including the lack of torque feedback from the haptic device, requiring adaptations to prevent excessive deformations and ensure stability during interactions. Additionally, real-time dynamic registration of tactile surfaces was a complex process, as it was essential to ensure that surfaces were generated and synchronized accurately with visual objects during the simulation. Another critical challenge was fine-tuning movement parameters to respect anatomical constraints without compromising overall system functionality.
The technical results demonstrate the effectiveness of the chosen approach. The haptic simulation accurately replicates the sensation of weight and resistance to touch, which are fundamental aspects of realistic medical training. The rigging and joint configuration enabled natural and consistent movements, allowing fluid interactions such as repositioning tissues or manipulating the umbilical cord. Furthermore, the system proved flexible, supporting the creation and manipulation of tactile surfaces in real time, expanding its applicability to various surgical scenarios and training needs.
We believe in using these devices for medical training.
