PMD2418 PROJECT PHASE

VR-based Surgical Simulator with Haptic

Feedback

PROJECT BY
SUBATHIRA R
REG.NO : 3122234005012
ME Medical Electronics,
SSN College of Engineering

INTERNAL SUPERVISOR EXTERNAL SUPERVISOR

Dr. B. DIVYA, Dr. LAURA CERCENELLI

Assistant Professor, Adjunct Professor
Department of Biomedical Engineering, Department of Biomedical Engineering
SSN college of Engineering. University of Bologna, Italy

1 v 1.2
 PROBLEM STATEMENT

• To Develop an VR surgical simulators for medical training, integrating haptic feedback

to enhance tactile realism.
• Focus on addressing current shortcomings, particularly in orthopedic surgery, to improve
training quality and preparedness of future medical professionals.

2 v 1.2
 OBJECTIVE

To create a VR-based surgical simulator with haptic feedback to provide a

realistic and immersive training environment for medical professionals.

• Developing Realistic 3D Modeling: Create highly detailed and anatomically accurate 3D

models thereby enhancing the realism of the simulated environment.

• Haptic Rendering: Integrate haptic feedback systems to accurately replicate the tactile
sensations experienced during surgical procedures.

• User Interface (UI) Rendering: Design an intuitive and user-friendly interface that
facilitates easy interaction with the simulation system.

• Develop Simulation Logic: Build robust simulation logic that governs the behavior of virtual
environments, ensuring that procedures and scenarios are accurately replicated.

3 v 1.2
 Existing Methods of Surgery Training for Medical
Professionals

Existing Methods

1. Traditional Apprenticeship 2. Cadaver-Based Training

(Halstedian Model) Uses human cadavers for practicing surgical
"See one, do one, teach one" approach. techniques.
Limitations: High patient risk, variable Limitations: High cost, limited availability, lack
training quality, and limited hands-on of physiological responses, and ethical concerns.
practice opportunities.

3. Animal Models 4. Physical Simulators (Synthetic Models)

Live or preserved animal tissues are used for Uses artificial anatomical models made of
surgical practice. silicone, plastic, or other materials.
Limitations: Ethical concerns, anatomical Limitations: Limited realism, static nature, and
differences from humans, and high high production costs
maintenance costs
6. Robotic Surgery Simulators
5. Computer-Based Training (CBT) and E- Mechanical trainers and digital interfaces for
Learning minimally invasive procedures.
Online surgical modules, videos, and Limitations: High cost, limited procedure
interactive tutorials. variety, and lack of real-world complications.
Limitations: No hands-on skill development
and limited engagement.
4 v 1.2
 LITERATURE SURVEY
TITLE METHODOLOGY FINDINGS

Haptic-enabled virtual The paper likely outlines the study • Skill improvement in the VR
training in orthognathic design: group compared to the
• Participants: Trainees (novices traditional group.
surgery or experienced surgeons) who • Impact of haptic feedback on
Medellin-Castillo, HI, Zaragoza- participated in the study. skill acquisition.
Siqueiros, J, Govea-Valladares, • CT to 3D model and • Trainee confidence levels.
EH, de la Garza-Camargo, H, Lim, animations.
T & Ritchie, JM 2021, Algorithms used here:
'Hapticenabled virtual training in • Axis-Aligned Bounding Box
orthognathic surgery', Virtual (AABB) Collision Detection.
Reality, vol. 25, pp. 53–67. Two AABBs collide if they overlap in all
Published: 02 April 2020. dimensions (x, y, and z). If their ranges
https://doi.org/10.1007/s10055- on each axis intersect, a collision is
020-00438-6. detected.
• GJK Algorithm (Gilbert-
Johnson-Keerthi)
Iteratively finds the closest points on
each shape and checks if the distance
between them is zero or less than a
certain threshold.

5 v 1.2
 TITLE METHODOLOGY FINDINGS

Development and Testing of • Voxel-Based Rendering for • This study developed a VR

a Visuo-Haptic Surgical Bone Removal simulator for a Lefort I
Training Simulator for • Multi-Point Collision Haptic osteotomy bone-sawing
Feedback procedure.
Orthognathic Surgery • CT Scan-Based Detailed Bone • The simulator allows users to
Yanping, L., Xiaojun, C., Xudong, Models practice at different speeds and
W., Guofang, S., Chengtao, W. • Haptic rendering – force / records performances for
(2014). Development and Testing tactile / vibration feedback. review.
of a Visuo-Haptic Surgical • Simulated forces were
Training Simulator for compared to real-world
Orthognathic Surgery. In: Goh, J. measurements.
(eds) The 15th International
Conference on Biomedical
Engineering. IFMBE Proceedings,
vol 43. Springer, Cham.
https://doi.org/10.1007/978-3-319-
02913-9_117.

6 v 1.2
 TITLE METHODOLOGY FINDINGS

Intelligent Haptic Virtual • Human Model Integration • The system combines VR

Simulation for Suture • Variety of Surgical Tools visualization with haptic
Surgery • Visuo-Haptic Rendering devices to simulate the surgical
Sung, M. Y., Kang, B., Kim, J., • Replica Training environment.
Kim, T., & Song, H. (2020). File-Based Replica Training: • Collision Detection
Intelligent haptic virtual simulation Users practice with pre-recorded Algorithm- The implemented
for suture surgery. International scenarios. algorithm ensures precise
Journal of Advanced Computer Real-Time Network-Based collision detection, enhancing
Science and Applications, 11(2). Replica Training: Live training realism.
with feedback over a network • Stable Touch Presentation- The
connection. system provides stable and
accurate touch feedback during
suture practice.
• Previous Work- The algorithm
builds upon a “precise
collision detection algorithm”
proposed in the author’s earlier
research.

7 v 1.2
 TITLE METHODOLOGY FINDINGS

Modeling the behavior of Tissue Modelling Force Feedback Accuracy:

human body tissues on human body is segmented into The haptic device successfully
penetration different tissue layers, including differentiated between skin, fat,
Conci, A., Brazil, A. L., Popovici, skin, fat, muscle, and deeper and muscle, allowing trainees to
D., Jiga, G., & Lebon, F. (2018, structures sense changes in resistance and
February). Modelling the behavior For instance, skin requires 6.0 N of adjust their technique accordingly.
of human body tissues on force to puncture, fat requires 2.0 System Performance:
penetration. In AIP Conference N, and muscle requires 4.4 N. The integration of force feedback
Proceedings (Vol. 1932, No. 1). Haptic Deice Integration significantly enhanced the realism
AIP Publishing. A haptic device, such as a force- of the simulation compared to
feedback stylus, is calibrated to visual-only training methods.
produce force outputs correlating
with tissue resistance values.

8 v 1.2
 User interface Control unit

VR Headset

Database
Haptic Device
3D Anatomical
Model

Simulation Engine Surgical Procedures

Physics Engine

Graphics Engine
BLOCK DIAGRAM

Haptic feedback

9
 METHODOLOGY

• 3D Modeling: Anatomical structures and surgical instruments for orthopedic

procedures are created using 3D Slicer and Blender, ensuring detailed and accurate

models.

• Haptic Integration: The 3D Systems Touch device is integrated to provide

synchronized haptic feedback with visual cues, enhancing the realism of surgical

interactions.

• Real-Time Rendering: Utilizing Unity's capabilities for real-time rendering, with

advanced lighting, shading, and dynamic effects to achieve visual fidelity.

10 v 1.2
 • Algorithm Development: Custom algorithms simulate surgical procedures,

incorporating physics-based interactions that realistically model tissue behavior, tool

handling, and physiological responses like bleeding.

• Haptic Rendering: Haptic rendering algorithms simulate tactile properties of tissues

with force models that replicate various resistances and textures.

• User Interface (UI) Design: An intuitive UI enables seamless navigation and tool

selection, fully integrated into the 3D environment for an immersive user experience.

11
 DELIVERABLES

12 v 1.2
 TIMELINE
Work / Aug Sep Oct Nov Dec Jan Feb Mar Apr
Months
Planning &
Analysing
3D modelling

Real time
rendering
Testing &
Refining
Surgery tools
preparation
Haptic
integration
Procedure
Simulation
UI integration

Testing &
Refining

13 v 1.2
 PHASE 1 WORKFLOW

14 v 1.2
 TOTAL KNEE REPLACEMENT
• Total knee replacement (TKR), also known as total knee arthroplasty, is a surgical
procedure performed to relieve pain and restore function in patients with severe
knee joint damage.
• The procedure involves replacing the damaged knee joint surfaces with artificial
implants made of metal and plastic components.

15 v 1.2
 ANATOMY OF KNEE BONE

Model : Knee Bone

Reference image : CT image of Knee

Femur

Patella

Tibia

Fibula
https://www.dreamstime.com/ct-scan-knee-joint-d-rendering-showing-knee-structure image2903
18338

16 v 1.2
 STEP 1 : 3D MODEL
(KNEE)
Creating a 3D model

Creating using Blender

• Ensuring the model contains all necessary anatomical parts: bones

(femur, tibia, fibula, patella), ligaments, muscles, and cartilage.
• Verify anatomical accuracy.
• Review the polygon count.
• Refine the Model – cleaning up the unnecessary vertices, faces etc…
• Assign Material Properties
• Texturing

17 v 1.2
 3D MODELLING

Simplifying the Mesh

Initial scaling and positioning

Adjust vertices for anatomical

accuracy

FEMUR Sculpting the Bones FIBULA

TIBIA

18 v 1.2
 ASSEMBLING THE PARTS

19 v 1.2
 STEP: 2 APPLYING TEXTURES
Here the texture is applied to each mesh using blender’s shading feature.

20 v 1.2
 WHOLE KNEE WITH TEXTURES

21 v 1.2
 PHASE 1 WORKFLOW AND OUTPUTS

Polygon Optimization
3D Model development Testing Rigging
and Texturing

Tried adding Colliders

Environment setup in Unity

Demo of incision process

22 v 1.2
 PHASE II WORKFLOW

23 v 1.2
 MODEL ENHANCEMENT AND RIGGING

• Applying a Subdivide Modifier

 Increases mesh resolution for smoother deformations.
• Adding inverse kinematics
 Helps maintain realistic knee bending constraints.
 Allows intuitive control over joint movement without needing
manual bone rotation.
 Reduces animation complexity by enabling fluid motion.
• Adding Automatic Weights
 Distributes influence based on bone proximity, ensuring
smooth deformation. Reduces manual weight-painting effort
by calculating initial values. Helps prevent mesh distortion
when bending.
24 v 1.2
 WEIGHT PAINTING
• Weight Painting
A method used in 3D modeling to assign influence of bones on
mesh deformation. Critical for realistic joint movement in
simulations and animations. Used for ensuring smooth bending
without artifacts.
• Importance in Knee Simulation
Prevents unrealistic stretching or collapsing of skin and muscle.
Ensures bones remain rigid while soft tissues deform naturally.
Crucial for realistic surgical simulations and biomechanics.

25 v 1.2
 Soft Body Physics Constraint Setup
• Configuring Cloth Constraints in Unity to simulate soft tissue behavior.
• Red vertices indicate fully constrained areas, while green vertices represent movable
regions.
• Adjusted Max Distance and Constraint Size for controlled deformation.

26 v 1.2
 HAPTIC INTEGRATION
HAPTIC SETUP IN UNITY

27 v 1.2
 Haptic Material Properties (0-1)

Reference : Conci, A., Brazil, A. L., Popovici, D., Jiga, G., & Lebon, F. (2018, February). Modelling the
behaviour of human body tissues on penetration. In AIP Conference Proceedings (Vol. 1932, No. 1). AIP
Publishing.

28 v 1.2
 Depth-Based Resistance

29 v 1.2
 INSTRUMENTS 3D MODEL

RETRACTOR

RECIPROCATING SAW
30 v 1.2
 MAIN MENU SCREEN
1. EXPLORE THE ANATOMY
2. SURGERY SIMULATION

31 v 1.2
 SIMULATION LOGIC

START OVER

32 v 1.2
 USER INTERFACE – INSTRUCTIONS AND ALERT

INSTRUCTIONS

33 v 1.2
 ALERTS

34 v 1.2
 FINAL OUTCOME

35 v 1.2
 THANK YOU

36 v 1.2