Medical Image Analysis

Introduction and Overview

Dr. Debdoot Sheet

Assistant Professor
Department of Electrical Engineering
Indian Institute of Technology Kharagpur

www.facweb.iitkgp.ernet.in/~debdoot/
A GREAT PIECE OF CAREER ADVICE
FOR EECS GRADUATES
INTERESTED IN MACHINE VISION
Medical Image Analysis - Introduction [Debdoot Sheet] 2
 Market Scenario and Career
Modality
Media, Market for Machine Vision • X-ray
surveillance, Systems in 2020 • Ultrasound
automotive, • Computed Tomography (CT)
• Magnetic Resonance (MRI)
graphics, etc.
• Nuclear Imaging (PET & SPECT)
($ 6 Billion) Clinical Indications
63% • Radiology
• Cardiology
• Oncology
Medical Image • Neurology
Analysis • Obstetrics & gynecology
($ 3.5 Billion) • Breast mammography
37% End Users
• Hospitals
• Diagnostic centers
• Research centers
Report code: HIT 1309 and SE 2701 from www.marketsandmarkets.com
Medical Image Analysis - Introduction [Debdoot Sheet] 3
Overview of MedIA

Medical Image Analysis - Introduction [Debdoot Sheet] 4

 Key Research (Business) Areas
• Macro - meso - micro imaging
• Medical image registration
• Organ localization
• Organ segmentation
• Medical visualization and
Augmented reality
• Virtual anatomy
• Digital angiography
• Optical and ultrasonic despeckling
• 3D optical microscopy
• Digital pathology
• Computational imaging
Medical Image Analysis - Introduction [Debdoot Sheet] 5
End use of MedIA
Tong, S.; Sheet, D.;
Bhuiyan, S.; Zequer
Diaz, M.; Taberne, A.,
"BME Trends Around
the World : From
Baby X to frugal
technologies, here's
what biomedical
engineers are
excited about in
2015. [From the
Editors]," IEEE
Pulse, vol.6, no.1,
pp.4-6, Jan.-Feb.
2015

Medical Image Analysis - Introduction [Debdoot Sheet] 6

 Organization
• Week 1 • Week 3
– Introduction – Decision Trees
– X-ray and Computed – Random Forests
Tomography – Neural Networks
– Magnetic Resonance – Deep Learning for Medical
– Ultrasonic (US) Image Analysis
– Molecular (SPECT, PET) • Week 4
• Week 2 – Retinal Vessel
– Texture Segmentation
– Region Growing – Lung CT Vessel
Segmentation
– Random Walks
– Brain MRI Lesion
– Active Contours Segmentation
– Evaluation and Validation – Tissue Characterization
– Histology Segmentation
Medical Image Analysis - Introduction [Debdoot Sheet] 7
 Last 35 years of MedIA
• Pre 1980 – 1984: Era • 1999 – 2010: Machine
of Pattern Recognition Learning with Shallow
Analysis of 2D Images Reasoning
• 1985 – 1991: • 2010 and Beyond:
Knowledge based Machine Learning
Approaches with Complex
• 1992 – 1998: 3D Reasoning
Images and Towards Duncan, J.S.; Ayache, N., "Medical image
analysis: progress over two decades and the
Integrated Analysis challenges ahead," IEEE Trans. Pat. Anal., Mach.
Intell., vol.22, no.1, pp.85,106, Jan. 2000
Medical Image Analysis - Introduction [Debdoot Sheet] 8
 MedIA Challenges in 2017
• Automated Detection and
Classification of Breast Cancer
Metastasis 2017
(CAMELYON17)
• Diagnostic Classification of
Clinically Significant Prostate
Lesions (PROSTATEx)
• Multiple Sclerosis
Segmentation (MSSEG)
• Modeling and Monitoring of
Computer Assisted
Intervention (M2CAI)
• Ultrasound Nerve
Segmentation (Award of $
1,00,000 on Kaggle)

Medical Image Analysis - Introduction [Debdoot Sheet] 9

MedIA Challenges in 2017

Medical Image Analysis - Introduction [Debdoot Sheet] 10

CAREER ADVICES FOR MedIA
ASPIRANTS

Medical Image Analysis - Introduction [Debdoot Sheet] 11

 Find a (Research) Challenge
• Grand Challenges in Biomedical Image Analysis
– www.grand-challenges.org
• MICCAI
– www.miccai.org
– www.miccai2016.org
• ISBI
– www. biomedicalimaging.org
• SPIE Medical Imaging
– http://spie.org/conferences-and-exhibitions/medical-
imaging

Medical Image Analysis - Introduction [Debdoot Sheet] 12

 Tool(boxes) of the Trade
• BioDigital Human
– www.biodigital.com
• MeVisLab
– Medical image processing, analysis and visualization SDK
– http://www.mevislab.de/
• Anaconda
– Python 2.7 with scientific computing library for custom building tools
– https://www.continuum.io/downloads/
• Torch
– Scientific computing with Lua. Used for Deep Learning
– www.torch.ch
• Matlab
– Matrix laboratory scientific computing tool
– https://www.mathworks.com/
• CUDA
– Library for using NVIDIA GPUs
Medical Image Analysis - Introduction [Debdoot Sheet] 13
 Where to Read for MedIA?
Specialist Literature General Reading
• Medical Image Analysis • IEEE Trans. Image
• IEEE Trans. Medical Imaging Processing
• IEEE Trans. Computational • IEEE Trans. Biomedical
Imaging Engineering
• IEEE J. Biomedical and • Int. J. Computer Vision
Health Informatics • IET Image Processing
• SPIE J. Medical Imaging • IET Computer Vision
• Computerized Medical • ACM Trans. Graphics
Imaging and Graphics
• Proc. Int. Conf. Image. Proc.
Medical Image Analysis - Introduction [Debdoot Sheet] 14
 Where to Read for ML?
Journal Conferences
• IEEE Trans. Pattern Analysis • Computer Vision and
Pattern Recognition (CVPR)
and Machine Intelligence
• Machine Learning confs.
• Machine Learning – International (ICML)
• J. Machine Learning – European (ECML)
Research – Asian (ACML)
– Neural Information
• IEEE Trans. Knowledge and Processing System (NIPS)
Data Engineering • Computer Vision conf.
• IEEE Trans. Neural Networks – International (ICCV)
– European (ECCV)
• IEEE Trans. Sys. Man. Cyber. – Asian (ACCV)

Medical Image Analysis - Introduction [Debdoot Sheet] 15

 Where to Listen and Socialize?
Workshops and Schools Conferences
• Medical Imaging Summer • Medical Image Computing and
School (MISS) Computer Assisted
– MICCAI Society
Intervention (MICCAI)
• Int. Symp. Biomed. Imaging
• IEEE SPS Summer School on (ISBI)
Biomedical Image Analysis
• SPIE Med. Imaging
– IEEE Signal Processing Soc.
• Information Processing in
• IEEE EMBS Int. Summer School Medical Imaging (IPMI)
on Biomedical Imaging • Information Processing in
– IEEE Engineering in Medicine Computer-Assisted
and Biology Society Interventions (IPCAI)
• Biomedical Image Analysis • Indian Conference on
Summer School Computer Vision, Graphics and
Image Processing (ICVGIP)
Medical Image Analysis - Introduction [Debdoot Sheet] 16
 Take home message
Some Texts MedIA in Classroom
• Toennies, Guide to medical • IIT Kharagpur:
image analysis, 2012. http://www.bit.do/eemia2016
• Bankman, Handbook of Medical
Image Processing and Analysis, • IIT Bombay:
2012. http://www.cse.iitb.ac.in/~suya
• Dougherty, Medical Image sh/cs736/
Processing techniques and • Carnegie Mellon:
applications, 2013. http://www.cs.cmu.edu/~galeo
• M. N. Gurcan, "Histopathological tti/methods_course/
Image Analysis: A Review", IEEE
Rev. Biomed. Engg., vol. 2, pp. • TU Munich:
147-171, 2009. http://campar.in.tum.de/Chair/
• J. A. Noble, “Ultrasound Image TeachingWiSe2014CAMPONE
Segmentation: A Survey”, IEEE
Trans. Med. Imaging, vol. 25, no.
8, pp. 987-1010, 2006.
Medical Image Analysis - Introduction [Debdoot Sheet] 17
X-Ray Imaging and Computed
Tomography

Dr. Debdoot Sheet

Assistant Professor
Department of Electrical Engineering
Indian Institute of Technology Kharagpur

www.facweb.iitkgp.ernet.in/~debdoot/
 Contents
• X-ray generation
• Photon matter interaction
• X-ray attenuation in inhomogeneous
materials
• CT image formation
• Radon transform and Fourier slice theorem
• CT instrumentation
• CT reconstruction
X-Ray and Computed Tomography [Debdoot Sheet] 2
X-ray Generation

57.4 keV
(k-1 transition)
69.5 keV
(k-shell)

X-Ray and Computed Tomography [Debdoot Sheet] 3

 Photon Matter Interaction
coefficient (cm2/g)
Mass attenuation

𝑑𝑁 𝑥
= −𝜇𝑁 𝑥 𝑁 𝑙 = 𝑁(𝑒 *+,
𝑑𝑥

X-Ray and Computed Tomography [Debdoot Sheet] 4

Photon Matter Interaction

X-Ray and Computed Tomography [Debdoot Sheet] 5

 X-ray in Inhomogeneous Media

𝑁- = 𝑁( 𝑒 *+. /. *+0 /0 *+1 /1 𝑧

𝑥
𝑁 = 𝑁( 𝑒 * ∫5 + 3 43
𝑦

𝐼 𝑥, 𝑦 ∝ 𝑒𝑥𝑝 − ; 𝜇 𝑥, 𝑦, 𝑧 𝑑𝑧
=
X-Ray and Computed Tomography [Debdoot Sheet] 6
Computed Tomography

X-Ray and Computed Tomography [Debdoot Sheet] 7

 CT – Challenge?
𝐼> = 𝐼( 𝑒𝑥𝑝 −𝜇? 𝑑 − 𝜇@ 𝑑
𝐼A = 𝐼( 𝑒𝑥𝑝 −𝜇- 𝑑 − 𝜇B 𝑑
𝐼C = 𝐼( 𝑒𝑥𝑝 −𝜇? 𝑑 − 𝜇- 𝑑
𝐼4 = 𝐼( 𝑒𝑥𝑝 −𝜇@ 𝑑 − 𝜇B 𝑑

Solve for 𝜇? , 𝜇@ , 𝜇- , 𝜇B ?

With 𝑛 unknowns to solve for, 𝑛@ set of equations are needed.

X-Ray and Computed Tomography [Debdoot Sheet] 8
Radon Transform and Fourier Slice
𝑝 𝜌, 𝜃 = ; 𝜇 𝑠 𝑑𝑠 = ℜ 𝜌, 𝜃 𝜇
/

𝜇 𝑠 = ℜ *? 𝑝 𝜌, 𝜃
I

𝑝 𝑥 = ; 𝜇 𝑥, 𝑦 𝑑𝑦
*I
I I

𝑃 𝑢 = ; ; 𝜇 𝑥, 𝑦 𝑑𝑦 𝑒 *L@MNO 𝑑𝑥
*I *I

X-Ray and Computed Tomography [Debdoot Sheet] 9

 Fourier Slice Theorem
I I

𝑃 𝜃, 𝑤 = ; ; 𝜇 𝑥, 𝑦 𝑒 *L@MQ O RST UVW TXY U 𝑑𝑥𝑑𝑦

*I *I
I I

𝑀 𝑢, 𝑣 = ; ; 𝜇 𝑥, 𝑦 𝑒 *L@M OQ RST UVWQ TXY U 𝑑𝑥𝑑𝑦

*I *I

𝑃 𝜃, 𝑤 = 𝑀 𝑤 cos 𝜃 + 𝑤 sin 𝜃

X-Ray and Computed Tomography [Debdoot Sheet] 10

CT Instrumentation
Hounsfield Unit
𝜇 𝑥, 𝑦 − 𝜇Q>fgh
𝐻𝑈 𝑥, 𝑦 = 1000
𝜇Q>fgh

X-Ray and Computed Tomography [Debdoot Sheet] 11

CT Reconstruction
• Filtered Back Projection
(FBP)
• Algebraic
Reconstruction
Technique (ART)
• Simultaneous Iterative
Reconstruction
Technique (SIRT)
• Simultaneous Algebraic
Reconstruction
Technique (SART)

X-Ray and Computed Tomography [Debdoot Sheet] 12

CT Machine

X-Ray and Computed Tomography [Debdoot Sheet] 13

 Take home message
• M. A. Haidekker, Medical Imaging
Technology [Ch. 2 and 3], Springer, 2013.
• A. C. Kak and M. Slaney, Principles of
Computerized Tomographic Imaging [Ch.
7], SIAM Press, 2001.

X-Ray and Computed Tomography [Debdoot Sheet] 14

Magnetic Resonance Imaging

Dr. Debdoot Sheet

Assistant Professor
Department of Electrical Engineering
Indian Institute of Technology Kharagpur

www.facweb.iitkgp.ernet.in/~debdoot/
 Contents
• Proton spins in external magnetic field
• Spin echo
• Spin echo pulse sequence
• T1 and T2 relaxation times
• Gradient fields for MRI
• Instrument design

X-Ray and Computed Tomography [Debdoot Sheet] 2

Proton Spin in External Magnetic
Field

Larmor frequency
𝜔" = 𝛾𝐵

X-Ray and Computed Tomography [Debdoot Sheet] 3

 Spin Echo

𝛾
= 42.58𝑀𝐻𝑧
2𝜋
X-Ray and Computed Tomography [Debdoot Sheet] 4
 Spin Echo

X-Ray and Computed Tomography [Debdoot Sheet] 5

Spin Relaxation Time

X-Ray and Computed Tomography [Debdoot Sheet] 6

Spin Echo Pulse Sequence

X-Ray and Computed Tomography [Debdoot Sheet] 7

Gradient Field in MRI

X-Ray and Computed Tomography [Debdoot Sheet] 8

Instrument Design

X-Ray and Computed Tomography [Debdoot Sheet] 9

Clinical Use

X-Ray and Computed Tomography [Debdoot Sheet] 10

Contrast Comparison

X-Ray and Computed Tomography [Debdoot Sheet] 11

 Take home message
• M. A. Haidekker, Medical Imaging
Technology [Ch. 5], Springer, 2013.
• Tutorial Video -
https://youtu.be/djAxjtN_7VE

X-Ray and Computed Tomography [Debdoot Sheet] 12

Ultrasound Imaging

Dr. Debdoot Sheet

Assistant Professor
Department of Electrical Engineering
Indian Institute of Technology Kharagpur

www.facweb.iitkgp.ernet.in/~debdoot/
 Contents
• Acoustic wave propagation
• Reflection of waves
• Interference of waves
• Beamforming
• Ultrasound imaging instrumentation
• Speed of sound and properties
• Pulse propagation in lossy media
• Clinical use tally of ultrasound frequencies
Ultrasound Imaging [Debdoot Sheet] 2
 Acoustic Wave Propagation

Fresnel Zone Fraunhoffer Zone

(Near Field) (Far Field)
Surface
Ultrasound Imaging [Debdoot Sheet] 3
 Wave Reflection

Surface
Ultrasound Imaging [Debdoot Sheet] 4
Wave Interference

Beam

Ultrasound Imaging [Debdoot Sheet] 5

 Beam Forming

T1
Ampl. T2
time
Ultrasound Imaging [Debdoot Sheet] 6
Beam Forming

Ultrasound Imaging [Debdoot Sheet] 7

Imaging Instrumentation

Ultrasound Imaging [Debdoot Sheet] 8

Speed of Sound and Properties

Ultrasound Imaging [Debdoot Sheet] 9

Pulse Propagation in Lossy Media

Ultrasound Imaging [Debdoot Sheet] 10

Clinical Use Tally

Ultrasound Imaging [Debdoot Sheet] 11

Practical Device

Linear Probe Curvilinear Probe

Sectoral Probe
Ultrasound Imaging [Debdoot Sheet] 12
 Take home message
• M. Ali, D. Magee and U. Dasgupta, “Signal
Processing Overview of Ultrasound
Systems for Medical Imaging,” Texas
Instruments White Paper, Nov. 2008.
• M. A. Haidekker, Medical Imaging
Technology [Ch. 6], Springer, 2013.

Ultrasound Imaging [Debdoot Sheet] 13

Optical Microscopy and Molecular
Imaging

Dr. Debdoot Sheet

Assistant Professor
Department of Electrical Engineering
Indian Institute of Technology Kharagpur

www.facweb.iitkgp.ernet.in/~debdoot/
 Contents
• Microscope instrumentation
• Optical magnification
• Objective specifications
• Numerical aperture and light cone
• Numerical aperture and image resolution
• Focus depth
• Transmission vs. reflected light microscope
• Fluorescence microscopy
• Macroscopic molecular imaging with SPECT and
PET
X-Ray and Computed Tomography [Debdoot Sheet] 2
Microscope Instrumentation

X-Ray and Computed Tomography [Debdoot Sheet] 3

 Optical Magnification

Low Magnification Mid Magnification High Magnification

X-Ray and Computed Tomography [Debdoot Sheet] 4
 Objective Specifications

Opt. aber. corr.

Magnification
Water immersion
Focal correction

X-Ray and Computed Tomography [Debdoot Sheet] 5

NA and Light Cone

X-Ray and Computed Tomography [Debdoot Sheet] 6

NA and Image Resolution

X-Ray and Computed Tomography [Debdoot Sheet] 7

Focus Depth

X-Ray and Computed Tomography [Debdoot Sheet] 8

Transmission vs. Reflected Light
Microscope

X-Ray and Computed Tomography [Debdoot Sheet] 9

Fluorescence Microscopy

X-Ray and Computed Tomography [Debdoot Sheet] 10

Single Photon Emission Computed
Tomography (SPECT)

X-Ray and Computed Tomography [Debdoot Sheet] 11

Positron Emission Tomography
(PET)

X-Ray and Computed Tomography [Debdoot Sheet] 12

 Take home message
• K.D. Toennies, Guide to Medical Image
Analysis, [Ch. 2], Springer, 2012.
• Tutorial - http://zeiss-
campus.magnet.fsu.edu/tutorials/

X-Ray and Computed Tomography [Debdoot Sheet] 13