6/28/24, 2:38 PM Section 3: Introduction to Medical Imaging.

ipynb - Colab

Open in Colab

Author: Mana Moassefi, MD

keyboard_arrow_down Medical Imaging

keyboard_arrow_down Medical Imaging and Radiology:
Medical Imaging plays an essential role in all aspects of a patient's medical care. Different imaging modalities benefit us from the initial stages
of diagnosis to when the prognosis and survival need to be determined. The rapid technological advancement in this field caused wide
application. Depending on our needs, today, we can use various imaging methods. This chapter categorizes medical imaging technology,
divides each class into different modalities, and introduces them briefly.

According to the wavelength used for acquisition, medical images divide into non-radiologic and radiologic images. In non-radiologic ones,
visible light is used to observe. Typical examples are dermatologic photographs and pathology slides. In radiologic ones, the device uses a
different spectrum of electromagnetic and mechanical energy wavelengths, like X-ray in radiography, fluoroscopy, mammography, and
computed tomography (CT); radio frequency (RF) in magnetic resonance imaging (MRI); gamma-rays in nuclear medicine; the mechanical
energy of fast vibrating transducers, which exist in the form of high-frequency sound waves in ultrasound (US) imaging.

Should add ultrasound to the figure below--Maybe we should re-make anyway due to copyright issues? Also, probably change 'CT-SCAN' to
be: 'Radiography, Fluoroscopy and CT'(DONE)

Medical imaging can also be categorized into three distinct categories: structural, functional, and molecular imaging. Structural imaging
modalities are capable of visualizing different anatomical body parts. Functional imaging can capture dynamic alteration of regions of interest
(ROIs), interpreted as a specific function. This can be captured by modalities like functional MRI (fMRI) and functional near-infrared
spectroscopy (fNIRS). Recent improvements in imaging spatial and temporal resolution have made this technology possible. Molecular imaging
exhibits the structural and dynamical features of a biological molecule. Positron emission tomography (PET), magnetic resonance
spectroscopy (MRS), and other different modalities are now available to assess the ROI’s molecular information.

keyboard_arrow_down A. Structural Imaging:

keyboard_arrow_down 1. X-ray
Introduction:

https://colab.research.google.com/github/Mayo-Radiology-Informatics-Lab/MIDeL/blob/main/chapters/3A.ipynb#printMode=true 1/10
6/28/24, 2:38 PM Section 3: Introduction to Medical Imaging.ipynb - Colab
Plain radiography captures X-rays that have passed through the patient's body and captures the shadows on the image receptor. It creates a
two-dimensional image of a three-dimensional patient. Because it is a projection, it will have variable geometric effects such as magnification
that depends on distances of X-ray to patient versus patient to the receptor. In the past, the receptor was film but now, the receptor may be a
storage phosphor that needs to be 'read' to create the image (Computed Radiography) or the photons of the X-ray may be directly detected
(Digital Radiography). Fewer photons means more were absorbed by the body (e.g. bones) and these regions are traditionally rendered as
lighter/white. The total number of photons used depends on the X-ray generator, and it is adjusted to reflect the tissue being imaged (more
photons for thicker and more absorbing body parts). There is no standard number of photons for radiography, so the pixel values do not have a
mapping to the tissues being imaged.

keyboard_arrow_down Some medical applications and examples:

Radiography can be utilized on almost any body part, but structures containing radio-opaque materials like calcium in bones will have the
highest contrast. Therefore, it is very useful for muskuloskeletal diseases. Another common application is for the detection of lung
abnormalities. Chest x-ray (CXR) can visualize pathologies in the lungs, heart, lymph nodes, etc.(figure) Mammography, which also uses x-ray
technology, is utilized for breast cancer evaluation.

keyboard_arrow_down Physics:

A radiographic image is produced with the X-ray beam source on one side of the patient’s targeting area and the image receptor on the other
side. The radiation source is a small spot on a tube, and the X-ray beam diverges as it passes from the source and gets to the patient and
ultimately the receptor. Due to this divergence effect, as the beam moves further from the source, it becomes larger, and the intensity
decreases. Therefore, the image is a magnified version of the object being radiographed. As the x-ray beams pass through the patient, they are
weakened to different degrees by anatomical structures in the exposed field. Materials with high electron density will absorb the most radiation.

https://colab.research.google.com/github/Mayo-Radiology-Informatics-Lab/MIDeL/blob/main/chapters/3A.ipynb#printMode=true 2/10
6/28/24, 2:38 PM Section 3: Introduction to Medical Imaging.ipynb - Colab

source:https://www.researchgate.net/publication/325342963_X-ray_tubes_development

After detection and processing steps, areas where the beam was not attenutated will traditionally be rendered as black (though inverse
rendering is occasionally used) and areas where the beam was heavily attenuated will be white. A difference in density, as well as in thickness,
of the structure within the body, determines the overall contrast in the image.

Type of data and format:

Medical images are typically stored in either a Picture Archiving and Communication System (PACS) which is usually a departmental system, or
in a Vendor-Neutral Archive (VNA) that is usually shared accross a hospital. Medical images are typically saved as Digital Imaging and
Communications in Medicine (DICOM) files. A DICOM file contains pixel values (image data) and metadata. The metadata includes patient
information (like age, gender, name, medical record number) and image acquisition information such as: modality, pixel spacing, number of
pixels in each direction, device information like the vendor and model, and physical properties like kVp and mAs. DICOM images (which
frequently have '.dcm' as extension but sometimes have no extension) can be converted to other imaging formats that are used for non-medical
applications. A few examples are:

Portable Network Graphics (PNG)

Tag Image File Format (TIFF)

Joint Photographic Expert Group (JPEG)

Neuroimaging Informatics Technology Initiative (NIFTI)

These formats do not support all the metadata such as patient and acquistion parameters, but some will save the physical properties like pixel
spacing.

There are also DICOM files that do not contain pixel data. DICOMDIR files provide a description of a collection of a collection of DICOM files and
can be faster than parsing each individual file. There are also DICOM files that contain other information like segmentation masks (DICOM-
SEG), report information (DICOM-SR) and treatment plans that can also have information about the spatial relationship of one set of images
and another set (DICOM-RT).

keyboard_arrow_down 2. CT scan
Introduction:

Computed Tomography (CT) is a diagnostic imaging technique that builds cross-sectional images, using X-ray. The patient lying on a bed is
moved through a doughnut-shaped structure that contains an X-ray source (or sources) and a ring (today, there are many rings) of detectors. By
acquiring these projections from at least 180 degrees around the patient, a cross-sectional image can be computed, hence its name.

https://colab.research.google.com/github/Mayo-Radiology-Informatics-Lab/MIDeL/blob/main/chapters/3A.ipynb#printMode=true 3/10
6/28/24, 2:38 PM Section 3: Introduction to Medical Imaging.ipynb - Colab

keyboard_arrow_down Some medical applications and examples:

Cross-sectional CT scan images are commonly used in medicine. These can be utilized for diagnosis and follow up of various situations:

Neurologic diseases like intracranial hemorrhage or stroke.

Cardiothoracic diseases like lung cancer, coronary artery calcification, or interstitial lung disease.
Abdominal diseases like cancers, inflammation, or internal bleeding.
Musculoskeletal disease like fractures, and tumors.

keyboard_arrow_down Physics:

CT scan acquisition is based on the principle of calculating the attenuation coefficient of multiple x-ray beams to determine the density of a
tissue. CT allows for the reconstruction of cross-sectional images of the body's density. The x-ray emitter rotates around the patient. There is a
ring of detectors opposite the source that detects this 'fan beam'. These beam and detector ring move together in synchrony (or in some cases,
the detector ring goes all around the patient and is stationary). In this way, they collect projections from all angles (actually only 180 degrees is
required) to produce an anatomical section image. Modern scanners typically have multiple detector rings, allowing simultaneous acquisition of
multiple slices (e.g. '64-slice scanner'). Modern scanners also typically move the patient continuously as the source and detector move around
the patient. This is known as 'helical' CT and allows for faster acquisitions.

We won't describe the math here, but it is important to know that the reconstruction has a 'kernel' and that kernel can accentuate edges or
suppress noise (but these are a tradeoff). Therefore, it is common to produce more than 1 set of images from a given acquisition. And while the
images are acquired such that the slices are perpendicular to the long axis of the body (hence the term 'Axial') they can be mathematically
reformatted to be coronal (this comes from the coronal suture and is a plane approximately parallel to the forehead) or sagittal (sagittal suture
is the joint between the left and right halves of the skull) or any arbitrary plane (often referred to as 'oblique').

source: https://bookmerilab.com/blog/cect-abdomen-cost/

https://colab.research.google.com/github/Mayo-Radiology-Informatics-Lab/MIDeL/blob/main/chapters/3A.ipynb#printMode=true 4/10
6/28/24, 2:38 PM Section 3: Introduction to Medical Imaging.ipynb - Colab

CT images show differences in X-ray attenuation within the body. Typically, the CT image consists of 512 rows of 512 pixels, the square matrix
being 512 x 512 = 262,144 pixels. Since we acquire a volume of tissue attenuation measurements, the 3D array of pixels is often referred to as
'voxels' (volume pixels). The unit of attention in CT is the Hounsfield Unit (HU), in honor of its inventor Sir Godfrey Hounsfield 6. He specified a
linear attenuation map in which water was 0 HU and air was -1000 HU. All other tissues then map to this scale. While air and water are
homogeneous and consistent, biological tissues are usually not, and so a range of intensities is seen, and they often overlap. Some example
values are:

lung = -850 to -910 HU

fat = -60 to -120 HU
brain = 20 to 45 HU
muscle = 35 to 65 HU
hematoma = 50 to 90 HU
compact bone = >1000 HU

Type of data and format:

A typical CT examination will include 1 or more 'scout' images that are projection radiographs--the patient slides through the gantry but the
source and detectors do not move. This is used to set the top and bottom of the body region to be scanned. There will then be 1 or more CT
acquisitions--sometimes one is done before injecting intravenous contrast, and sometimes 1 or more acquisitions (also known as 'series') after
injecting contrast, capturing different phases of its movement through the vascular system. Because the filename of each slice does not
necessarily encode any meaning about the series and slice location, it is necessary to examine the values in each file of an examination and
sort them. This allows you to create 3D datasets for each series (other than the scout series). Typically, all the images of a series are sorted and
then saved into a NIFTI file (one poular tool that does all the sorting and conversion is called dcm2nii). The NIFTI file may be read into a 3D
Numpy array. This array can be visualized using libraries like matplotlib or Pillow. It is usual to find CT-scan slices in other conventional imaging
formats after conversions like JPG or PNG. See chapter XX for details about the DICOM format and its conversion in other chapters.

keyboard_arrow_down 3. MRI
Introduction:

MRI scans create detailed cross-sectional images of internal body organs using a large magnetic field, radiofrequency waves, and math.
Typically, a scanner is shaped like a large tube with a table in the middle, allowing the patient to slide in and out. This tube houses the large
magnet that has a static field that describes its strength (e.g. 3 Tesla). The transmitter emits radio waves into the body, and because of the
static field, protons are impacted by the correct frequency radio wave. The impact is to flip some of them, and the amount they are flipped
depends not only on the radio wave but also on the properties of the tissue. Because of the large field, the protons will then revert back to being
aligned with the magnet, but as they do so, they emit a radio frequency signal reflecting their properties. A computer transforms the signal into
an image using a techinque called the Fourier Transform. Contrary to CT scanning, MR imaging does not use ionizing radiation and it can
directly acquire images in any plane, not just axial.

keyboard_arrow_down Some medical applications and examples:

In MRI, highly detailed information is collected about the contents of the tissue. Soft tissue does not have much difference in X-ray attention so
often disease is not well detected. MRI is amazingly flexible and can show many different properties of tissue such as how many protons there
are, how easily water can diffuse, whether tissue (like blood) is moving, and specific chemical properties known as 'T1' and 'T2' times. There are
other forms of MR like spectropscopy that we will only mention here, and we note that MRI is a highly innovative field with many new imaging
types being created. (This is wonderful for medicine, but standardizing the DICOM tags when there is rapid innovation is a challenge).

Some of the most common MRI applications are:

Musculoskeletal Imaging, tendons, ligaments, and also bone diseases e.g osteomyelitis.
Abdominal Imaging, Small bowel diseases e.g inflamatory bowel disease (IBD).
Neuroimaging, Brain and spinal cord diseases e.g multiple sclerosis (MS).

As an alternative to CT or conventional angiography, MR angiography (MRA) and MR venography (MRV) show arteries and veins, respectively.

https://colab.research.google.com/github/Mayo-Radiology-Informatics-Lab/MIDeL/blob/main/chapters/3A.ipynb#printMode=true 5/10
6/28/24, 2:38 PM Section 3: Introduction to Medical Imaging.ipynb - Colab

keyboard_arrow_down Physics:

MR images are more complex and difficult to decipher than CT images or ultrasound images. The following is an abbreviated description of the
acquisition process. A four-part process can be outlined as follows:

1. preparation

2. excitation

3. spatial encoding

4. signal acquisition

keyboard_arrow_down Preparation:

Each hydrogen atom in a tissue possesses an intrinsic magnetization called spin. The patient lies in a static magnetic field generated by the
MRI scanner's magnet. It results in the magnetization of a majority (it is actually only a slight majority: 1 in 100,000 for 1.5T scanner) of protons
inside the patient's body tissues. A stronger field increase this proportion slightly. However, for clarity purposes, it is typically described as if all
were aligned with the field; a simplification we will follow here.

Excitation:

In order to capture an image, the scanner emits a radiofrequency (RF) pulse at the Larmor frequency (approximately 64Mhz times the field
strength so around 98Mhz for 1.5T). This will cause (some) protons to 'flip' so that they are not aligned with the static field. How far they flip
depends on the duration of the RF pulse, but commonly they are flipped 90 or 180 degrees. Since the static field is still present, the protons will
'process' to align back with the field, but as they do so, they emit their own RF signal. That signal is detected by a receiver coil. Just as with
other radio signals, the signal will be stronger if the coil is closer, but today we also often use arrays of coils just like an array of antennas.

The procession of the protons as they realign is affected by two simultaneous relaxation processes. The signal is attenuated due to loss of
coherence among the protons by a time constant called the transverse relaxation time (T2). Concurrently, the protons realign to the magnetic
field, and this is called the spin-lattice relaxation time (T1). T1-weighted images strongly reflect T1 times (tisues with short T1 times like fat are
bright). T2-weighted images reflect the T2 times (tissues with long T2 times like water are bright). We should note here that while all tissues
have some protons, the protons must be able to flip as described above. Cortical bone has tightly bound protons, and thus it appears black--no
signal is produced.

Spatial encoding:

https://colab.research.google.com/github/Mayo-Radiology-Informatics-Lab/MIDeL/blob/main/chapters/3A.ipynb#printMode=true 6/10
6/28/24, 2:38 PM Section 3: Introduction to Medical Imaging.ipynb - Colab
MRI signals are encoded spatially using magnetic field gradients: The procession rate of protons is made to vary slightly by super-imposing a
magnetic field over the patient's body, enabling the frequency to identify the location that an RF signal came from. The direction of this gradient
is referred to as the 'frequency encoding' direction.

MRI also applies a gradient field for a set time, causing some protons to 'get ahead' of the others. When it is turned off, the phase of the RF
signal reflects the position. This is referred to as 'phase encoding'.

Signal acquisition:

MRI signal is created due to different frequencies corresponding to the different tissue spin positions. This acquisition happens with the use of
magnetic field gradients. After sampling, a digital MRI signal is created from an analog signal and stored for processing. An image is formed by
separating the signals from different spatial locations represented by pixels. This is achievable by using a method referred to as a Fourier
transform.

Types of data and format:

Like the CT, MR images are stored in DICOM and can then be converted to NIFTI or other formats.

keyboard_arrow_down 4. US
Introduction:

US is an imaging technology that uses high-frequency sound waves to characterize tissue. It is a valuable and flexible modality and often
provides a unique characterization of tissues, compared with other modalities such as X-ray or MRI. According to the probe's position, different
planes of the body can be imaged.

keyboard_arrow_down Some medical applications and examples:

A few US application examples are:

Neonatal neuroimaging: screening the fetus health, brain hemorrhage (possible in neonates because the skull is not ossified).
Musculoskeletal: joint effusion, ligament, and tendon pathologies.
Echocardiography: heart and large vessel pathologies and cardiac function assessment.
Genitourinary system: kidneys, proximal ureters, bladder, prostate gland.
Gynecological: adnexal, uterus, cervical, tubal pathologies.
Obstetrics: Diagnosis fetal disease, fetal growth follow up.

keyboard_arrow_down Physics:

An ultrasound machine generates ultrasound waves and receives their echoes. Spatial and contrast information is contained in the echoes. In
order to create images on the ultrasound screen, waves reflect from various body tissues to the transducer.

https://colab.research.google.com/github/Mayo-Radiology-Informatics-Lab/MIDeL/blob/main/chapters/3A.ipynb#printMode=true 7/10
6/28/24, 2:38 PM Section 3: Introduction to Medical Imaging.ipynb - Colab

The resistance of ultrasound waves to propagation is called the acoustic impedance. The impedance value varies with the density of the
material that ultrasound passes through.

Air: Air does not transmit Mhz frequency sound well, and so it is challenging to visualize structures behind air containing structures like
lung or GI tract.
Fluid: Sound waves are transmitted more easily by fluid than by solid materials. This will result in a reduction in ultrasound reflection from
fluids.
Solid: The sound waves reflected by solid-body components like a bone are brighter than those reflected by fluids. A black acoustic
shadow will appear behind solids as ultrasound waves cannot transmit through them.

Medical ultrasound uses sound waves with a frequency in the millions of cycles per second (MHz). Ultrasound images are acquired rapidly and
are useful for rapidly moving structures like the heart.

In addition to measuring reflection of sound waves, Doppler imaging allows the measurement of speeds, such as blood flow. Based on
experience, certain flow velocities are known to reflect certain degrees of stenosis, such as in carotid arteries.

There is an evolutional trend in ultrasound imaging functionality today. There are several new characteristics like 3D imaging acquisition as well
as the ability to measure tissue stiffness.

keyboard_arrow_down 5. Visible light imaging:

Introduction:

Various non-radiologic imaging methods rely on visible light. Photography is an example of visible light imaging we do in everyday life. Similar
technology is used for assessing skin lesions. More specialized cameras can get images of the retina (e.g. fundoscope). One may also attach a
camera to a microscope to capture histopathology and similar types of images.

keyboard_arrow_down Some medical applications and examples:

Microscopy: Microscopy is a method used to inspect objects that are too tiny to be seen by the bare human eyes. Different microscopes
have different applications in medicine. The most famous microscope model used in general is a light microscope. This type is utilized for
finding pathologies in tissues. The other type is a fluorescent microscope. Other than cell structure, this type can detect specific and
marked molecules in a field of evaluation. The last type we mention here is an electron microscope which can evaluate tissues at
subcellular and big molecular levels.

Fundoscopy: Fundoscopy is the exam of looking at structures in the innermost area of the optic globe. This inspection is done using an
instrument called an ophthalmoscope. Valuable information around a person’s general health and also specific optical defects can be
gathered by this examination. Hypertension and diabetes are two common diseases, and their long-term effects can be evaluated using
fundoscopy.

Dermoscopy: Dermoscopy is a technique to magnify suspicious skin lesions.

https://colab.research.google.com/github/Mayo-Radiology-Informatics-Lab/MIDeL/blob/main/chapters/3A.ipynb#printMode=true 8/10
6/28/24, 2:38 PM Section 3: Introduction to Medical Imaging.ipynb - Colab

keyboard_arrow_down Physics:

These instruments work primarily by collecting visible light from a small zone on the specimen and passing it through the objective lens. The
sample image gets magnified by the lens, and also it passes through the eyepieces. The microscopes with more powerful configurations have
stronger lenses. Another reason why this imaging category is more practical for machine learning applications is the very high-resolution
cameras attached to the microscopes to collect high-quality data for analysis. Ophthalmoscopes throw light on the retina through the pupil. The
reflection of the light rays passes back through the pupil. The observing window of the ophthalmoscope contains a lens that magnifies the
captured image of the retina. A second lens directs the light onto the retina of the observer.

source:
https://www.jaypeedigital.com/book/9789350908525/chapter/ch11

Types of data and format:

The images collected by cameras attached to the microscopes, fundoscopy, or dermatoscopy are called digital slides. Due to the digital slide’s
high resolution, special technqiues are used to make them manageable. Such images are often gigabytes in size, and it is common to want to
see low-resolution and high-resolution images of the same region. While there is a DICOM specification for pathology images, it is not widely
used. Instead, another format called TIFF is used for saving and restoring these files. Slides can also be stored in other conventional imaging
formats such as JPG and PNG.

keyboard_arrow_down B. Functional Imaging:

keyboard_arrow_down 1. fMRI:
Introduction:

fMRI measures slight changes in blood flow associated with brain activity. Other than the clinical field, there is extensive utilization of this
modality in the research area. It uses the same technology as MRI. ​Through physical sensations (sight, sound, touch, taste, smell) or activities
such as problem-solving and movement (limited by the machine), it can measure the activation of brain areas. Although there are more invasive
techniques for measuring neuronal activity in animals, fMRI is a less invasive technique that enables researchers to investigate human brain
function and dysfunction.. This modality focuses on dynamic changes in the brain. That is why other imaging techniques may not find
abnormalities within the brain detected with fMRI.

keyboard_arrow_down Some medical applications and examples:

Using fMRI makes it possible to perform different types of analyses and evaluations. The first can be identifying a connection between an
activity (or inside the resting state) and a region of interest. The second is finding a circuit of nodes connected through a task or resting

https://colab.research.google.com/github/Mayo-Radiology-Informatics-Lab/MIDeL/blob/main/chapters/3A.ipynb#printMode=true 9/10
6/28/24, 2:38 PM Section 3: Introduction to Medical Imaging.ipynb - Colab
situation. Finding an abnormal function in the node or the circuit and association of that with a disease or abnormality can be the next step.
fMRI is frequently used in cognitive neuroscience to examine the correlation between neural activity and specific cognitive processes.

Physics and physiology:

Like structural MRI, hydrogen atoms provide blood oxygen level-dependent (BOLD) fMRI signals. BOLD imaging is a special sequence that is
particularly sensitive to dephasing, and the magnetic field around deoxyhemoglobin is inhomogeneous, as opposed to oxyhemoglobin. Hence,
As deoxyhemoglobin concentration increases, image intensity decreases.

The brain increases local blood flow in response to glucose and oxygen demands. While one might expect this to decrease oxygen levels in the
veins, the opposite is actually true because the brain over-compensates. This reaction also takes a few seconds to happen. Still, the signal is
weak, so multiple acquisitions are required to achieve a sufficiently high signal-to-noise ratio. Then the images without and with the stimulus
are compared, and the images produced reflect how different the images are.

keyboard_arrow_down C. Molecular Imaging (a.k.a. Nuclear Medicine):

keyboard_arrow_down 1. Positron Emission Tomography (PET):
Introduction:

PET is an imaging technique utilized in nuclear medicine that relies upon the disintegration of positrons. Commonly used positron-emitting
atoms are Fluorine, Carbon, and Oxygen. These are popular because they can be synthesized into common biological molecules. For example,
fluoro-deoxy glucose (FDG) is where flourine replaces an oxygen in glucose. The fluorine atom gets 'stuck' in the reactions that break down
glucose, and thus reflects where glucose is being consumed.

keyboard_arrow_down Some medical applications and examples:

https://colab.research.google.com/github/Mayo-Radiology-Informatics-Lab/MIDeL/blob/main/chapters/3A.ipynb#printMode=true 10/10