Capstone Speech
Capstone Speech
affected by colon cancer within 3D medical images. This model will help in accurately pinpointing
areas of concern, aiding medical professionals in diagnosis and treatment planning.
To improve model accuracy, we will have to employ transfer learning. This technique leverages pre-
trained models.
While our ultimate goal is to segment 3D images, we will train and validate our model using 2D
images. These 2D images will be derived from the 3D medical scans by slicing them into individual 2D
sections.
Once trained, we will demonstrate the model's capability by performing segmentation on both 2D
slices and entire 3D images. This dual approach ensures the model's versatility and effectiveness in
real-world scenarios.
Code
import os
test_images_path = '/content/drive/MyDrive/Soumya S
Moharana/Mock_Project/imagesTs'
test_image_files = [f for f in os.listdir(test_images_path) if
f.endswith('.nii.gz') and not f.startswith('._')]
print("Number of testing dataset: {}".format(len(test_image_files)))
train_labels_path = '/content/drive/MyDrive/Soumya S
Moharana/Mock_Project/labelsTr'
train_labels_files = [f for f in os.listdir(train_labels_path) if
f.endswith('.nii.gz') and not f.startswith('._')]
print("Number of label dataset: {}".format(len(train_labels_files)))
The code scans specific directories (imagesTr, imagesTs, labelsTr) in Google Drive to collect the names
of files (images and labels) for training and testing.
It filters the files based on their extensions (.nii.gz) to ensure they are the correct type of medical
image files.
It counts and prints the number of files found in each directory. This provides an overview of the
dataset size, indicating there are 126 training images, 64 testing images, and 126 corresponding label
images.
Code
import os
import nibabel as nib
import numpy as np
import matplotlib.pyplot as plt
nifti_image = nib.load(nifti_image_path)
plt.figure(figsize=(15, 3))
for i, slice_index in enumerate(slice_indices):
plt.subplot(1, num_slices, i + 1)
plt.imshow(image_data[:, :, slice_index])
plt.title(f'Slice {slice_index}')
plt.axis('off')
plt.show()
The code defines a function display_nifti_slices that takes a path to a NIfTI image (nifti_image_path)
and an optional parameter num_slices (default is 5) that specifies the number of slices to display.
It loads the NIfTI image from the specified path using nib.load(nifti_image_path). This function
returns a NIfTI image object.
It extracts the image data as a Numpy array using nifti_image.get_fdata(). This step is crucial as it
converts the NIfTI image data into a format that can be easily manipulated and visualized using
Numpy and Matplotlib.
It calculates the indices of the slices to display based on the number of slices requested (num_slices)
and the total number of slices in the image (image_data.shape[2]).
It then iterates over the selected slice indices, plots each slice using Matplotlib (plt.imshow), and
displays the slice number as the title. The plt.subplot function is used to arrange the slices
horizontally.
When the function is called with a path to a NIfTI image, it will display the specified number of slices
from the image, providing a visual representation of the image data.
Code
def display_all_nifti_slices(nifti_image_path):
nifti_image = nib.load(nifti_image_path)
image_data = nifti_image.get_fdata()
num_slices = image_data.shape[2]
print("Number of slices: {}".format(num_slices))
num_cols = 10
num_rows = (num_slices // num_cols) + (1 if num_slices % num_cols !
= 0 else 0)
The code defines a function display_all_nifti_slices that takes a path to a NIfTI image
(nifti_image_path) as input.
It loads the NIfTI image from the specified path using nib.load . This function returns a NIfTI image
object.
It calculates the total number of slices in the image (num_slices) using the third dimension of the
image data array (image_data.shape[2]).
It calculates the number of columns (num_cols) to display the slices in each row and the number of
rows (num_rows) needed to display all slices. It ensures that there are at most num_cols slices in
each row, and calculates the number of rows accordingly.
It iterates over all slices and plots each slice using Matplotlib (plt.imshow). The plt.subplot function is
used to arrange the slices in a grid layout based on the calculated number of rows and columns.
Code
Then i have applied the same code for labels and testing data.
Code
if train_image_files:
image_path = os.path.join(train_images_path, train_image_files[1])
print(f"Displaying all slices for: {train_image_files[1]}")
display_all_nifti_slices(image_path)
else:
print("No training images found.")
The code checks if train_image_files is not empty . If there are training image files available, it
executes the following block of code. Otherwise, it prints "No training images found."
‘’’If there are training image files, the code constructs the full path to the second image file in the
train_image_files list (train_image_files[1]) by joining train_images_path (the directory path where
the images are stored) with the image file name.’’’
def get_num_slices(nifti_image_path):
nifti_image = nib.load(nifti_image_path)
image_data = nifti_image.get_fdata()
return image_data.shape[2]
this function loads the NIfTI image located at nifti_image_path using nib.load(nifti_image_path).It
then retrieves the image data from the NIfTI image .The function returns the number of slices in the
image, which is the size of the third dimension of the image data array (image_data.shape[2]).
‘’’After defining the function, the code iterates through each file in train_image_files, which contains
the list of training image file names. For each image file, it constructs the full path to the image file
and calls the get_num_slices function to get the number of slices in that image.It then prints a
message indicating the file name and the number of slices in that file.
plt.figure(figsize=(15, 6))
for i, slice_index in enumerate(slice_indices):
plt.subplot(2, num_slices, i + 1)
plt.imshow(image[:, :, slice_index], cmap='gray')
plt.title(f'Image Slice {slice_index}')
plt.axis('off')
This function takes three arguments: image_path, label_path, and an optional argument num_slices
with a default value of 5.
It then calculates a list of slice_indices it generates num_slices equally spaced indices along the third
dimension of the image (slices).
It plots the corresponding label slice (label[:, :, slice_index]) in the bottom row
The code uses a for loop to iterate over a range of values. It limits the iteration to the minimum value
among num_images_to_display, the length of train_image_files, and the length of train_labels_files.
Inside the loop, it constructs the paths to the image and label files for the current iteration.
This function displays the image slices along with their corresponding label slices.
return {
'total_slices': total_slices,
'min_slices': min_slices,
'max_slices': max_slices,
'mean_slices': mean_slices,
'slice_counts': slice_counts
}
train_stats = calculate_statistics(train_image_files,
train_images_path)
test_stats = calculate_statistics(test_image_files, test_images_path)
Here The function calculate_statistics of the images. It iterates over each image_file in image_files.
For each image_file, it calculates the number of slices (num_slices) using the get_num_slices
function
For each image_file, it updates the total_slices count and appends num_slices to slice_counts.
It updates min_slices and max_slices based on the minimum and maximum num_slices found.
After the loop, it calculates the mean_slices by dividing total_slices by the number of image_files.
The function returns a dictionary containing the calculated statistics: total_slices, min_slices,
max_slices, mean_slices, and slice_counts.
This function is useful for understanding the distribution of the number of slices in the dataset, which
can provide insights into the dataset's characteristics.
These statistics can help in further analysis and decision-making, such as determining the range of
slices to consider for training or evaluating the segmentation model.
.(----continue----)
visualize_slice_distribution(train_stats['slice_counts'], 'Training
Images Slice Distribution')
visualize_slice_distribution(test_stats['slice_counts'], 'Testing
Images Slice Distribution')
The code generates histograms that visually represent the distribution of the number of slices across
all images in the training and testing datasets. These visualizations help in understanding how the
number of slices varies among the images.
os.makedirs(output_image_dir, exist_ok=True)
os.makedirs(output_label_dir, exist_ok=True)
num_slices = image.shape[2]
for i in range(num_slices):
image_slice = image[:, :, i]
label_slice = label[:, :, i]
image_slice_path = os.path.join(output_image_dir,
f"{os.path.basename(image_path)[:-7]}_{i}.png")
label_slice_path = os.path.join(output_label_dir,
f"{os.path.basename(label_path)[:-7]}_{i}.png")
cv2.imwrite(image_slice_path, image_slice)
cv2.imwrite(label_slice_path, label_slice)
output_image_dir, output_label_dir are the directories where the 2D image slices and their
corresponding label slices will be saved.
‘’’These lines create the output directories if they do not already exist. The exist_ok=True parameter
prevents errors if the directories already exist.’’’
‘’’num_slices = image.shape[2]
This line retrieves the number of slices in the 3D image by accessing the third dimension of the
array.’’’
‘’’for i in range(num_slices):
image_slice = image[:, :, i]
label_slice = label[:, :, i]
This loop iterates over each slice in the 3D image and its corresponding label.’’’
These lines create file paths for saving the image and label slices. The base name of the original
file is used with the slice index appended to create unique filenames.
OpenCV is used to save the image and label slices as PNG files in the specified directories.
This loop iterates over all training images and their corresponding labels, converting each pair of 3D
files into 2D slices and saving them using the convert_3d_to_2d function.
This comprehensive approach ensures that the 3D medical images are appropriately processed and
prepared for effective model training, enabling accurate semantic segmentation
import glob
import tensorflow as tf
from sklearn.model_selection import train_test_split
label = tf.io.read_file(label_path)
label = tf.image.decode_png(label, channels=1)
label = tf.image.resize(label, [128, 128])
label = tf.image.convert_image_dtype(label, tf.float32)
output_image_dir = 'output_images'
output_label_dir = 'output_labels'
image_files = sorted(glob.glob(os.path.join(output_image_dir,
'*.png')))
label_files = sorted(glob.glob(os.path.join(output_label_dir,
'*.png')))
Here I have prepared the image and label data for training a deep learning model. It ensures that the
data is properly loaded, processed, and split into training and validation sets. The shapes of the
images and labels are also checked to ensure they are compatible with the model architecture.
Loaded the image and label and decoded into into an image tensor then grayscale to RGB, Resized
the image to 128x128 pixels, Converted the image to a float32 type for model compatibility.
Then Createed a dataset from the file paths,Mapped the load_image function to load and
preprocess the images and labels, Shuffled the dataset with a buffer size of 1000, Prefetched data to
improve performance during training.
‘’’Used glob to list all PNG files in the output directories for images and labels.’’’
Splited the image and label files into training (80%) and validation (20%) sets.
Printed the shapes of the first batch of images and labels from the training and validation datasets.
This code prepares the dataset for training a semantic segmentation model, Efficiently handles large
datasets by using TensorFlow's data pipeline features like AUTOTUNE, shuffle, batch, and prefetch.
This systematic approach to dataset preparation and loading sets the stage for training a robust
semantic segmentation model for colon cancer detection.
Here I have Defined a function to calculate the Dice score, which is a metric for evaluating the
similarity between two sets of data. This function will be used to assess the performance of the
segmentation model.
y_true: The ground truth labels.
Flattens the y_true tensor into a 1D array:This simplification allows for easier calculation of the
intersection and union of the true and predicted labels.
Computes the intersection between the flattened true and predicted labels:This intersection
represents the overlap between the true and predicted segmentation areas, crucial for the Dice
score calculation.
The Dice score ranges from 0 to 1, where 1 indicates perfect overlap between the predicted and true
labels. Adding the smooth term prevents division by zero and stabilizes the score.
def build_unet(input_shape):
inputs = Input(shape=input_shape)
c1 = base_model.get_layer('block1_conv2').output
c2 = base_model.get_layer('block2_conv2').output
c3 = base_model.get_layer('block3_conv3').output
c4 = base_model.get_layer('block4_conv3').output
c5 = base_model.get_layer('block5_conv3').output
u6 = UpSampling2D((2, 2))(c5)
u6 = Concatenate()([u6, c4])
c6 = Conv2D(512, (3, 3), activation='relu', padding='same')(u6)
c6 = Conv2D(512, (3, 3), activation='relu', padding='same')(c6)
u7 = UpSampling2D((2, 2))(c6)
u7 = Concatenate()([u7, c3])
c7 = Conv2D(256, (3, 3), activation='relu', padding='same')(u7)
c7 = Conv2D(256, (3, 3), activation='relu', padding='same')(c7)
u8 = UpSampling2D((2, 2))(c7)
u8 = Concatenate()([u8, c2])
c8 = Conv2D(128, (3, 3), activation='relu', padding='same')(u8)
c8 = Conv2D(128, (3, 3), activation='relu', padding='same')(c8)
u9 = UpSampling2D((2, 2))(c8)
u9 = Concatenate()([u9, c1])
c9 = Conv2D(64, (3, 3), activation='relu', padding='same')(u9)
c9 = Conv2D(64, (3, 3), activation='relu', padding='same')(c9)
model = build_unet(input_shape)
model.compile(optimizer='adam', loss='binary_crossentropy',
metrics=['accuracy', dice_score])
model.summary()
Then I have created the U-Net model using the VGG16 architecture as a backbone. The U-Net is a
popular architecture for image segmentation tasks, which in this case will be used for segmenting
medical images.
Loads the VGG16 model pre-trained on ImageNet, excluding the top classification layer.
C1,c2,c3,c4,c5 Extracts the outputs of specific convolutional layers from VGG16. These layers serve as
the "contracting path" of the U-Net, capturing different levels of features.
Then Upsampled the feature maps and concatenate them with corresponding feature maps from the
contracting path. ‘’’it Combines high-level features with spatial information from earlier layers,
critical for precise segmentation.’’’
output: Defines the final output layer with a sigmoid activation function.‘’’Produces the final
segmentation mask with values between 0 and 1.’’’
batch_size = 16
epochs = 10
history = model.fit(train_dataset,
validation_data=val_dataset,epochs=epochs)
Then I have Evaluated the model's performance on the validation dataset : it Provides a final
assessment of the model's ability to generalize to unseen data after training.
The high validation accuracy and reasonable Dice score suggest that the model can reliably segment
medical images
prediction = model.predict(image_slice)[0]
plt.figure(figsize=(15, 5))
plt.subplot(1, 2, 1)
plt.title("Input Image")
plt.imshow(image_slice.numpy().squeeze())
plt.subplot(1, 2, 2)
plt.title("Prediction")
plt.imshow(prediction.squeeze(), cmap='gray')
plt.show()
here I have Loop through the first three test images and visualize their predictions. have taken a
random slice and applied prediction on it
image = nib.load(image_path).get_fdata()
image_slice = image[:, :, slice_idx]
image_slice = np.expand_dims(image_slice, axis=-1)
image_slice = cv2.resize(image_slice, (128, 128))
image_slice = tf.convert_to_tensor(image_slice, dtype=tf.float32)
# Ensure the last dimension is size 1
if image_slice.shape[-1] != 1:
image_slice = tf.expand_dims(image_slice, axis=-1)
image_slice = tf.image.grayscale_to_rgb(image_slice)
image_slice = tf.expand_dims(image_slice, axis=0)
return image_slice
Here I have Preprocessed a 3D medical image by Loading the 3D image, extracting a specific slice,
resize it, convert it to a tensor, ensure its shape is correct, convert it to RGB format, and expand its
dimensions to match the model's input shape.
Then Performed inference on a 3D medical image by processing each slice separately and collecting
the predictions. Loaded the 3D image, iterated over each slice, preprocessed each slice, made
predictions using the model, and collected the predictions.
’’’ The `np.squeeze` function removes dimensions of size 1 from the array. This means that if any
dimension of the array has only one element, that dimension is removed.
- This can be useful for simplifying the shape of the array and making it easier to work with.’’’
output_path = '/content/drive/MyDrive/Soumya S
Moharana/Mock_Project/predictions.nii.gz'
save_predictions_as_nifti(predictions, test_image_path, output_path)
here is the function…to save the model’s prediction as a NIfTI file , Loaded the original image to
extract the affine transformation and header information. Then, created a new NIfTI image with the
model's predictions using the same affine and header information and saved it to the specified
output path.
display_original_and_predicted_slices(test_image_path,output_path)
finally displaying the original and predicted slices, it loads the test image and its corresponding
predicted image then it calculates the indices of slices to display This ensures that the selected slices
are evenly spaced across the z-axis (slices dimension) of the 3D image.