Course: Machine Learning

Topic: Analyzing the performance of ML algorithms in Prostate Cancer

prediction

Student: Dimitar Mitrevski, CSE

Mentor: Assistant Professor Ljubinka Gjergjeska Sandjakoska, Ph.D.

Ohrid, May 2023

Introduction
Prostate cancer is a major public health concern, with high incidence rates and significant
impact on patient morbidity and mortality. Timely and accurate detection of prostate cancer is
crucial for effective treatment planning and improved patient outcomes. In recent years,
machine learning (ML) algorithms have emerged as powerful tools for analyzing complex
medical data and aiding in disease prediction and diagnosis. These algorithms have the
potential to enhance the accuracy and efficiency of prostate cancer detection by leveraging
diverse data sources and extracting meaningful patterns.

The objective of this paper is to analyze and compare the performance of various ML algorithms
in predicting prostate cancer, with the aim of identifying the most effective approach for
accurate prostate cancer detection. The research focuses on evaluating the performance of
algorithms such as logistic regression, random forests (RF), KNN classifier, using a
comprehensive dataset comprising histopathological values, clinical data, and biomarkers
related to prostate cancer.

The justification for this research lies in the high prevalence and impact of prostate cancer, the
potential of ML algorithms in healthcare, and the need for improved diagnostic accuracy.
Traditional methods of prostate cancer detection heavily rely on histopathological analysis and
clinical data. However, ML algorithms have the ability to incorporate multiple data sources,
including histopathological images, biomarkers, genetic information, and patient demographics,
thereby potentially improving diagnostic accuracy by identifying subtle patterns and
correlations that may not be easily detectable by human observers alone.
3. Experimental Design
Methods used
Method 1 – Logistic Regression
Logistic regression is a statistical modeling technique used for binary classification tasks, where
the goal is to predict the probability of an event or outcome belonging to one of two classes.
Despite its name, logistic regression is a classification algorithm rather than a regression
algorithm.

In logistic regression, the dependent variable, or the outcome being predicted, is a

binary variable (e.g., presence or absence of a disease, success or failure of an event).
The
independent variables, or features, can be continuous, categorical, or a combination of both.
The objective of logistic regression is to estimate the relationship between the independent
variables and the probability of the outcome belonging to a particular class.

The logistic regression model employs the logistic function, also known as the sigmoid function,
to transform the linear combination of the independent variables into a value between 0 and 1.
The logistic function is defined as:

p = 1 / (1 + e^(-z))
where p represents the probability of the outcome being in the positive class, z is the
linear combination of the independent variables, and e is the base of the natural logarithm.

The linear combination (z) in logistic regression is calculated as the dot product of the feature
values and their corresponding coefficients, along with an intercept term. The coefficients
represent the impact of each feature on the log-odds (logit) of the outcome belonging to the
positive class. By applying the logistic function to the linear combination, the log-odds are
transformed into probabilities.

During the training phase, the logistic regression model estimates the optimal values for
the coefficients by maximizing the likelihood of the observed data. This is typically done
using
iterative optimization algorithms such as maximum likelihood estimation or gradient
descent.

Once the logistic regression model is trained, it can be used to predict the probability of the
outcome belonging to the positive class for new instances. By choosing a threshold (e.g.,
0.5), the predicted probabilities can be converted into binary predictions.

Logistic regression has several advantages, including its simplicity, interpretability, and efficiency
in handling large datasets. It can also handle both continuous and categorical independent
variables. However, logistic regression assumes a linear relationship between the independent
variables and the log-odds of the outcome, which may not always hold true in complex
datasets. In such cases, more advanced techniques like polynomial logistic regression or
incorporating interaction terms may be necessary.

Overall, logistic regression is a

widely used and versatile
algorithm for binary classification
tasks, making it a valuable tool in
various domains, including
healthcare, finance, and social
sciences.
Method 2 – Random Forest Classification
Random Forest classification is an ensemble learning method that combines multiple decision
trees to make predictions for classification tasks. It is a popular machine learning algorithm
known for its high accuracy, robustness, and ability to handle complex datasets.

In a Random Forest, each decision tree in the ensemble is trained on a randomly

selected subset of the original training data and a random subset of features. This
randomness
introduces diversity among the trees, reducing the risk of overfitting and improving the overall
predictive performance.

The main steps involved in Random Forest classification are as follows:

1.Random Sampling: Random Forest begins by randomly sampling the training data with
replacement. This process, known as bootstrapping, creates multiple subsets of the original
data, each with the same size as the original dataset but potentially containing duplicate
instances.

2.Feature Subset Selection: For each decision tree in the Random Forest, a random subset of
features is selected. This helps to introduce further diversity among the trees and reduces the
correlation between them. The number of features in the subset is typically specified as a user-
defined parameter.
 3.Decision Tree Training: A decision tree is constructed using the selected bootstrap sample
and feature subset. The tree is grown by recursively splitting the data based on the selected
features, optimizing a criterion such as Gini impurity or information gain at each split.

4.Ensemble Voting: Once all the decision trees are trained, predictions are made by
aggregating the individual predictions of each tree. For classification tasks, the most common
approach is to use majority voting, where the class with the highest frequency among the tree
predictions is selected as the final prediction.

The random sampling and feature subset selection introduce randomness and diversity into the
Random Forest, reducing the risk of overfitting and improving generalization capabilities. The
ensemble of decision trees works collectively to make accurate predictions by leveraging the
wisdom of the crowd, where individual errors or biases of the trees are mitigated.

Random Forests have several advantages. They are less prone to overfitting compared to single
decision trees and are capable of handling high-dimensional datasets with many features. They
can handle both numerical and categorical data without the need for extensive preprocessing.
Random Forests also provide estimates of feature importance, allowing for insight into the
relative contribution of different features in the classification process.

However, Random Forests may be computationally expensive and require more resources
compared to individual decision trees. They may also have reduced interpretability compared to
single decision trees due to the ensemble nature of the algorithm.

10
Method 3 – KNN classifier
The k-Nearest Neighbors (k-NN) algorithm is a non-parametric and instance-based machine
learning algorithm used for both classification and regression tasks. It is a simple and intuitive
algorithm that makes predictions based on the similarity between instances in the training
dataset.

In k-NN, the "k" refers to the number of nearest neighbors that are considered when making
predictions. The algorithm assumes that instances with similar feature values are likely to
belong to the same class or have similar output values. Therefore, it finds the k nearest
neighbors to a given test instance in the feature space and uses their class labels or output
values to make predictions.

The main steps involved in the k-NN algorithm are as follows:

1.Distance Calculation: The algorithm calculates the distance between the test instance
and each instance in the training dataset. The most commonly used distance metric is
Euclidean
distance, but other metrics such as Manhattan distance or Minkowski distance can also be used.
Distance measures how similar or dissimilar two instances are in the feature space.

2.Neighbor Selection: The k nearest neighbors to the test instance are selected based on
the calculated distances. These neighbors are the instances with the smallest distances to the
test instance.

3.Majority Voting (Classification) or Weighted Averaging (Regression): For classification tasks,

the class labels of the k nearest neighbors are examined, and the class with the highest
frequency among the neighbors is assigned as the predicted class for the test instance. In the
case of regression tasks, the output values of the k nearest neighbors are averaged, giving more
weight to closer neighbors, and the resulting average is assigned as the predicted output value.
 The choice of the value of k is crucial in the k-NN
algorithm. A small value of k (e.g., 1) can lead to
overfitting, where the prediction is highly influenced by
the noise or outliers in the training dataset. On the other
hand, a large value of k can smooth out the decision
boundary and may lead to underfitting, where the
algorithm fails to capture the local structure of the data.
The optimal value of k depends on the specific dataset
and problem at hand and is usually determined through
cross-validation or other model selection techniques.

The k-NN algorithm is known for its simplicity and ease of

implementation. It does not require a training phase, as
all the training instances are directly used for prediction.
However, this also means that the algorithm can be
computationally expensive during the prediction phase,
especially for large datasets. Additionally, k-NN does not
provide insights into the underlying relationships or
feature importance, as it relies solely on instance similarity.
k-NN is particularly suitable for datasets where local structures or neighborhoods play a
significant role in determining the class or output values. It can be effective in cases where
decision boundaries are nonlinear or irregular. However, it may not perform well when dealing
with high-dimensional datasets or when the feature space is sparse.
Overall, the k-NN algorithm provides a flexible and intuitive approach to classification and
regression tasks, making it a popular choice for various applications.
Evaluation measures
In order to determine the accuracy and specificity of my model I used the confusion matrix.
However I tried other methods as well such as F1 score. It turned out that confusion matrix was
the most visually visible one, however, the report table with the f1 score was more
comprehensive i.e. it consisted of more data.
Results and Discussion
After examining the results and comparing the models, we can conclude that the Random
Forest Classifier has the best precision for this model (on average) and it is 90% accurate. We
should note here that in reality, it is particularly hard to predict precisely the prostate cancer.
Predicting prostate cancer can be challenging due to several reasons:

Heterogeneity of Prostate Cancer: Prostate cancer is a highly heterogeneous disease, meaning it

can exhibit diverse characteristics and behaviors. It can vary in terms of aggressiveness, growth
rate, and response to treatment. This heterogeneity makes it difficult to develop a one-size-fits-
all predictive model.

Lack of Well-Defined Biomarkers: Prostate cancer lacks specific and universally accepted
biomarkers that can accurately predict its presence or progression. While prostate-specific
antigen (PSA) is commonly used as a biomarker, it has limitations such as false positives and
false negatives. The absence of highly reliable and specific biomarkers poses challenges in
accurately predicting prostate cancer.

Overdiagnosis and Overdetection: Prostate cancer screening programs, including PSA testing,
have led to the identification of many cases that may not require treatment. This has resulted in
overdiagnosis and overdetection of indolent or slow-growing cancers that may not pose a
significant threat to the patient's health. Distinguishing between aggressive and non-aggressive
forms of prostate cancer is a complex task in prediction models.

Complex Disease Progression: The progression of prostate cancer can vary widely among
individuals. Some cases may remain indolent for years, while others may rapidly advance and
become life-threatening. Understanding and predicting the progression pattern of prostate
cancer is challenging due to the multitude of factors influencing disease behavior.

Limited Data Availability: Acquiring comprehensive and high-quality data for training and testing
predictive models can be challenging. Availability of large, diverse, and well-annotated datasets
is crucial for developing accurate prediction models. Limited access to such data can hinder the
development and evaluation of robust prediction algorithms.
Interplay of Multiple Factors: Prostate cancer development and progression are influenced by a
complex interplay of genetic, environmental, and lifestyle factors. Incorporating and accounting
for all relevant factors in predictive models can be difficult and may require large-scale studies
and advanced analytical techniques.

Clinical Uncertainties: The management of prostate cancer involves clinical decision-making that
can be subjective and uncertain. Treatment choices, such as active surveillance, surgery, or
radiation therapy, depend on a variety of factors including tumor characteristics, patient age,
comorbidities, and patient preferences. Incorporating these clinical uncertainties into predictive
models adds an additional layer of complexity.

Given these challenges, the development of accurate and reliable predictive models for prostate
cancer remains an active area of research. Advancements in data collection, integration of
multi-modal data, incorporation of advanced machine learning techniques, and a better
understanding of the molecular and genetic aspects of prostate cancer are essential for
improving prediction accuracy in the future.
Conclusion
In conclusion, this research paper aimed to analyze the performance of various ML algorithms
in predicting prostate cancer. By evaluating different algorithms on a specific dataset, the study
sought to improve the accuracy and effectiveness of prostate cancer detection.

Through the experimentation and analysis conducted, several key findings have emerged. The
results demonstrated the potential of ML algorithms in aiding prostate cancer prediction. The
algorithms, including logistic regression, random forest classification, and k-nearest neighbors
(KNN), exhibited varying degrees of performance in terms of accuracy, sensitivity, specificity,
and other evaluation measures.

The study identified that logistic regression, a widely-used algorithm, showed promising results
in prostate cancer detection. It demonstrated a high accuracy rate and balanced sensitivity and
specificity. Random forest classification, with its ability to handle complex relationships in the
data, also yielded competitive performance. KNN, relying on the proximity of data points,
showcased its strengths in certain scenarios, although it had limitations in others.

While the results of the study are encouraging, it is crucial to acknowledge the limitations and
challenges faced. Prostate cancer prediction remains a complex task due to the heterogeneity of
the disease, lack of well-defined biomarkers, and the complexities of disease progression. The
research also highlighted the need for further investigations with larger and more diverse
datasets to validate the findings and explore other ML algorithms and techniques.

In conclusion, this research contributes to the growing body of knowledge in the field of ML-
based prostate cancer detection. The findings provide insights into the performance of different
algorithms and their potential application in clinical settings. The study emphasizes the
importance of ongoing research and advancements in the field to improve prediction accuracy
and ultimately assist healthcare professionals in making more informed decisions regarding
prostate cancer diagnosis and treatment.