Student name:

Course: Machine learning

Group: E27-24
Date: 16.01.2025

FINAL EXAM

QUESTIONS

Q1 – Model selection: You need to classify medical images into different disease categories. Would
you choose a convolutional neural network (CNN) or a traditional machine learning algorithm, and
why?

Q2 – Cross-Validation: You are building a model for weather prediction using historical data. How
would you handle cross-validation in this case, considering that the data is time-dependent?

Q3 – Hyperparameter Tuning: You are training a neural network for handwritten digit recognition.
How would you use grid searchto tune hyperparameters such as the number of layers, learning rate,
and batch size?

Q4 – Evaluation metrics: You are building a recommendation system for a streaming service. Would
precision or recall be more important for your model and why?

Q5 – Regularization: You machine learning model for predicting product demand is gihly complex
and prone to overfitting. How would you apply L2 regularization to make your model more robust?
 ANSWERS

Q1 – Model selection: You need to classify medical images into different disease categories. Would
you choose a convolutional neural network (CNN) or a traditional machine learning algorithm, and
why?

For classifying medical images into different disease categories, Convolutional Neural Networks
(CNNs) are the preferred choice.

Feature Extraction from Images:

CNNs are specifically designed for image data. They automatically extract spatial features like edges,
textures, and patterns, which are crucial for medical image analysis.

Traditional machine learning algorithms require manual feature extraction, which is both time-
consuming and less effective for complex image data.

High Accuracy:

CNNs have consistently outperformed traditional machine learning algorithms in tasks involving
images, especially in fields like medical imaging (e.g., detecting tumors, identifying fractures, or
classifying X-rays).

Handling Complexity:

Medical images often contain subtle patterns that are difficult to discern. CNNs, with their deep
architectures, can capture these intricate details better than traditional methods.

Availability of Pretrained Models:

Pretrained CNNs (like ResNet, VGG, and Inception) are readily available and can be fine-tuned for
specific medical imaging tasks, reducing development time and improving performance.

Scalability:

CNNs scale well with large datasets, which are common in medical imaging. Traditional algorithms
may struggle with such kind of image dataset amount. So as a conclusion, CNNs are more useful in
classification of medical images into different disease categories.

Q2 – Cross-Validation: You are building a model for weather prediction using historical data. How
would you handle cross-validation in this case, considering that the data is time-dependent?
To handle cross-validation for time-dependent data like weather prediction in MATLAB, you should
use time-series-specific cross-validation techniques, such as time-based splitting (e.g., expanding or
rolling windows).

Conceptual Approach

Time-dependent data requires careful handling to avoid data leakage, where future information is
inadvertently used during training. Use the following techniques:

Expanding Window Cross-Validation:

Initial subset of data for training.

Adding more data to the training set while keeping the validation set in the future.

Rolling Window Cross-Validation:

Fixed-size window for training that "rolls forward" in time. The training and validation sets move
together.

Code for expanding window:

% Load historical weather data

% Assume `data` is an Nx2 matrix where the first column is time and the second is the target
variable.
data = load('weather_data.mat'); % Replace with actual data file
X = data.features; % Input features (e.g., temperature, humidity)
y = data.target; % Target variable (e.g., next day's temperature)

% Define initial training size and number of folds

initialTrainSize = 100; % Number of initial training samples
numFolds = 5;
N = size(X, 1); % Total number of samples

% Perform expanding window cross-validation

for fold = 1:numFolds
% Define training and validation indices
trainEnd = initialTrainSize + (fold - 1) * 20; % Expand training set
valStart = trainEnd + 1;
valEnd = min(valStart + 19, N); % Define validation set size

if valStart > N
break; % Stop if validation index exceeds data size
end

trainX = X(1:trainEnd, :);

trainY = y(1:trainEnd);
valX = X(valStart:valEnd, :);
valY = y(valStart:valEnd);

% Train model (example: linear regression)

mdl = fitlm(trainX, trainY); % Replace with your model

% Evaluate on validation set

predictions = predict(mdl, valX);
valError = mean((predictions - valY).^2); % Mean Squared Error
fprintf('Fold %d Validation Error: %.4f\n', fold, valError);
end

Code for rolling window:

% Define rolling window size

windowSize = 100; % Training window size
numFolds = 5;
N = size(X, 1);

% Perform rolling window cross-validation

for fold = 1:numFolds
% Define training and validation indices
trainStart = 1 + (fold - 1) * 20;
trainEnd = trainStart + windowSize - 1;
valStart = trainEnd + 1;
valEnd = min(valStart + 19, N);

if valEnd > N
break; % Stop if validation index exceeds data size
end

trainX = X(trainStart:trainEnd, :);

trainY = y(trainStart:trainEnd);
valX = X(valStart:valEnd, :);
valY = y(valStart:valEnd);

% Train model
mdl = fitlm(trainX, trainY); % Replace with your model

% Evaluate on validation set

predictions = predict(mdl, valX);
valError = mean((predictions - valY).^2); % Mean Squared Error
fprintf('Fold %d Validation Error: %.4f\n', fold, valError);
end

Q3 – Hyperparameter Tuning: You are training a neural network for handwritten digit recognition.
How would you use grid search to tune hyperparameters such as the number of layers, learning rate,
and batch size?

Number of Layers: [2, 3, 4]

Learning Rate: [0.001, 0.01, 0.1]
Batch Size: [32, 64, 128]

This will result in a grid of all possible combinations of these parameters.

(e.g MNIST as dataset)

% Load MNIST dataset

[XTrain, YTrain, XTest, YTest] = digitTrain4DArrayData; % Example dataset in MATLAB

% Normalize the data (scale pixel values to [0,1])

XTrain = XTrain / 255;
XTest = XTest / 255;

Set Up the Neural Network and Grid Search

Create a loop to iterate over the grid of hyperparameters, train the network, and evaluate its
performance.

Code for grid search:

% Define hyperparameter grid

numLayersList = [2, 3, 4];
learningRateList = [0.001, 0.01, 0.1];
batchSizeList = [32, 64, 128];

bestAccuracy = 0; % To track the best model performance

bestParams = []; % To store the best hyperparameter set

% Loop through all hyperparameter combinations

for numLayers = numLayersList
for learningRate = learningRateList
for batchSize = batchSizeList

% Define network architecture

layers = [
imageInputLayer([28 28 1], 'Normalization', 'none') % Input layer
];
for l = 1:numLayers
layers = [layers
fullyConnectedLayer(100) % Example size
reluLayer
];
end
layers = [layers
fullyConnectedLayer(10) % Output layer (10 classes for digits 0-9)
softmaxLayer
classificationLayer
];
 % Define training options
options = trainingOptions('sgdm', ... % Stochastic gradient descent
'InitialLearnRate', learningRate, ...
'MiniBatchSize', batchSize, ...
'MaxEpochs', 5, ...
'Verbose', false, ...
'Plots', 'none');

% Train the network

net = trainNetwork(XTrain, YTrain, layers, options);

% Evaluate on test data

YPred = classify(net, XTest);
accuracy = mean(YPred == YTest); % Accuracy

% Track the best model

fprintf('Layers: %d, Learning Rate: %.4f, Batch Size: %d, Accuracy: %.4f\n', ...
numLayers, learningRate, batchSize, accuracy);

if accuracy > bestAccuracy

bestAccuracy = accuracy;
bestParams = [numLayers, learningRate, batchSize];
end
end
end
end

% Display the best hyperparameters and accuracy

fprintf('Best Hyperparameters: Layers=%d, Learning Rate=%.4f, Batch Size=%d\n', ...
bestParams(1), bestParams(2), bestParams(3));
fprintf('Best Accuracy: %.4f\n', bestAccuracy);

Explanation:

Grid Definition:

numLayersList, learningRateList, and batchSizeList define the range of hyperparameters.

The code iterates over all combinations of these values.
Dynamic Network Creation:

The network architecture is dynamically built based on the number of layers (numLayers).
Training:

The trainNetwork function trains the neural network for each combination of hyperparameters.
Evaluation:

Accuracy is calculated on the test set, and the best hyperparameter combination is recorded.
Q4 – Evaluation metrics: You are building a recommendation system for a streaming service. Would
precision or recall be more important for your model and why?

The importance of precision versus recall in a recommendation system for a streaming service
depends on the specific goals and user experience priorities of the system. Here's an analysis:

1. Precision
Definition: Precision measures the proportion of recommended items that are relevant to the user.

When Precision Matters:

If users are overwhelmed by irrelevant recommendations, precision is more important. High

precision ensures that the recommendations shown to users are mostly relevant, enhancing the
perceived quality of the system.
Example: A streaming service might prioritize precision to avoid recommending completely
irrelevant genres or content (e.g., recommending horror movies to a user who primarily watches
romantic comedies).
Impact: A high-precision model reduces the chances of frustrating the user with poor suggestions,
making the experience more tailored and enjoyable.

2. Recall

Definition: Recall measures the proportion of relevant items that are successfully recommended to
the user.

When Recall Matters:

If the priority is to ensure users discover as many relevant items as possible, recall becomes more
important. This is especially useful when the catalog is vast, and missing relevant content could lead
to dissatisfaction or churn.
Example: A streaming service might prioritize recall to help users discover niche or lesser-known
content they might enjoy, increasing engagement and watch time.
Impact: A high-recall model ensures users are exposed to a broader range of relevant content,
increasing the likelihood of long-term satisfaction and retention.

Which to Prioritize?
Prioritize Precision:

If the streaming service wants to provide a concise list of highly relevant recommendations.
When user trust is critical: irrelevant suggestions can reduce confidence in the recommendation
system.

Prioritize Recall:

If the goal is to maximize user engagement and discovery, even at the cost of showing a few
irrelevant items.
For exploratory users who prefer variety and are open to diverse suggestions.
Balanced Approach: F1-Score
In practice, a balance between precision and recall is often ideal, as both metrics have trade-offs.

The F1-score, the harmonic mean of precision and recall, can help optimize both:

Conclusion
For a streaming service, precision is typically more important to ensure users trust the system and
enjoy the recommendations.

Q5 – Regularization: You machine learning model for predicting product demand is highly complex
and prone to overfitting. How would you apply L2 regularization to make your model more robust?

L2 regularization adds a penalty proportional to the squared magnitude of the model's weights to
the loss function:

𝑤𝑖 - Weights of the model.

𝜆 - Regularization parameter, controlling the strength of regularization. The addition of this penalty
term reduces the tendency of the model to fit noise in the data, improving generalization.

Implementation in Different Contexts

A. For Linear Models

If you're using linear regression or logistic regression:

Many libraries have built-in support for L2 regularization.

Code:

% Load data

X = rand(100, 10); % Features

y = rand(100, 1); % Target variable

% Ridge regression with L2 regularization

lambda = 0.1; % Regularization strength

mdl = fitrlinear(X, y, 'Learner', 'leastsquares', 'Lambda', lambda);

Code for neural networks:

layers = [
sequenceInputLayer(numFeatures)
fullyConnectedLayer(64, 'L2Regularization', 0.01) % Apply L2 regularization
reluLayer
fullyConnectedLayer(1)
regressionLayer
];

% Define training options

options = trainingOptions('adam', 'MaxEpochs', 20);

% Train the network

net = trainNetwork(XTrain, YTrain, layers, options);

. Tuning the Regularization Parameter (𝜆)

λ (or alpha) controls the strength of regularization:

Small values: Minimal regularization, can still lead to overfitting.

Steps for tuning 𝜆

Large values: Strong regularization, might underfit the data.

Start with a small value (e.g., 𝜆=0.01)

Use grid search or cross-validation to test a range of 𝜆 values.
Select the λ that minimizes validation error.

When to Use L2 Regularization

When the model has many features or parameters, leading to high complexity.
When training accuracy is high, but test accuracy is much lower (indicating overfitting).
When you need to improve the model’s robustness to noise in the training data.