Machine Learning

UNIT-3

Dr Rashmi Popli
Associate Professor
Department of Computer Engineering

Dr Rashmi Popli
Evaluating Machine Learning
Algorithms
Evaluating machine learning algorithms and selecting the right model are crucial
steps in building effective and accurate predictive models.
1. Define Objectives and Metrics
• Clearly define the problem you are trying to solve and the goals of your model.
• Choose appropriate evaluation metrics based on the nature of the problem
(e.g., accuracy, precision, recall, F1-score, mean squared error, R-squared for
regression).
2. Data Preprocessing
• Handle missing data, outliers, and perform data normalization or
standardization as needed.
• Encode categorical variables and handle any other data-specific preprocessing
steps.
Dr Rashmi Popli
3. Split Data
• Split your dataset into training and testing sets to evaluate model
performance on unseen data.
• Optionally, use techniques like cross-validation for more robust evaluation.
4. Choose Algorithms
• Select a set of candidate algorithms based on the nature of the problem
(e.g., decision trees, support vector machines, neural networks).
• Consider the characteristics of your data (size, complexity) when choosing
algorithms.
5. Train Models
• Train each selected algorithm on the training data.
• Use hyperparameter tuning to find the optimal settings for each algorithm.

Dr Rashmi Popli
6. Evaluate Performance:
• Use the testing set to evaluate the performance of each model.
• Compare performance metrics and choose the model(s) with the best
results.
7. Consider Cross-Validation:
• Perform cross-validation to assess how well the models generalize to
different subsets of the data.
• This helps in identifying models that are less prone to overfitting.
8. Model Interpretability:
• Consider the interpretability of the models, especially if the application
requires understanding the reasoning behind predictions.
9. Ensemble Methods:
• Explore ensemble methods (e.g., random forests, gradient boosting) to
combine the strengths of multiple models for improved performance.
Dr Rashmi Popli
10. Deployment Considerations
• Consider practical aspects of deploying the model, such as computational
resources, real-time requirements, and interpretability.
11. Iterative Process
• Model evaluation and selection are iterative processes. If initial results
are not satisfactory, revisit previous steps, adjust hyperparameters, or try
different algorithms.
12. Documentation
• Document your choices, decisions, and results to facilitate
communication with stakeholders and for future reference.

Dr Rashmi Popli
Hyper parameter Tuning
• Hyper parameter tuning is the process of selecting the best set of
hyper parameters for a machine learning algorithm.
• Hyper parameters are configuration settings that are external to the
model itself and cannot be directly learned from the training data.
• Examples of hyperparameters include the the depth of a decision
tree, the number of hidden layers in a neural network, and the
regularization parameter in regression.
• Hyperparameter tuning aims to find the optimal combination of
hyperparameters that maximizes the performance of the model on a
validation dataset or through cross-validation.

Dr Rashmi Popli
Cross-validation
• Cross-validation is a technique used in machine learning to assess how
well a model will generalize to an independent dataset.
• It's particularly useful when the dataset is limited, as it allows
maximizing the amount of data used for both training and testing, thus
providing a more reliable estimate of the model's performance.
• Cross-validation helps to provide a more accurate estimate of a model's
performance compared to a single train-test split because it uses
multiple splits of the data.
• This helps to reduce the variability in performance estimates that may
arise from using a single train-test split, especially when the dataset is
small or the data distribution is heterogeneous.
Dr Rashmi Popli
Statistical learning Theory
• Statistics is the mathematical study of data.
• It is a field of study within machine learning that deals with the
theoretical foundations of learning algorithms.
• Statistical learning theory is a framework that draws from the fields
of statistics and functional analysis.
• Using statistics, an interpretable statistical model is created to describe
the data, and this model can then be used to infer something about
the data or even to predict values that are not present in the sample
data used to create the model.
• Objective: The main goal of statistical learning is understanding and
prediction. It aims to build models that can draw meaningful
conclusions from data and make accurate predictions.
Dr Rashmi Popli
Categories of Learning
• Supervised Learning: In this category, we have labeled data pairs
(input-output pairs), and the goal is to learn a mapping from inputs to
outputs.
• Unsupervised Learning: Here, we work with unlabeled data and aim to
discover underlying patterns or structures.
• Semi-supervised Learning: Combines aspects of both supervised and
unsupervised learning.
• Online Learning: Learning from data streams in real-time.
• Reinforcement Learning: Learning through interaction with an
environment.

Dr Rashmi Popli
Components of Statistical
Learning
• Features (Predictors): The variables used to make predictions.
• Response (Outcome): The variable to be predicted.
• Training Data: The dataset used to train the model.
• Model: The algorithm or mathematical function that learns from the
training data.

Dr Rashmi Popli
Applications
• Computer Vision: Developing algorithms for image recognition, object
detection, and scene understanding.

• Speech Recognition: Creating models that can transcribe spoken language

into text.

• Bioinformatics: Analyzing biological data, such as DNA sequences and protein

structures.

Dr Rashmi Popli
 Example: Predicting House Prices
1. Problem Statement: Imagine you are a real estate agent, and you want to predict the selling prices of
houses based on certain features like the number of bedrooms, square footage, and neighborhood.

2. Dataset: Collect a dataset that includes information about houses, such as:
• Number of bedrooms
• Square footage
• Neighborhood
• Distance to public amenities
• Previous sale prices (target variable)

3. Features and Target Variable:

• Features (X): Number of bedrooms, square footage, neighborhood, distance to public amenities.
• Target Variable (Y): Sale prices of houses.

4. Data Splitting: Divide the dataset into two parts:

• Training set: Used to train the model.
• Testing set: Used to evaluate the model's performance.
Dr Rashmi Popli
5. Model Selection: Choose a statistical learning algorithm. For this example, let's use a simple
linear regression model. The model will learn a linear relationship between the features and the
target variable.
6. Training the Model: Feed the training data into the linear regression algorithm. The algorithm
adjusts its parameters to minimize the difference between the predicted house prices and the
actual prices in the training set.
7. Model Evaluation: Use the testing set to assess the model's performance. Common metrics
include mean squared error or R-squared, which measure how well the model predicts house prices
on new, unseen data.
8. Prediction: Once the model is trained and evaluated, use it to predict house prices for new
houses or existing ones not included in the training set.
9. Model Improvement: If the model performance is not satisfactory, consider trying more complex
models, adding more features, or fine-tuning hyperparameters.
10. Deployment: Deploy the trained model in a real-world scenario to make predictions on new
data.
• statistical learning helps us automate the process of predicting house prices based on relevant
features, providing a valuable tool for real estate decision-making.

Dr Rashmi Popli
Ensemble methods
• They combine multiple models to improve predictive performance.
Here are three common ensemble methods:
• Bagging (Bootstrap Aggregating): Uses multiple base models (often
decision trees) trained on different subsets of the data and averages
their predictions.
• Random Forests: An extension of bagging that builds multiple
decision trees and combines their predictions.
• Boosting
: Iteratively builds weak models (e.g., decision trees) and combines th
em to create a strong model
1
Dr Rashmi Popli
Difference between Bagging and
Boosting
• Bagging: Training a bunch of individual models in a parallel way. Each
model is trained by a random subset of the data.

• Boosting: Training a bunch of individual models in a sequential way.

Each individual model learns from mistakes made by the previous
model.

Dr Rashmi Popli
Dr Rashmi Popli
Dr Rashmi Popli