30/01/2025

Chapter 1 - Introduction to
Machine Learning

Course Title: Directed Studies in BME

Instructor: Dr Mujeeb Rahman
Biomedical Engineering Dept.

Machine Learning Vs Artificial Intelligence

Artificial intelligence (AI) Machine learning (ML)
• The broad field of creating • A subset of AI that specifically focuses on
machines or systems that can how machines can learn, adapt, and make
perform tasks requiring human- decisions, often using techniques like
like intelligence, including machine learning (ML), deep learning, and
learning, reasoning, problem- neural networks.
solving, and perception. • Focuses on learning from data, improving
• It include rule-based systems over time, and making autonomous
that follow pre-programmed decisions.
logic without adapting. • Primarily used in areas involving learning
• Includes robotics, expert and adaptation.
systems, natural language
processing (NLP), computer
vision, etc.

1
 30/01/2025

What is Machine Leaning?

Human Learning

2
 30/01/2025

Let’s ask same question to Tarzan

He is a human being, Indeed, he can observe and learn

Let’s make him learn systematically

3
 30/01/2025

What about machine (Computer)?

Other tasks

4
 30/01/2025

What do we do?
• Just like what we did to Tarzan
• We need to provide experience to machine
• First, we give enough data to the machine
• This is called training data

5
 30/01/2025

Develop Algorithms to process data, extract patterns

or relations in the data and identify applicable rules

What is ML?
• ML is a field of artificial intelligence where computers learn to
perform tasks without being explicitly programmed, relying on
patterns and inference instead.

6
 30/01/2025

Traditional Programming Vs Machine Learning

• TP involves writing explicit • ML relies on training algorithms
instructions for a computer to • Learn patterns from data
follow • Make predictions or decisions
• TP is rule-based (if … else) without explicit programming.
• TP is task-specific • ML data-driven
• Incapable of generalizing to new • ML adaptable
situations. • ML is capable of generalizing to
new situations.

ML Paradigms
Depends on the nature of the problems or
type of the dataset, ML methods are
classified in to:
1. Supervised learning
2. Unsupervised learning
3. Reinforcement learning

7
 30/01/2025

What is Supervised learning?

Supervised learning is a type of machine learning where an algorithm is trained on a labeled

dataset, meaning that each input in the training data is associated with a corresponding
output. The goal of supervised learning is for the algorithm to learn a mapping from inputs to
outputs so that, given new, unseen inputs, it can predict or classify the correct output.

Supervised Learning

8
 30/01/2025

Unsupervised ML
• Unsupervised learning is a type of machine learning where the
algorithm is given unlabeled data and must find patterns,
relationships, or structures within the data without explicit guidance
on the correct output.
• The goal of unsupervised learning is to explore the inherent structure
of the data, discover hidden patterns, or group similar data points.

Unsupervised Learning

9
 30/01/2025

Clustering

Clustering –
Grouping similar
items together

Ground truth (labels) not

available

Reinforcement Learning

10
 30/01/2025

Reinforcement Learning (Continue)

Child learn from Trial and error – Punishment Vs Rewards

Reinforcement Learning (RL) is a type of machine learning paradigm where an agent learns to make decisions by
interacting with an environment. The agent receives feedback in the form of rewards or penalties based on the actions it
takes in the environment. The goal of reinforcement learning is for the agent to learn a strategy or policy that maximizes
the cumulative reward over time.

11
 30/01/2025

Types of ML Algorithms

Discrete Labels or Target Continuous Target No Labels

12
 30/01/2025

Regression models
• If the labels are continuous
• These model can predict output (continuous data) from the
previously learned experience

Regression is a type of statistical modeling technique used in ML to analyze the relationships between dependent and
independent variables. The goal of regression is to predict a continuous numerical output based on one or more input
features.

Types of ML – Based on Algorithms

Logistic Regression (LR) Artificial Neural Network Convolutional Neural Networks

Random Forest (RF) based models – Requires (CNN), Recurrent Neural Networks
Support Vector Machine features from the input data (RNN) and Long short term memory
(SVM) (LSTM), etc. (capable of extracting
K – Nearest Neigbours (KNN), features from input data)
etc.

13
 30/01/2025

Examples for Supervised Learning

• Supervised Learning (Labeled Data → Predict Output)
Example 1: Email Spam Detection:
Given a dataset of emails labeled as "spam" or "not spam", the model learns to
classify future emails correctly.
• Input: Email content
• Output: Spam or Not Spam

Example 2: Disease Diagnosis

A machine learning model is trained on medical records labeled with diseases.
Input: Patient symptoms & test results
Output: Predicted disease (Tuberculosis, Breast cancer, Retinopathy, Heart attack)

Examples for Unsupervised Learning

Unsupervised Learning (No Labels → Find Pa erns)

Example 1: Customer Segmentation in Marketing

Businesses analyze customer purchase history without predefined categories to
group similar customers.
Input: Purchase behavior data
Output: Clusters of similar customers (e.g., budget buyers vs. luxury shoppers)

Example 2: Anomaly Detection in Banking

Banks use unsupervised learning to detect fraudulent transactions based on unusual
spending patterns.
Input: Transaction history
Output: Anomalous (potential fraud) or normal transactions

14
 30/01/2025

Examples for RL
• Reinforcement Learning (Trial & Error → Learn from Rewards)
Example 1: Self-Driving Cars
A car learns to drive by interacting with the environment, receiving rewards for
safe driving and penalties for accidents.
Action: Turn, accelerate, brake
Reward: Safe driving (positive) / Collision (negative)

Example 2: Game Playing (Chess, Go, Video Games)

AI like AlphaGo learns to play games by playing millions of matches, improving
through wins and losses.
Action: Move pieces
Reward: Winning the game

ML in Mechanical Engineering
• ML is increasingly being integrated into Mechanical Engineering to optimize
processes, improve efficiency, and enhance predictive capabilities.
Example 1. Predictive Maintenance
ML models analyze sensor data (vibration, temperature, pressure, etc.) to predict
equipment failures before they occur.
Helps in reducing downtime and maintenance costs.
Used in industries like aerospace, automotive, and manufacturing.

Example 2. Design Optimization

Generative Design: AI algorithms generate optimized mechanical designs based on
constraints (e.g., material, weight, stress).T
Topology Optimization: ML assists in optimizing component shapes to reduce weight
while maintaining strength.

15
 30/01/2025

Continue..
Example 3: Quality Control & Defect Detection
• ML-powered computer vision inspects manufacturing defects using high-
resolution images.
• Reduces human errors in quality inspection.
• Used in automotive, aerospace, and additive manufacturing.
Example 4: Smart Manufacturing:
ML improves factory automation through digital twin and real-time process
optimization.
Used in CNC machining, welding, and robotic manufacturing.

Types of ML Classifiers (Based on output class)

1. Binary class:
a) Normal or Abnormal
b) Diabetic or Non diabetic

2. Multiclass:
Car make: Toyota, Range rover, FJ Cruiser, BMW, Benz, Honda
Fruit type: Mango, Orange, Apple, Pineapple

16
 30/01/2025

Binary Vs Multiclass Classifier

First ML Model from Scratch

• Let’s consider an example teaching a machine to recognize vehicle
types.
• Given a dataset with cars and bikes
• We should train the model first using large dataset that contains wide
range of cars and bikes
• A successfully trained model can make predictions for any given
unseen vehicle

17
 30/01/2025

Steps involved
1. First task is to find features that can represents objects.

Formation of Feature set

18
 30/01/2025

Let’s consider a single feature

19
 30/01/2025

Given the #wheel, can the model predict the

Prediction based on probability
vehicle?

Probability (Vehicle type | # wheel)?

Probability (Car | # wheel = 4)  100%

Probability (Bike| # wheel = 4)  0%

Probability (Car | # wheel = 2)  0%

Probability (Bike | # wheel = 2)  100%

An alternate method (Threshold)

20
 30/01/2025

A slightly different dataset

21
30/01/2025

22
 30/01/2025

To summarize
1. Identify the features
2. Represent the vehicles by the features
3. Remove non-informative features
4. Build classification model from the features (training the model)
5. Perform classification of the unknown data (testing the model)
6. Optimization of the model scores

23
 30/01/2025

Exploratory Data Analysis (EDA)

• EDA is a vital step in the process of machine learning (ML) and data analysis.
• It involves the exploration and analysis of data sets to understand their main
characteristics to decide the subsequent steps in ML.
Some of main purposes:
1. Understand the Data:
• Examine the structure and format of the data.
• Check for missing values, duplicates, and any data quality issues.
• Explore the distribution of variables.
2. Descriptive Statistics:
• Calculate summary statistics such as mean, median, standard deviation, and
percentiles.
• Understand the central tendency and spread of the data.

EDA continue..
3. Data Visualization:
• Use visualizations like histograms, box plots, scatter plots, and correlation matrices to
explore relationships and distributions.
4. Feature Analysis:
• Analyze the characteristics and properties of individual features.
• Assess the impact of each feature on the target variable.
5. Correlation Analysis:
• Investigate relationships between variables using correlation matrices
6. Outlier Detection:
• Identify and handle outliers that might negatively impact model performance.
• Understand whether outliers are errors or genuine data points.

24
 30/01/2025

EDA cont.

7. Data Transformation:
• Explore the need for data transformations, such as normalization or scaling, based on
the distribution of the data.
8. Missing Data Handling:
• Understand the extent of missing data and devise strategies for handling it
(imputation, removal, etc.).
9. Target Variable Analysis:
• For supervised learning tasks, analyze the distribution of the target variable.
• Identify potential class imbalances or regression challenges.

What are outliers in datasets

• Outliers are data points that significantly differ
from the majority of the data in a dataset.
• These points are often distant from other
observations and can have a substantial impact
on ML models data interpretation.
• Outliers can occur for various reasons, including
measurement errors, natural variability in the
data, or sometimes as a result of genuine, but
unusual, phenomena.
• Depends in the dataset, different methods are
available to handle missing values and outliers in
datasets

25
 30/01/2025

Feature Engineering
• Feature engineering is the process of transforming raw data into a format that is
better suited for machine learning models.

• It involves creating new features, selecting relevant features, and transforming

existing features to enhance the performance of a ML algorithm.

• Feature engineering plays a crucial role in improving the accuracy and efficiency of
models by providing them with more meaningful and relevant input data.

Various feature engineering techniques

1. Feature Creation:
• Generate new features based on existing ones.
2. Dimensionality Reduction:
• Reduce the number of features by selecting the most important ones.
• Use techniques like Principal Component Analysis (PCA) or feature selection algorithms.
3. Normalization and Scaling:
• Ensure that features are on a similar scale to prevent certain features from dominating
others (Min-Max scaling or Z-score normalization).
4. Handling Missing Values:
• Impute or fill in missing values using appropriate strategies.
• Consider creating additional features indicating the presence or absence of missing
values.
5. Encoding Categorical Variables:
• Convert categorical variables into numerical representations - one-hot encoding, label encoding,
or target encoding.

26
 30/01/2025

Data splitting – Training, Validation and Test

Splitting a dataset into training and test sets is a fundamental step in the
development of machine learning models.
The purpose of this split is to assess the model's performance on unseen data.
The two main subsets are:

1. Training Set:
• The training set is used to train the ML model. The model learns the patterns
and relationships within the data using this subset.
• The training set comprises a significant portion of the available data, typically
around 70-80% of the entire dataset, depending on the size and
characteristics of the data.

Data splitting – Training, Validation and Test

(Cont.)
2. Test Set:
The test set is reserved for evaluating the model's performance after it has
been trained.
It consists of data that the model has not seen during training.
The test set provides an unbiased evaluation of the model's ability to
generalize to new, unseen data.
The test set usually constitutes the remaining 20-30% of the dataset.

27
 30/01/2025

Validation set
• In addition to the training and test sets, ML researchers often use a third
dataset called the validation set.
• The validation set is used to fine-tune model hyperparameters and to
assess the model's performance during intermediate stages of training.
• Typical amount: 10% of dataset
• Training: 70%
• Validation: 10%
• Testing: 20%

Underfitting
• This Underfitting occurs when a model is too simple to capture the
underlying patterns in the training data.
• It fails to capture complex relationships in the data.
• Training performance is poor, and the model may perform poorly on
both the training and test sets.
Remedies:
• Increase model complexity.
• add more features.
• use a more advanced algorithm.

28
 30/01/2025

Overfitting:
• Overfitting occurs when a model is too complex and fits the training data
too closely, capturing noise and outliers.
• The model has low bias but high variance.
• It performs very well on the training set but poorly on new, unseen data.
• There is a risk of memorizing noise rather than learning genuine patterns.
Remedies:
1. Use simpler models.
2. Regularize the model (e.g., add regularization terms).
3. Increase the amount of training data.
4. Use feature engineering to reduce dimensionality.

Best Fit (Balanced Model):

Achieving the best fit involves finding a balance between model simplicity and
complexity, such that the model generalizes well to new data.
• The model captures underlying patterns without being too simple or too
complex.
• It performs well on both the training and test sets.
• The bias and variance are balanced.
• Fine-tune hyperparameters based on validation performance.
• Regularize the model appropriately.
• Use cross-validation to assess generalization performance.

29
 30/01/2025

Fittings of Models

Data representation in ML/Data science

1. Continuous Data:
• Continuous data refers to a type of numerical data that can take any real value
within a given range (can have infinite number of possible values).
• Examples of continuous data include:
• Temperature: It can be any real number within a certain range.
• Height: The height of individuals can take any real value within a given range.
• Weight: Similar to height, weight is a continuous variable.
• Continuous data is typically analyzed using statistical methods.
• Regression analysis, correlation analysis, and various machine learning
algorithms often work well with continuous data.

30
 30/01/2025

Data representation (Continue)

2. Discrete data
e.g. Number of Employees, Number of Products Sold - It involves counting
individual units.
3. Categorical data - Features like gender, color, or product categories are also
examples of discrete data.
• In ML, discrete data is often encountered in classification problems where the
goal is to assign an input to one of several predefined categories or classes.
• Not all the algorithms handles discrete data, but decision trees, random
forests, and Naive Bayes classifiers are commonly used for handling discrete
data.
• These data can be nominal or ordinal

Nominal Data
• Nominal data consists of categories with no inherent order or ranking.
• The categories are distinct and represent different groups without any
implied hierarchy.
• Examples:
• Colors (e.g., red, blue, green)
• Types of fruit (e.g., apple, banana, orange)
• Gender (e.g., male, female, non-binary)
• In nominal data, you can't say that one category is "greater" or "higher" than
another; they are simply different.

31
 30/01/2025

Ordinal Data
• Ordinal data also consists of categories, but these categories have a
meaningful order or ranking.
• The intervals between categories are not necessarily uniform or well-defined.
• Examples:
• Educational levels (e.g., high school, bachelor's, master's)
• Customer satisfaction ratings (e.g., low, medium, high)
• Socioeconomic status (e.g., low income, middle income, high income)
• In ordinal data, the order matters, but the precise differences between the
categories may not be known or consistent.
For example, the difference in satisfaction between "low" and "medium" may
not be the same as between "medium" and "high."

Time Series Data

• Time series data represents data points collected over successive and
evenly spaced intervals of time.
• It is commonly used in forecasting and analysis of trends over time.
Examples: Continuously monitored vital signs (body temp, ECG), vibration
of machineries, fluid flow/level flow measurements over specific period
of time.

32
 30/01/2025

Text data
• Text data is a prevalent and valuable form of data in ML, and its analysis
falls under the domain of Natural Language Processing (NLP).
• In NLP, the goal is to enable computers to understand, interpret, and
generate human language. Here are some key aspects and techniques
related to handling text data in machine learning:
• Text Preprocessing:
• Tokenization: Breaking down the text into individual words or tokens.
• Lowercasing: Converting all text to lowercase to ensure consistency.
• Removing Stop words: Eliminating common words (e.g., "the," "and") that don't
carry significant meaning.
• Language models

Scaling and normalization

• These are preprocessing techniques used in ML to ensure that numerical
features are on a similar scale, which can be important for the performance
of certain algorithms.
• These techniques help prevent features with larger magnitudes from
dominating the learning process, particularly in algorithms that are
sensitive to the scale of input features.
• Two common methods for scaling and normalization are Min-Max Scaling
and Standardization (Z-score normalization).

33
 30/01/2025

Min Max Scaling

•𝑋 =
• Scales the data to a specific range, commonly between 0 and 1.
• Xmin is the minimum value of the feature.
• Xmax is the maximum value of the feature.
Advantages:
• Simple and easy to understand
• Preserves the shape of the original distribution.
Disadvantage:
• Sensitive to outliers, as they can disproportionately affect the scaling.

Example

• For a dataset, the min and max observable values as 30 and -10.
Apply min-max normalization for a data point 18.8.
y = (X— Xmin) / (Xmax-Xmin)
y = (18.8 — (-10)) / (30 — (-10))
y = 28.8 / 40
y = 0.72

34
 30/01/2025

Data before and after Min-Max

Z-score normalization
• Also known as standardization, is a preprocessing technique in which
the numerical values of a variable are transformed to have a mean of
0 and a standard deviation of 1.
• This is achieved by subtracting the mean of the variable from each
data point and dividing the result by the standard deviation.
• The formula for Z-score normalization is given by:

35
 30/01/2025

Z – score scaling

Min-Max scaling Vs Z score

• Range: • Range:
• Scales the data between 0 and 1. • Output values can take any real value.
• Scale Independence: • Scale Independence:
• Suitable for algorithms that expect input • When the algorithm is sensitive to the scale of
features in a specific range. features.
• Sensitivity to Outliers: • Sensitivity to Outliers:
• Sensitive to outliers, as extreme values can • Less sensitive to outliers, as the effect of extreme
values is mitigated by the standardization process.
disproportionately affect the scaling.
• Interpretability:
• Interpretability:
• Alters the interpretability of the original data, as
• Maintains the interpretability of the original values are transformed based on their relationship
data, as values are scaled proportionally. to the mean and standard deviation.
• Use Cases: • Use Cases:
• Well-suited for algorithms that require • Commonly used in statistical analyses, ML like k-
features to be on a bounded scale (SVM, NN) means clustering.

36
 30/01/2025

Handling of Categorical data

• Dealing with categorical data in machine learning is
essential, as many machine learning algorithms
require numerical input.
• Depending on the nature of the data and the specific
requirements of the machine learning algorithm,
there are different methods:
1. Label Encoding:
• Convert categorical labels into numerical
representations.
• Assign a unique integer to each category.
• Useful for ordinal categorical data where the order
matters.

2. One-Hot Encoding:
• Create binary columns for each category.
• Assign a value of 1 to the column corresponding to the category and 0
otherwise.
• Suitable for nominal categorical data where there is no inherent order.

37
 30/01/2025

Measure Performance of an ML model

• A confusion matrix is a table that summarizes the performance of a
classification algorithm on a set of data for which the true values are known.
• The confusion matrix provides a detailed breakdown of correct and incorrect
predictions made by a classification model.
True Positive (TP) - The predicted value (POS) matches the actual (POS)
True Negative (TN) - The predicted value (NEG) matches the actual value
(NEG)
False Positive (FP) – The actual value was negative, but the model predicted a
positive value, falsely predicted as positive.
False Negative (FN) – The actual value was positive, but the model predicted
a negative value, falsely predicted as negative.

38
 30/01/2025

Metrics used to assess model performance

39