BCS602 | MACHINE LEARNING

Module-1

Chapter – 01 - Introduction

Need for Machine Learning in Business:

Data Explosion and Integration Challenges:

o Businesses generate massive amounts of data daily.

o Earlier, the potential of this data was not utilized due to scattered data sources and
lack of integration tools.

Reasons for Popularity of Machine Learning:

o High Volume of Data:

 Companies like Facebook, Twitter, and YouTube generate enormous data,
which doubles approximately every year.
o Reduced Storage Costs:
 Declining hardware and storage costs make it easier to capture, store,
process, and distribute digital data.
o Availability of Complex Algorithms:
 The development of advanced algorithms, particularly deep learning, has
contributed to the growing popularity of machine learning.

Knowledge Pyramid:

o Data: Raw facts, numbers, or text that can be processed by computers.

o Information: Processed data revealing patterns or relationships, e.g., identifying
the best-selling product from sales data.
o Knowledge: Condensed and actionable information, such as historical trends or
future predictions from sales data.
o Intelligence: Applied knowledge for making decisions and taking actions.

CMRIT, ISE Page 1

 BCS602 | MACHINE LEARNING

o Wisdom: The ultimate objective, representing the maturity of human decision-

making, which machine learning helps organizations to approach.

Role of Machine Learning:

o Machine learning helps businesses process large amounts of data to make informed
decisions.
o It aids in designing new products, improving business processes, and developing
effective decision support systems.

Machine Learning Explained

Definition of Machine Learning:

CMRIT, ISE Page 2

 BCS602 | MACHINE LEARNING

 Arthur Samuel's Definition: "Machine learning gives computers the ability to learn
without being explicitly programmed."
 Focus: Systems learn autonomously by analyzing data without manual
programming.

Conventional Programming vs. Machine Learning:

Conventional Programming:

o Requires explicit instructions and logic.

o Involves detailed program design like flowcharts and algorithms.

Limitations:

o Ineffective for complex problems (e.g., image recognition, puzzles).

o Early AI relied on expert systems that converted expert knowledge into rules (e.g.,
MYCIN for medical diagnosis), but these lacked true intelligence.

Machine Learning Approach:

o Shifts from rule-based systems to data-driven models.

o Uses input data to develop models that can predict unknown data.

Concepts in Machine Learning:

Model: A mathematical or logical representation of patterns in data.

o Forms of models:
1. Mathematical equations.
2. Relational diagrams (e.g., trees, graphs).
3. Logical rules (if/else).
4. Clusters (groupings).

CMRIT, ISE Page 3

 BCS602 | MACHINE LEARNING

Pattern vs. Model:

o Pattern: Local, applies to certain attributes.

o Model: Global, fits the entire dataset.

Example: A model predicting whether an email is spam or not is derived automatically

from data.

Tom Mitchell's Definition of Machine Learning:

“A computer program learns from experience E, with respect to task T, and

performance measure P, if its performance on T measured by P improves with
experience E.”

Example:

o Task (T): Object detection in images.

o Experience (E): Training on thousands of labeled images.
o Performance (P): Metrics like precision and recall measure detection accuracy.

Steps in Machine Learning:

1. Data Collection: Gathering raw data from various sources.

2. Abstraction: Forming abstract concepts from data (e.g., recognizing an elephant’s
features).
3. Generalization:
o Converts abstractions into actionable intelligence.
o Involves ranking concepts, inferencing, and forming heuristics (rules of
thumb).
4. Evaluation:
o Tests the thoroughness of models.
o Identifies and corrects errors to improve system performance.

CMRIT, ISE Page 4

 BCS602 | MACHINE LEARNING

Machine Learning as Human Experience:

Computers mimic human experience through data.

Human Analogy:

o Humans learn by rote, observation, teachers, books, and trial-and-error.

o When faced with new problems, humans apply past experiences and heuristics.

Machine Analogy:

o Machines "learn" from data, generalize patterns, and form rules (heuristics) to solve
tasks.

Role of Data in Machine Learning:

Quality of Data: Directly affects the accuracy of the learning system.

Statistical Learning Function: Maps input (x) to output (y) as y=f(x).

Goal: Learn the function f from data to accurately predict outcomes for new inputs.

CMRIT, ISE Page 5

 BCS602 | MACHINE LEARNING

Limitations of Heuristics:

Heuristics are educated guesses and may fail occasionally.

Course corrections are made through evaluation and improved formulations.

Machine Learning in Relation to Other Fields

Machine Learning and Artificial Intelligence (AI):

Machine learning is a sub-branch of AI, which aims to develop intelligent agents (e.g.,
robots, humans, or autonomous systems).

AI originally focused on logic and reasoning but evolved into data-driven systems for
finding relationships in data.

Machine learning focuses on extracting patterns for prediction, while deep learning, a
sub-branch of machine learning, uses neural networks modeled on human neurons.

CMRIT, ISE Page 6

 BCS602 | MACHINE LEARNING

Machine Learning and Data Science:

Data science is an umbrella term encompassing multiple fields, including machine

learning.

Machine learning starts with data and focuses on analyzing patterns for prediction.

Some Key fields under data science include:

Big Data: Focuses on large datasets with the following characteristics:

 Volume: Huge amounts of data (e.g., from Facebook, YouTube).

 Variety: Data in different forms (e.g., images, videos).
 Velocity: Speed at which data is generated and processed.

Data Mining: Unearths hidden patterns in data (similar to machine learning but
focuses on pattern discovery).

Data Analytics: Extracts useful knowledge from raw data. Predictive data analytics is
closely related to machine learning.

CMRIT, ISE Page 7

 BCS602 | MACHINE LEARNING

Pattern Recognition:

An engineering field that uses machine learning algorithms for feature extraction,
pattern analysis, and classification.

Considered a specific application of machine learning.

Machine Learning and Statistics:

Statistics:

Focuses on hypotheses, experiments, and rigorous mathematical models to find

relationships in data.

Requires a strong theoretical background and often involves complex equations and
assumptions.

Machine Learning:

Less assumption-heavy compared to statistics and automates the learning process.

Often viewed as an evolved version of "old statistics."

Difference: Statistics aims to validate hypotheses, while machine learning focuses on

predictions and automation.

Some Key Takeaways:

Machine learning bridges AI, data science, statistics, and pattern recognition.

Deep learning, big data, and data analytics significantly influence its development.

Machine learning simplifies traditional statistical processes while enhancing predictive

power.

CMRIT, ISE Page 8

 BCS602 | MACHINE LEARNING

Types of Machine Learning

Machine learning involves learning through the interaction of a program with its
environment.

The types of learning are categorized based on the data used and the supervision
provided during the learning process.

Labelled and Unlabelled Data

Labelled Data:

CMRIT, ISE Page 9

 BCS602 | MACHINE LEARNING

Data with target attributes (labels) is used for supervised learning.

For example, the Iris dataset includes features (e.g., sepal length, petal width) with a
target label (class of Iris flower).

Unlabelled Data:

Data without target attributes is used for unsupervised learning.

The algorithm identifies patterns or groups without explicit supervision.

Supervised Learning

Involves a teacher or supervisor providing labelled data for training and testing.

Two Key Methods:

Classification:

Predicts discrete labels or classes.

CMRIT, ISE Page 10

 BCS602 | MACHINE LEARNING

Example: Classifying images as cats or dogs.

Algorithms:

 Decision Trees
 Random Forest
 Support Vector Machines (SVM)
 Naïve Bayes
 Artificial Neural Networks (e.g., CNNs for deep learning)

Classification models:

Generative Models: Focus on data generation (e.g., probabilistic models).

Discriminative Models: Focus on classification only.

CMRIT, ISE Page 11

 BCS602 | MACHINE LEARNING

Regression:

Predicts continuous variables, such as prices or sales.

Example: Predicting product sales over weeks using linear regression.

Algorithm:

Linear Regression: Fits a line (e.g., y=0.66x+0.54y = 0.66x + 0.54y=0.66x+0.54) to

predict outcomes based on independent variables.

Unsupervised Learning

Self-instruction-based learning with no supervisor or labelled data.

Algorithms observe examples, recognize patterns, and group similar objects.

CMRIT, ISE Page 12

 BCS602 | MACHINE LEARNING

Methods:

Cluster Analysis:

Groups objects into disjoint clusters based on attributes.

Example:

 Image segmentation to detect abnormal growth in medical images.

 Clustering gene signatures in a database.

Algorithms:

 k-means algorithm
 Hierarchical algorithms

Dimensionality Reduction:

 Simplifies data while retaining essential features.

CMRIT, ISE Page 13

 BCS602 | MACHINE LEARNING

Semi-Supervised Learning

Combines labelled and unlabelled data, leveraging small labelled datasets and larger
unlabelled datasets.

Useful when labelling data is expensive or time-consuming.

Differences Between Classification and Regression

Dimensionality Reduction

Dimensionality reduction is a type of unsupervised learning where high-dimensional

data is transformed into a lower-dimensional space while retaining the critical variance
or patterns in the dataset.

The goal is to simplify the dataset by reducing the number of features without losing its
essential information.

Differences between Supervised and Unsupervised Learning

Applications:

 Reducing computational complexity for machine learning models.

CMRIT, ISE Page 14

 BCS602 | MACHINE LEARNING

 Visualizing high-dimensional data in 2D or 3D (e.g., t-SNE for visualizing

clusters).
 Preprocessing data to remove noise or redundant features.

Examples of Dimensionality Reduction Algorithms:

Principal Component Analysis (PCA): Projects data into components that explain
the maximum variance.

Linear Discriminant Analysis (LDA): Finds the linear combinations of features that
best separate classes.

t-Distributed Stochastic Neighbor Embedding (t-SNE): Focuses on visualizing

high-dimensional data in 2D or 3D.

Autoencoders: Neural networks that compress and reconstruct data.

Semi-Supervised Learning

Semi-supervised learning operates in scenarios where a large portion of the dataset is

unlabelled, and only a small portion is labelled.

Since labelling data is expensive and time-consuming, semi-supervised algorithms make

use of unlabelled data by assigning pseudo-labels (predicted labels).

Process:

Unlabelled data is assigned pseudo-labels based on model predictions.

The pseudo-labelled data is combined with the labelled data to train the model.

Applications:

 Image recognition (with limited labelled samples).

CMRIT, ISE Page 15

 BCS602 | MACHINE LEARNING

 Natural language processing for text classification.

 Medical diagnosis where labelling data is costly.

Reinforcement Learning

Reinforcement learning is inspired by how humans learn through interaction and

feedback.

An agent interacts with the environment, perceives its state, takes actions, and receives
rewards or penalties.

The agent aims to maximize cumulative rewards by learning the best actions to take
over time.

Key Concepts:

Agent: The decision-maker (e.g., a robot, software, or a program).

Environment: The space where the agent interacts.
Actions: Choices available to the agent (e.g., move left, right, up, down in a grid
game).
Reward: Feedback signal for actions (positive or negative).

CMRIT, ISE Page 16

 BCS602 | MACHINE LEARNING

Policy: The strategy the agent uses to determine actions.

Goal: Achieve the highest cumulative reward over time.

Example - Grid Game:

In the grid game:

 Gray tiles represent danger.

 Black tiles are blocks.
 Diagonal-striped tile is the goal.
The agent learns to navigate from the bottom-left grid to the goal by exploring
various paths and gaining rewards for successful actions or penalties for poor
decisions.

Characteristics of Reinforcement Learning:

No labelled data: The agent learns through interaction instead of relying on labelled
datasets.

Sequential decision-making: The agent must take multiple steps to achieve the
goal.

Reward-driven learning: Positive rewards reinforce desirable behavior, while

negative rewards discourage it.

Challenges of Machine Learning

Machine learning, while powerful, faces several challenges that stem from its reliance
on data quality, computational resources, and algorithmic efficiency.

Below are the key challenges, explained in detail:

CMRIT, ISE Page 17

 BCS602 | MACHINE LEARNING

Problems with Ill-posed Problems

Machine learning performs well on well-posed problems, where specifications are

complete, clear, and well-defined.

However, ill-posed problems—where there is insufficient information or multiple valid

solutions—are a significant challenge.

o For instance, given a dataset where y = X1 X X2 , it’s equally plausible

that y = X1 ÷ X2 or y = X1 + X2 . Without additional data or constraints,
it’s impossible to determine the correct function.
o Solution: Ill-posed problems require more comprehensive data, better-
defined objectives, or external constraints to improve their solvability.

Quality of Data

Machine learning systems heavily rely on data quality.

Challenges with data include:

o Insufficient Data: Models require large datasets to learn effectively.

o Incomplete Data: Missing values can reduce model accuracy.
o Incorrect Data: Erroneous or noisy data introduces biases and errors.

CMRIT, ISE Page 18

 BCS602 | MACHINE LEARNING

o Imbalanced Data: Datasets where certain classes are underrepresented lead to

poor performance for those classes.

Solution: High-quality preprocessing, data augmentation, and imputation techniques

can address these issues.

High Computational Power Requirements

The growth of Big Data and complex machine learning tasks (e.g., deep learning)
demands significant computational resources:

Hardware Requirements: Systems need GPUs, TPUs, or distributed computing for

faster processing.

Time Complexity: Training sophisticated models can take hours, days, or even weeks.

Solution: Leveraging cloud computing platforms, distributed training, and optimized

frameworks (e.g., TensorFlow, PyTorch) can mitigate computational bottlenecks.

Algorithmic Complexity

The complexity of machine learning algorithms presents challenges in:

Selection: Identifying the best algorithm for a specific task.

Implementation: Understanding and applying advanced algorithms.

Optimization: Designing and fine-tuning algorithms to maximize efficiency and

accuracy.

Evaluation: Comparing algorithm performance to ensure optimal model selection.

Solution: Continuous research, education, and development of automated machine

learning (AutoML) tools simplify the design and selection process.

CMRIT, ISE Page 19

 BCS602 | MACHINE LEARNING

Bias/Variance Tradeoff

Bias and variance are sources of error in machine learning models:

Bias: Occurs when the model oversimplifies the problem, leading to underfitting.

Variance: Occurs when the model overfits the training data, failing to generalize to
new data.

Striking the right balance between bias and variance—called the bias-variance
tradeoff—is a persistent challenge.

Solution:

Use cross-validation techniques to assess model performance.

Regularization methods (e.g., L1, L2 regularization) can help reduce overfitting.

Ensemble methods like bagging (e.g., Random Forests) and boosting (e.g., XGBoost)
improve generalization.

Interpretability and Explainability

Many machine learning models, particularly deep learning, are considered black
boxes, making it difficult to explain their predictions or decisions.

Solution:

Use simpler, interpretable models where feasible.

Employ explainability tools like SHAP (SHapley Additive exPlanations) or LIME (Local
Interpretable Model-agnostic Explanations).

CMRIT, ISE Page 20

 BCS602 | MACHINE LEARNING

Ethical Concerns and Bias in Data

Biases present in training data can lead to biased models, perpetuating unfair decisions
(e.g., in hiring, lending, or law enforcement).

Solution: Implement fairness-aware machine learning techniques, conduct audits, and

use diverse datasets.

Scalability

Scaling machine learning models to work with massive datasets across distributed
systems is a technical challenge.

Solution: Use scalable tools and frameworks like Apache Spark, Dask, or cloud-based
machine learning services.

Machine Learning Process (CRISP-DM)

CMRIT, ISE Page 21

 BCS602 | MACHINE LEARNING

The CRISP-DM (Cross-Industry Standard Process for Data Mining) methodology

provides a structured framework for building data mining and machine learning
solutions.

It is widely adopted because of its versatility across industries. Below are the six steps
of the CRISP-DM process:

Understanding the Business

Objective: Grasp the business goals and define the problem statement.

Activities:

o Identify the organization's objectives and challenges.

o Formulate a clear problem statement for the machine learning solution.
o Choose a data mining or machine learning algorithm that aligns with the
business goals.

Understanding the Data

Objective: Explore and understand the data available for solving the problem.

Activities:

o Collect and analyze data from different sources.

o Study the data's characteristics, such as trends, anomalies, and distributions.
o Formulate hypotheses based on the data and match patterns to the
hypotheses.

Data Preparation

Objective: Clean and preprocess the raw data for analysis.

Activities:

CMRIT, ISE Page 22

 BCS602 | MACHINE LEARNING

o Handle missing values, duplicates, and irrelevant data.

o Normalize or scale the data as required.
o Prepare a finalized dataset suitable for the machine learning task.

Importance: Data quality directly impacts the performance of the machine learning
model.

For instance, missing values can degrade model accuracy.

Modeling

Objective: Apply machine learning algorithms to extract patterns or build models.

Activities:

o Select appropriate machine learning algorithms (e.g., classification, regression,

clustering).
o Train the model on the prepared dataset.
o Tune hyperparameters to optimize model performance.

Evaluation

Objective: Assess the performance of the model using statistical and visualization
techniques.

Activities:

o Measure accuracy, precision, recall, F1-score, and other metrics.

o Compare results with baseline models or domain benchmarks.
o Ensure the model aligns with business objectives.

Challenges: Tasks like email classification may require expert domain knowledge for
proper evaluation.

CMRIT, ISE Page 23

 BCS602 | MACHINE LEARNING

Deployment

Objective: Deploy the model to provide actionable insights or improve processes.

Activities:

o Integrate the model into the business workflow or system.

o Monitor its performance in real-world scenarios.
o Update the model as new data becomes available.

Applications of Machine Learning

Machine learning is transforming various industries and is increasingly present in

everyday life.

Below are some key applications:

Sentiment Analysis

Use Case: Natural Language Processing (NLP) techniques analyze text to determine
sentiments (e.g., happy, sad, angry).

Example: Movie and product reviews can be analyzed to assign ratings (e.g., stars)
automatically.

Recommendation Systems

Use Case: Personalizes user experiences by suggesting items based on preferences.

Example:

 Amazon: Recommends related books or products.

 Netflix: Suggests shows or movies based on viewing history.

CMRIT, ISE Page 24

 BCS602 | MACHINE LEARNING

Voice Assistants

Use Case: Use speech recognition and natural language understanding to perform
tasks.

Examples: Amazon Alexa, Apple Siri, Google Assistant, Microsoft Cortana.

Navigation Systems

Use Case: Optimize routes and reduce travel time using machine learning-based
algorithms.

Examples: Google Maps, Uber.

CMRIT, ISE Page 25

CMRIT, ISE
CMRIT, ISE
CMRIT, ISE
CMRIT, ISE
CMRIT, ISE
CMRIT, ISE
CMRIT, ISE
CMRIT, ISE
CMRIT, ISE
CMRIT, ISE
CMRIT, ISE
CMRIT, ISE
CMRIT, ISE
CMRIT, ISE
CMRIT, ISE
CMRIT, ISE
CMRIT, ISE