RV Educational Institutions®

RV Institute of Technology and Management

(Affiliated to VTU, Belagavi)
JP Nagar 8th Phase, Bengaluru - 560076

Department of Computer Science and Engineering

Course Name: Machine Learning

Course Code: BCS602

VI Semester
2022 Scheme
 RV Institute of Technology and Management®

_________________________________________________________________________________

Module-1

Chapter – 01 - Introduction

Need for Machine Learning in Business:

Data Explosion and Integration Challenges:

o Businesses generate massive amounts of data daily.

o Earlier, the potential of this data was not utilized due to scattered data sources and lack of
integration tools.

Reasons for Popularity of Machine Learning:

o High Volume of Data:

 Companies like Facebook, Twitter, and YouTube generate enormous data, which
doubles approximately every year.
o Reduced Storage Costs:
 Declining hardware and storage costs make it easier to capture, store,
process, and distribute digital data.
o Availability of Complex Algorithms:
 The development of advanced algorithms, particularly deep learning, has
contributed to the growing popularity of machine learning.

Knowledge Pyramid:

o Data: Raw facts, numbers, or text that can be processed by computers.

o Information: Processed data revealing patterns or relationships, e.g., identifying the
best-selling product from sales data.
o Knowledge: Condensed and actionable information, such as historical trends or future
predictions from sales data.
o Intelligence: Applied knowledge for making decisions and taking actions.

Machine Learning- BCS602

Page 2
 RV Institute of Technology and Management®

_________________________________________________________________________________

o Wisdom: The ultimate objective, representing the maturity of human decision-

making, which machine learning helps organizations to approach.

Role of Machine Learning:

o Machine learning helps businesses process large amounts of data to make informed
decisions.
o It aids in designing new products, improving business processes, and developing
effective decision support systems.

Machine Learning Explained

Definition of Machine Learning:

Machine Learning- BCS602

Page 3
 RV Institute of Technology and Management®

_________________________________________________________________________________

 Arthur Samuel's Definition: "Machine learning gives computers the ability to learn
without being explicitly programmed."
 Focus: Systems learn autonomously by analyzing data without manual
programming.

Conventional Programming vs. Machine Learning:

Conventional Programming:

o Requires explicit instructions and logic.

o Involves detailed program design like flowcharts and algorithms.

Limitations:

o Ineffective for complex problems (e.g., image recognition, puzzles).

o Early AI relied on expert systems that converted expert knowledge into rules (e.g.,
MYCIN for medical diagnosis), but these lacked true intelligence.

Machine Learning Approach:

o Shifts from rule-based systems to data-driven models.

o Uses input data to develop models that can predict unknown data.

Concepts in Machine Learning:

Model: A mathematical or logical representation of patterns in data.

o Forms of models:
1. Mathematical equations.
2. Relational diagrams (e.g., trees, graphs).
3. Logical rules (if/else).
4. Clusters (groupings).

Machine Learning- BCS602

Page 4
 RV Institute of Technology and Management®

_________________________________________________________________________________

Pattern vs. Model:

o Pattern: Local, applies to certain attributes.

o Model: Global, fits the entire dataset.

Example: A model predicting whether an email is spam or not is derived automatically from
data.

Tom Mitchell's Definition of Machine Learning:

“A computer program learns from experience E, with respect to task T, and performance
measure P, if its performance on T measured by P improves with experience E.”

Example:

o Task (T): Object detection in images.

o Experience (E): Training on thousands of labeled images.
o Performance (P): Metrics like precision and recall measure detection accuracy.

Steps in Machine Learning:

1. Data Collection: Gathering raw data from various sources.

2. Abstraction: Forming abstract concepts from data (e.g., recognizing an elephant’s
features).
3. Generalization:
o Converts abstractions into actionable intelligence.
o Involves ranking concepts, inferencing, and forming heuristics (rules of
thumb).
4. Evaluation:
o Tests the thoroughness of models.
o Identifies and corrects errors to improve system performance.

Machine Learning- BCS602

Page 5
 RV Institute of Technology and Management®

_________________________________________________________________________________

Machine Learning as Human Experience:

Computers mimic human experience through data.

Human Analogy:

o Humans learn by rote, observation, teachers, books, and trial-and-error.

o When faced with new problems, humans apply past experiences and heuristics.

Machine Analogy:

o Machines "learn" from data, generalize patterns, and form rules (heuristics) to solve tasks.

Role of Data in Machine Learning:

Quality of Data: Directly affects the accuracy of the learning system.

Statistical Learning Function: Maps input (x) to output (y) as y=f(x).

Goal: Learn the function f from data to accurately predict outcomes for new inputs.

Machine Learning- BCS602

Page 6
 RV Institute of Technology and Management®

_________________________________________________________________________________

Limitations of Heuristics:

Heuristics are educated guesses and may fail occasionally.

Course corrections are made through evaluation and improved formulations.

Machine Learning in Relation to Other Fields Machine

Learning and Artificial Intelligence (AI):

Machine learning is a sub-branch of AI, which aims to develop intelligent agents (e.g., robots,
humans, or autonomous systems).

AI originally focused on logic and reasoning but evolved into data-driven systems for finding
relationships in data.

Machine learning focuses on extracting patterns for prediction, while deep learning, a sub-
branch of machine learning, uses neural networks modeled on human neurons.

Machine Learning- BCS602

Page 7
 RV Institute of Technology and Management®

_________________________________________________________________________________

Machine Learning and Data Science:

Data science is an umbrella term encompassing multiple fields, including machine learning.

Machine learning starts with data and focuses on analyzing patterns for prediction. Some Key

fields under data science include:

Big Data: Focuses on large datasets with the following characteristics:

 Volume: Huge amounts of data (e.g., from Facebook, YouTube).

 Variety: Data in different forms (e.g., images, videos).
 Velocity: Speed at which data is generated and processed.

Data Mining: Unearths hidden patterns in data (similar to machine learning but focuses on
pattern discovery).

Data Analytics: Extracts useful knowledge from raw data. Predictive data analytics is closely
related to machine learning.

Machine Learning- BCS602

Page 8
 RV Institute of Technology and Management®

_________________________________________________________________________________

Pattern Recognition:

An engineering field that uses machine learning algorithms for feature extraction, pattern
analysis, and classification.

Considered a specific application of machine learning.

Machine Learning and Statistics:

Statistics:

Focuses on hypotheses, experiments, and rigorous mathematical models to find relationships

in data.

Requires a strong theoretical background and often involves complex equations and assumptions.

Machine Learning:

Less assumption-heavy compared to statistics and automates the learning process. Often

viewed as an evolved version of "old statistics."

Difference: Statistics aims to validate hypotheses, while machine learning focuses on predictions
and automation.

Some Key Takeaways:

Machine learning bridges AI, data science, statistics, and pattern recognition. Deep

learning, big data, and data analytics significantly influence its development.

Machine learning simplifies traditional statistical processes while enhancing predictive power.

Machine Learning- BCS602

Page 9
 RV Institute of Technology and Management®

_________________________________________________________________________________

Types of Machine Learning

Machine learning involves learning through the interaction of a program with its environment.

The types of learning are categorized based on the data used and the supervision provided
during the learning process.

Labelled and Unlabelled Data

Labelled Data:

Machine Learning- BCS602

Page 10
 RV Institute of Technology and Management®

_________________________________________________________________________________

Data with target attributes (labels) is used for supervised learning.

For example, the Iris dataset includes features (e.g., sepal length, petal width) with a target
label (class of Iris flower).

Unlabelled Data:

Data without target attributes is used for unsupervised learning.

The algorithm identifies patterns or groups without explicit supervision.

Supervised Learning

Involves a teacher or supervisor providing labelled data for training and testing.

Two Key Methods: Classification:

Predicts discrete labels or classes.

Machine Learning- BCS602

Page 11
 RV Institute of Technology and Management®

_________________________________________________________________________________

Example: Classifying images as cats or dogs.

Algorithms:

 Decision Trees
 Random Forest
 Support Vector Machines (SVM)
 Naïve Bayes
 Artificial Neural Networks (e.g., CNNs for deep learning)

Classification models:

Generative Models: Focus on data generation (e.g., probabilistic models).

Discriminative Models: Focus on classification only.

Machine Learning- BCS602

Page 12
 RV Institute of Technology and Management®

_________________________________________________________________________________

Regression:

Predicts continuous variables, such as prices or sales.

Example: Predicting product sales over weeks using linear regression.

Algorithm:

Linear Regression: Fits a line (e.g., y=0.66x+0.54y = 0.66x + 0.54y=0.66x+0.54) to predict

outcomes based on independent variables.

Unsupervised Learning

Self-instruction-based learning with no supervisor or labelled data. Algorithms

observe examples, recognize patterns, and group similar objects.

Machine Learning- BCS602

Page 13
 RV Institute of Technology and Management®

_________________________________________________________________________________

Methods:

Cluster Analysis:

Groups objects into disjoint clusters based on attributes. Example:

 Image segmentation to detect abnormal growth in medical images.

 Clustering gene signatures in a database.

Algorithms:

 k-means algorithm
 Hierarchical algorithms

Dimensionality Reduction:

 Simplifies data while retaining essential features.

Machine Learning- BCS602

Page 14
 RV Institute of Technology and Management®

_________________________________________________________________________________

Semi-Supervised Learning

Combines labelled and unlabelled data, leveraging small labelled datasets and larger unlabelled
datasets.

Useful when labelling data is expensive or time-consuming.

Differences Between Classification and Regression

Dimensionality Reduction

Dimensionality reduction is a type of unsupervised learning where high-dimensional data is

transformed into a lower-dimensional space while retaining the critical variance or patterns in
the dataset.

The goal is to simplify the dataset by reducing the number of features without losing its essential
information.

Differences between Supervised and Unsupervised Learning

Applications:

Machine Learning- BCS602

Page 15
 RV Institute of Technology and Management®

_________________________________________________________________________________
 Reducing computational complexity for machine learning models.

 Visualizing high-dimensional data in 2D or 3D (e.g., t-SNE for visualizing

clusters).
 Preprocessing data to remove noise or redundant features.

Examples of Dimensionality Reduction Algorithms:

Principal Component Analysis (PCA): Projects data into components that explain the
maximum variance.

Linear Discriminant Analysis (LDA): Finds the linear combinations of features that best
separate classes.

t-Distributed Stochastic Neighbor Embedding (t-SNE): Focuses on visualizing high-

dimensional data in 2D or 3D.

Autoencoders: Neural networks that compress and reconstruct data.

Semi-Supervised Learning

Semi-supervised learning operates in scenarios where a large portion of the dataset is unlabelled,
and only a small portion is labelled.

Since labelling data is expensive and time-consuming, semi-supervised algorithms make use of
unlabelled data by assigning pseudo-labels (predicted labels).

Process:

Unlabelled data is assigned pseudo-labels based on model predictions.

The pseudo-labelled data is combined with the labelled data to train the model.

Applications:

Machine Learning- BCS602

Page 16
 RV Institute of Technology and Management®

_________________________________________________________________________________
 Image recognition (with limited labelled samples).

 Natural language processing for text classification.

 Medical diagnosis where labelling data is costly.

Reinforcement Learning

Reinforcement learning is inspired by how humans learn through interaction and feedback.

An agent interacts with the environment, perceives its state, takes actions, and receives rewards
or penalties.

The agent aims to maximize cumulative rewards by learning the best actions to take over
time.

Key Concepts:

Agent: The decision-maker (e.g., a robot, software, or a program).

Environment: The space where the agent interacts.
Actions: Choices available to the agent (e.g., move left, right, up, down in a grid game).
Reward: Feedback signal for actions (positive or negative).

Machine Learning- BCS602

Page 17
 RV Institute of Technology and Management®

_________________________________________________________________________________

Policy: The strategy the agent uses to determine actions.

Goal: Achieve the highest cumulative reward over time.

Example - Grid Game:

In the grid game:

 Gray tiles represent danger.

 Black tiles are blocks.
 Diagonal-striped tile is the goal.
The agent learns to navigate from the bottom-left grid to the goal by exploring various
paths and gaining rewards for successful actions or penalties for poor decisions.

Characteristics of Reinforcement Learning:

No labelled data: The agent learns through interaction instead of relying on labelled datasets.

Sequential decision-making: The agent must take multiple steps to achieve the goal.

Reward-driven learning: Positive rewards reinforce desirable behavior, while negative

rewards discourage it.

Challenges of Machine Learning

Machine learning, while powerful, faces several challenges that stem from its reliance on data
quality, computational resources, and algorithmic efficiency.

Below are the key challenges, explained in detail:

Machine Learning- BCS602

Page 18
 RV Institute of Technology and Management®

_________________________________________________________________________________

Problems with Ill-posed Problems

Machine learning performs well on well-posed problems, where specifications are complete,
clear, and well-defined.

However, ill-posed problems—where there is insufficient information or multiple valid

solutions—are a significant challenge.

o For instance, given a dataset where y = X1 X X2 , it’s equally plausible that y =

X1 ÷ X2 or y = X1 + X2 . Without additional data or constraints, it’s impossible
to determine the correct function.
o Solution: Ill-posed problems require more comprehensive data, better-
defined objectives, or external constraints to improve their solvability.

Quality of Data

Machine learning systems heavily rely on data quality.

Challenges with data include:

o Insufficient Data: Models require large datasets to learn effectively.

o Incomplete Data: Missing values can reduce model accuracy.
o Incorrect Data: Erroneous or noisy data introduces biases and errors.

Machine Learning- BCS602

Page 19
 RV Institute of Technology and Management®

_________________________________________________________________________________

o Imbalanced Data: Datasets where certain classes are underrepresented lead to poor
performance for those classes.

Solution: High-quality preprocessing, data augmentation, and imputation techniques can

address these issues.

High Computational Power Requirements

The growth of Big Data and complex machine learning tasks (e.g., deep learning) demands
significant computational resources:

Hardware Requirements: Systems need GPUs, TPUs, or distributed computing for faster
processing.

Time Complexity: Training sophisticated models can take hours, days, or even weeks.

Solution: Leveraging cloud computing platforms, distributed training, and optimized frameworks
(e.g., TensorFlow, PyTorch) can mitigate computational bottlenecks.

Algorithmic Complexity

The complexity of machine learning algorithms presents challenges in: Selection:

Identifying the best algorithm for a specific task.

Implementation: Understanding and applying advanced algorithms.

Optimization: Designing and fine-tuning algorithms to maximize efficiency and accuracy.

Evaluation: Comparing algorithm performance to ensure optimal model selection.

Solution: Continuous research, education, and development of automated machine learning

(AutoML) tools simplify the design and selection process.

Machine Learning- BCS602

Page 20
 RV Institute of Technology and Management®

_________________________________________________________________________________

Bias/Variance Tradeoff

Bias and variance are sources of error in machine learning models:

Bias: Occurs when the model oversimplifies the problem, leading to underfitting.

Variance: Occurs when the model overfits the training data, failing to generalize to new
data.

Striking the right balance between bias and variance—called the bias-variance tradeoff—is
a persistent challenge.

Solution:

Use cross-validation techniques to assess model performance.

Regularization methods (e.g., L1, L2 regularization) can help reduce overfitting.

Ensemble methods like bagging (e.g., Random Forests) and boosting (e.g., XGBoost) improve
generalization.

Interpretability and Explainability

Many machine learning models, particularly deep learning, are considered black boxes,
making it difficult to explain their predictions or decisions.

Solution:

Use simpler, interpretable models where feasible.

Employ explainability tools like SHAP (SHapley Additive exPlanations) or LIME (Local
Interpretable Model-agnostic Explanations).

Machine Learning- BCS602

Page 21
 RV Institute of Technology and Management®

_________________________________________________________________________________

Ethical Concerns and Bias in Data

Biases present in training data can lead to biased models, perpetuating unfair decisions (e.g.,
in hiring, lending, or law enforcement).

Solution: Implement fairness-aware machine learning techniques, conduct audits, and use
diverse datasets.

Scalability

Scaling machine learning models to work with massive datasets across distributed systems is
a technical challenge.

Solution: Use scalable tools and frameworks like Apache Spark, Dask, or cloud-based machine
learning services.

Machine Learning Process (CRISP-DM)

Machine Learning- BCS602

Page 22
 RV Institute of Technology and Management®

_________________________________________________________________________________

The CRISP-DM (Cross-Industry Standard Process for Data Mining) methodology

provides a structured framework for building data mining and machine learning
solutions.

It is widely adopted because of its versatility across industries. Below are the six steps of the
CRISP-DM process:

Understanding the Business

Objective: Grasp the business goals and define the problem statement.

Activities:

o Identify the organization's objectives and challenges.

o Formulate a clear problem statement for the machine learning solution.
o Choose a data mining or machine learning algorithm that aligns with the
business goals.

Understanding the Data

Objective: Explore and understand the data available for solving the problem.

Activities:

o Collect and analyze data from different sources.

o Study the data's characteristics, such as trends, anomalies, and distributions.
o Formulate hypotheses based on the data and match patterns to the
hypotheses.

Data Preparation

Objective: Clean and preprocess the raw data for analysis.

Machine Learning- BCS602

Page 23
 RV Institute of Technology and Management®

_________________________________________________________________________________

Activities:

o Handle missing values, duplicates, and irrelevant data.

o Normalize or scale the data as required.
o Prepare a finalized dataset suitable for the machine learning task.

Importance: Data quality directly impacts the performance of the machine learning model.

For instance, missing values can degrade model accuracy.

Modeling

Objective: Apply machine learning algorithms to extract patterns or build models.

Activities:

o Select appropriate machine learning algorithms (e.g., classification, regression,

clustering).
o Train the model on the prepared dataset.
o Tune hyperparameters to optimize model performance.

Evaluation

Objective: Assess the performance of the model using statistical and visualization techniques.

Activities:

o Measure accuracy, precision, recall, F1-score, and other metrics.

o Compare results with baseline models or domain benchmarks.
o Ensure the model aligns with business objectives.

Challenges: Tasks like email classification may require expert domain knowledge for proper
evaluation.

Machine Learning- BCS602

Page 24
 RV Institute of Technology and Management®

_________________________________________________________________________________

Deployment

Objective: Deploy the model to provide actionable insights or improve processes.

Activities:

o Integrate the model into the business workflow or system.

o Monitor its performance in real-world scenarios.
o Update the model as new data becomes available.

Applications of Machine Learning

Machine learning is transforming various industries and is increasingly present in everyday

life.

Below are some key applications:

Sentiment Analysis

Use Case: Natural Language Processing (NLP) techniques analyze text to determine
sentiments (e.g., happy, sad, angry).

Example: Movie and product reviews can be analyzed to assign ratings (e.g., stars) automatically.

Recommendation Systems

Use Case: Personalizes user experiences by suggesting items based on preferences. Example:

 Amazon: Recommends related books or products.

 Netflix: Suggests shows or movies based on viewing history.

Machine Learning- BCS602

Page 25
 RV Institute of Technology and Management®

_________________________________________________________________________________

Voice Assistants

Use Case: Use speech recognition and natural language understanding to perform tasks.

Examples: Amazon Alexa, Apple Siri, Google Assistant, Microsoft Cortana.

Navigation Systems

Use Case: Optimize routes and reduce travel time using machine learning-based
algorithms.

Examples: Google Maps, Uber.

Machine Learning- BCS602

Page 26
 RV Institute of Technology and Management®

_________________________________________________________________________________

Page 27

Machine Learning- BCS602

 RV Institute of Technology and Management®

_________________________________________________________________________________

Page 28

Machine Learning- BCS602

 RV Institute of Technology and Management®

_________________________________________________________________________________

Page 29

Machine Learning- BCS602

 RV Institute of Technology and Management®

_________________________________________________________________________________

Page 30

Machine Learning- BCS602

 RV Institute of Technology and Management®

_________________________________________________________________________________

Page 31

Machine Learning- BCS602

 RV Institute of Technology and Management®

_________________________________________________________________________________

Page 32

Machine Learning- BCS602

 RV Institute of Technology and Management®

_________________________________________________________________________________

Page 33

Machine Learning- BCS602

 RV Institute of Technology and Management®

_________________________________________________________________________________

Page 34

Machine Learning- BCS602

 RV Institute of Technology and Management®

_________________________________________________________________________________

Page 35

Machine Learning- BCS602

 RV Institute of Technology and Management®

_________________________________________________________________________________

Page 36

Machine Learning- BCS602

 RV Institute of Technology and Management®

_________________________________________________________________________________

Page 37

Machine Learning- BCS602

 RV Institute of Technology and Management®

_________________________________________________________________________________

Page 38

Machine Learning- BCS602

 RV Institute of Technology and Management®

_________________________________________________________________________________

Page 39

Machine Learning- BCS602

 RV Institute of Technology and Management®

_________________________________________________________________________________

Page 40

Machine Learning- BCS602

 RV Institute of Technology and Management®

_________________________________________________________________________________

Page 41

Machine Learning- BCS602

 RV Institute of Technology and Management®

_________________________________________________________________________________

Page 42

Machine Learning- BCS602

 RV Institute of Technology and Management®

_________________________________________________________________________________

Page 43

Machine Learning- BCS602

 RV Institute of Technology and Management®

_________________________________________________________________________________

Machine Learning- BCS602 Page 44