MACHINE LEARNING

UNIT-01
Definition of Learning
• Learning involves changes in behaviour due to experiences, focusing on
adapting rather than relying on instinct or temporary states.
• Components of a Learning System:
o Performance Element : Determine actions based on existing strategies.
o Learning element : Improves the performance element by analysing past
outcomes.
INFLUENCES INCLUDE:
✓ Component of performance : Understanding existing capabilities.
✓ Feedback mechanism : Using feedback to enhance performance,
✓ Knowledge Representation : How information is organized and
accessed.

• Acquisition of New Knowledge:

• Essential to learning; involves understanding new information, similar to how students
learn new mathematical techniques.
• Problem Solving:
• Integrates new knowledge and deduces solutions when not all data is available, akin to a
doctor diagnosing illnesses with limited information.
Performance measures for learning
Generality
• Generality refers to a machine learning model's ability to perform well across
various datasets and environments, not just the one it was trained on. For
instance, a facial recognition system that can accurately identify faces in
diverse lighting conditions and angles demonstrates good generality.
Efficiency
• Efficiency in machine learning measures how quickly a model can learn from
data. A spam detection algorithm that quickly adapts to new types of spam
emails with minimal training data exhibits high efficiency.
Robustness
• Robustness is the ability of a model to handle errors, noise, and unexpected
data without failing. A voice recognition system that can understand
commands in a noisy room shows robustness.
Efficacy
•Efficacy is the overall effectiveness of a machine learning model in performing its
intended tasks. An autonomous driving system that safely navigates city traffic and avoids
accidents under various conditions demonstrates high efficacy.
Ease of Implementation
• This measures how straightforward it is to develop and deploy a machine
learning model. A recommendation system that can be integrated into an
existing e-commerce platform using standard algorithms and software
libraries highlight ease of implementation.
Supervised Learning
• Supervised learning involves training a machine learning model using labelled
data, which means the data is already associated with the correct answer.
• Example: Consider teaching a child to identify fruits. You show them pictures of
various fruits, like apples and bananas, while telling them, "This is an apple,"
and "This is a banana." Over time, the child learns to identify fruits correctly based on the
examples given.
Key Steps in Supervised Learning:
• Input and Output Pairing: Each input (e.g., a fruit picture) is paired with its
correct label (e.g., "apple").
• Training: The model learns by comparing its prediction with the actual label
and adjusting itself to improve accuracy.
• Error Correction: If the model predicts incorrectly (e.g., calls an apple a
banana), it adjusts its internal parameters to reduce the error.
• Outcome: The model eventually learns to map inputs (fruit images) to the
correct outputs (fruit names).
Unsupervised learning
• Unsupervised learning involves training a model without any labels, which
means the model tries to identify patterns and data groupings on its own.
• Example: Imagine placing a mix of different coins on a table and asking a child to
sort them. Without explaining any criteria, the child might start grouping the
coins by size, colour, or denomination on their own.
• Key Steps in Unsupervised Learning:
• Input Without Labels: The model receives data without any explicit
instructions on what to do with it.
• Pattern Recognition: The model analyses the data and tries to find any
natural groupings or patterns (e.g., clustering coins based on size or colour).
• Self-Organization: The model organizes data into different categories based
on the patterns it perceives.
• Outcome: The model creates its own system of categorization without
external guidance.
Well-defined learning problems
• A well-defined learning problem allows a computer program to improve at a
specific task through experience. This is characterized by three key elements:
• Task (T): The specific activity or challenge the program is expected to
perform.
• Performance Measure (P): The criteria used to gauge the program's
effectiveness at the task.
• Experience (E): The data or interactions from which the program learns.
CHECKERS GAME:
• Task (T): Playing the game of checkers.
• Performance Measure (P): The percentage of games won against various
opponents.
• Experience (E): Engaging in numerous practice games, possibly including self-play.
Handwriting Recognition:

• Task (T): Identifying and categorizing handwritten words in images.

• Performance Measure (P): The accuracy rate, measured as the percentage of words
correctly recognized.
• Experience (E): Analysis of a large dataset of labelled handwritten word images.
Autonomous Driving Robot:
• Task (T): Navigating public four-lane highways using vision-based sensors.
• Performance Measure (P): The average distance the robot travels without making a
mistake, as determined by a human supervisor.
• Experience (E): Processing sequences of images and corresponding steering commands
previously collected from human drivers.

Overview of the history of Machine Learning

Early Developments:

• 1943: Neurophysiologist Warren McCulloch and mathematician Walter Pitts

introduced the concept of a neural network by modelling neurons with electrical
circuits.

• 1952: Arthur Samuel developed the first computer program capable of learning from
its activities.
• 1958: Frank Rosenblatt created the Perceptron, the first artificial neural network,
which was designed for pattern and shape recognition
Early Developments:
• 1959: Bernard Widrow and Marcian Hoff developed two neural network models:
ADELINE, which could detect binary patterns, and MADELINE, which was used to
reduce echo on phone lines.
• 1982: John Hopfield proposed a network with bidirectional lines that mimicked
actual neuronal structures.

• 1986: The backpropagation algorithm was popularized, allowing the use of multiple
layers in neural networks, enhancing their learning capabilities.

• 1997: IBM’s Deep Blue, a chess-playing computer, famously beat the reigning world
chess champion.
• 1998: AT&T Bell Laboratories achieved significant progress in digit recognition,
notably enhancing the ability to recognize handwritten postcodes for the US Postal
Service.
 21st Century Innovations:

• The 21st century has seen a significant surge in machine learning, driven by both
industry and academia, to boost computational capabilities and innovation.
• Notable projects include:
• GoogleBrain (2012): A deep learning project.
• AlexNet (2012): A deep convolutional neural network.
• DeepFace (2014) and DeepMind (2014): Projects that advanced facial recognition
and AI decision-making.
• OpenAI (2015), ResNet (2015), and U-net (2015): Each contributed to
advancements in AI capabilities, from gameplay to medical imaging.
Machine learning
• Machine learning is a subset of artificial intelligence (AI) that enables computers to learn
from and make decisions based on data, without being explicitly programmed.
• Definition: Machine learning involves developing algorithms that allow computers to
process and learn from data automatically.
• Purpose: The aim is to enable computers to learn from their experiences and improve their
performance over time without human intervention.
• Functionality: Machine learning algorithms analyse vast amounts of data, enabling them to
perform tasks more efficiently and accurately. This could be anything from predicting
consumer behaviour to detecting fraudulent transactions.
• Integration: Combining machine learning with AI and cognitive technologies enhances its
ability to process and interpret large volumes of complex data.
Example: Consider a streaming service like Netflix. Machine learning is used to analyze your
viewing habits and the habits of others with similar tastes. Based on this data, the system
recommends movies and shows that you might like. Here, the algorithm learns from the
accumulated data to make increasingly accurate predictions over time, thereby enhancing
user experience without manual intervention. This demonstrates machine learning’s
capability to adapt and improve autonomously, making it a powerful tool in many tech-
driven applications.
Machine learning has a wide range of applications across different fields, here are some key
applications along with examples:
• Image Recognition:
• Application: Image recognition involves identifying objects, features, or patterns within
digital images or videos.
• Example: Used in facial recognition systems for security purposes or to detect defective
products on assembly lines in manufacturing.
• Medical Diagnosis:
• Application: Machine learning assists in diagnosing diseases by analysing clinical
parameters and their combinations.
• Example: Predicting diseases such as diabetes or cancer by examining patient data and
previous case histories to identify patterns that precede diagnoses.
• Statistical Arbitrage:
• Application: In finance, statistical arbitrage involves automated trading strategies that
capitalize on patterns identified in trading data. • Example: Algorithmic trading platforms
that analyse historical stock data to make buy or sell decisions in milliseconds to capitalize
on market inefficiencies.
Learning Associations:
• Application: This process uncovers relationships between variables in large databases,
often revealing hidden patterns.
• Example: Market basket analysis in retail, which analyses purchasing patterns to
understand product associations and optimize store layouts.
Information Extraction:
• Application: Information extraction involves pulling structured information from
unstructured data, like text.
• Example: Extracting key pieces of information from legal documents or news articles to
summarize content or populate databases automatically.
• Information Extraction:
• Application: Information extraction involves pulling structured information from
unstructured data, like text.
• Example: Extracting key pieces of information from legal documents or news articles to
summarize content or populate databases automatically.
Advantages of Machine Learning:
• Automation:
• Example: Autonomous vehicles use machine learning to interpret sensory data and make
driving decision without human input, improving transportation efficiency and safety.
• Continuous Improvement:
• Example: Credit scoring systems evolve by learning from new customer data, becoming
 more accurate in predicting creditworthiness over time.
• Handling Complex Data:
• Example: Financial institutions use machine learning algorithms to detect fraudulent
transactions by analysing complex patterns of customer behaviour that would be difficult for
humans to process.
Disadvantages of Machine Learning:
• Data Acquisition:
• Example: In healthcare, acquiring large datasets of patient medical records that are
comprehensive and privacy-compliant is challenging and expensive.
• Time and Resources:
• Example: Developing a machine learning model for predicting stock market trends
requires extensive computational resources and time to analyse years of market data before
it can be deployed
• Interpretation of Results:
• Example: In genomics research, interpreting the vast amounts of data produced by
machine learning algorithms requires highly specialized knowledge to ensure findings are
accurate and meaningful.
• High Error-Susceptibility:
• Example: Early stages of facial recognition technology showed high error rates,
particularly in accurately identifying individuals from minority groups, leading to potential
biases and inaccuracies.
Machine Learning Approaches Artificial Neural Network
Inspiration: ANNs mimic the structure and function of the nervous systems in animals,
particularly how neurons transmit signals.
Functionality: These networks are used for machine learning and pattern recognition,
handling complex data inputs effectively.
Components of ANNs:
• Neurons: Modelled as nodes within a network.
• Connections: Nodes are linked by arcs that represent synapses, with weights that signify
the strength of each connection.
• Processing: The network processes signals in a way analogous to neural activity in
biological brains.

• Operation:
• Signal Transmission: Connections in the network facilitate the propagation of data, similar
to synaptic transmission in biology.
• Information Processing: ANNs adjust the weights of connections to learn from data and
make informed decisions.
 Clustering

• Definition: Clustering is the process of sorting items into groups based on their similarities,
forming distinct clusters where items within each cluster are more alike to each other than
to those in other clusters.
• Visual Representation: Imagine organizing fruits into groups by type, such as grouping
apples together, oranges in another group, and bananas in a separate one, visually
representing how clusters segregate similar items.

• Characteristics: Clusters act like exclusive clubs, where members share common traits but
differ significantly from members of other clusters, illustrating the distinctiveness of each
group.
• Multidimensional Space: Clusters are akin to islands in an expansive ocean, with dense
population points representing similar items within each cluster, and low-density water
symbolizing dissimilar items separating clusters.

• Machine Learning Perspective: Clustering entails discovering patterns without explicit

guidance, akin to exploring a forest without a map, where similarities guide the grouping
process. It's a form of unsupervised learning, akin to solving a puzzle without knowledge of
the final solution.
• Unsupervised Learning: Clustering is learning through observation, not instruction. It's like
solving a puzzle without knowing what the final picture looks like.
• Data Reduction:
• Example: Imagine sorting a massive collection of books into genres (fiction, non-fiction,
sci-fi, etc.). Clustering reduces the data into manageable chunks for easier processing.
• Hypothesis Generation:
• Example: Grouping customer purchase data to generate hypotheses about shopping
preferences, which can then be tested with additional research.
• Hypothesis Testing:
• Example: Using clustering to verify if certain customer segments show different
purchasing behaviours, confirming or disproving existing hypotheses.
• Prediction Based on Groups:
• Example: Suppose we have a dataset of customer demographics and spending habits. By
clustering similar customers, we can predict the behaviour of new customers based on
their group's characteristics. For instance, if a new customer share similarity with the
"budget- conscious” cluster ,we can predict their spending patterns accordingly.
• Hierarchical Clustering:
• Agglomerative Hierarchical Clustering: Treats each data point as its own cluster, then
merges clusters into larger ones. For example, a dataset of academic papers starts with each
paper as its own cluster, then papers on similar topics merge into bigger clusters.
• Divisive Hierarchical Clustering: Starts with all data points in one cluster and splits them
into smaller clusters. For instance, starting with one cluster of all store customers, the cluster
is split based on purchasing behaviour until each customer forms their own cluster.
• Partitional Clustering:
• Centroid-based Clustering (e.g., K-means): Partitions data into clusters, each represented
by a centroid. Clusters minimize distance between data points and centroid, optimizing intra-
cluster similarity and inter-cluster dissimilarity. For example, retail customers can be
clustered by buying patterns, with each cluster's centroid reflecting average behavior.
• Model-based Clustering: Uses a statistical model for each cluster, finding the best data fit.
For instance, Gaussian mixture models assume data points in each cluster are Gaussian
distributed. This method is used in image processing to model different textures as coming
from different Gaussian distributions.

Density-based Clustering (e.g., DBSCAN):

• This method clusters points that are closely packed together, marking as outliers points
that lie alone in low-density regions. This is useful in geographical data analysis where, for
example, identifying regions of high economic activity based on point density of businesses
can be achieved.
• Grid-based Clustering:
• This method quantizes the space into a finite number of cells that form a grid structure and
then performs clustering on the grid structure. This is effective for large spatial data sets, as it
speeds up the clustering process. For example, in meteorological data, clustering can be
applied to grid squares to categorize regional weather patterns.
• Spectral Clustering:

• Uses the eigenvalues of a similarity matrix to reduce dimensionality before clustering in fewer
dimensions. This technique is particularly useful when the clusters have a complex shape, unlike
centroid-based clustering which assumes spherical clusters. For example, in social network analysis,
spectral clustering can help identify communities based on the patterns of relationships between
members.

DECISION TREE

A decision tree is a model used in data mining, statistics, and machine learning to predict an
outcome based on input variables. It resembles a tree structure with branches and leaves, where
each internal node represents a "decision" based on a feature, each branch represents the outcome
of that decision, and each leaf node represents the final outcome or class label.

• Advantages and Limitations:

• Advantages:

• Easy to interpret and visualize.

• Requires little data preparation compared to other algorithms.

• Can handle both numerical and categorical data.

• Limitations:

• Prone to overfitting, especially with many branches.

• Can be biased towards features with more levels.

• Decisions are based on heuristics, hence might not provide the best split in some cases.

BAYESIAN BELIEF NETWORKS

• Are tools for representing and reasoning under conditions of uncertainty. They capture the
probabilistic relationships among a set of variables and allow for the inference of probabilities even
with partial information.
• Structure: The core components of a Bayesian belief network include:

• Directed Acyclic Graph (DAG): Each node in the graph represents a random variable, which can be
either discrete or continuous. These variables often correspond to attributes in data. Arrows or arcs
between nodes represent causal influences.

• Conditional Probability Tables (CPTs): Each node has an associated table that quantifies the effect
of the parents on the node.

• Usage:

• Learning: Bayesian networks can be trained using data to learn the conditional dependencies.

• Inference: Once trained, the network can be used for inference, such as predicting the likelihood of
lung cancer given that a patient is a smoker with no family history.

• Classification: Bayesian networks can classify new cases based on learned probabilities.
 Reinforcement learning

• Reinforcement learning is a type of machine learning where an agent learns to make decisions by
performing actions and receiving feedback in the form of rewards or penalties. This method is similar
to how individuals learn from the consequences of their actions in real life.

• Key Concepts in Reinforcement Learning:

• Environment: The world in which the agent operates.

• State: The current situation of the agent.

• Actions: What the agent can do.

• Rewards: Feedback from the environment which can be positive (reinforcements) or negative
(punishments).

• Imagine a robot navigating a maze. The robot has to find the shortest path to a destination without
prior knowledge of the layout. Each step it takes provides new information:

• If it moves closer to the destination, it receives a positive reward.

• If it hits a wall or moves away from the goal, it receives a negative reward. Through trial and error,
the robot learns the optimal path by maximizing its cumulative rewards.

Support Vector Machine

• A Support Vector Machine (SVM) is a powerful machine most commonly used in classification
problems.

• SVM constructs a hyperplane or set of hyperplanes in a high-dimensional space, which can be used
for classification. The goal is to find the best hyperplane that has the largest distance to the nearest
training data points of any class (functional margin), in order to improve the classification
performance on unseen data.

• Applications of SVM:

• Text and Hypertext Classification: For filtering spam and categorizing text based content for news
articles.
• Image Classification: Useful in categorizing images into different groups (e.g., animals, cars, fruits).
• Handwritten Character Recognition: Used to recognize letters and digits from handwritten
documents.

• Biological Sciences: Applied in protein classification and cancer classification based on gene
expression data.

Genetic Algorithm

• A genetic algorithm (GA) is a search heuristic inspired by Charles Darwin's theory of natural
selection. It is used to find optimal or near-optimal solutions to complex problems which might
otherwise take a long time to solve.

• Overview of Genetic Algorithm:

• Purpose: Genetic algorithms are used to solve optimization and search problems by mimicking the
process of natural selection.

• Process: This involves a population of individuals which evolve towards a better solution by
combining the characteristics of high-quality individuals.

Flowchart of Genetic Algorithm Process:

1. Initialize Population: Start with a randomly generated population of n individuals.

2. Fitness Evaluation: Evaluate the fitness of each individual in the population. The fitness score
determines how good an individual solution is at solving the problem.

3. Selection: Select pairs of individuals (parents) based on their fitness scores. Higher fitness scores
generally mean a higher chance of selection.

4. Crossover (Recombination): Combine the features of selected parents to create offspring. This
simulates sexual reproduction.

5. Mutation: Introduce random changes to individual offspring to maintain genetic diversity within
the population.

6. Replacement: Replace the older generation with the new generation of offspring.

7. Termination: Repeat the process until a maximum number of generations is reached or a

satisfactory fitness level is achieved.
Example of Genetic Algorithm: Imagine we want to optimize the design of an aerodynamic car. The
objective is to minimize air resistance, which directly impacts fuel efficiency.

• Encoding: Each car design is encoded as a string of numbers (genes), representing different design
parameters like shape, size, and materials.

• Initial Population: Generate a random set of car designs.

• Fitness Evaluation: Use a simulation to calculate the air resistance of each design.

• Selection: Choose designs with the lowest air resistance.

• Crossover: Create new designs by mixing the features of selected designs.

• Mutation: Slightly alter the designs to explore a variety of design possibilities.

• Repeat: Continue the process to evolve increasingly efficient designs over multiple generations.

Issues in Machine Learning

• Data Quality:

• Importance of Quality: High-quality data is crucial for developing effective ML models. Poor data
can lead to inaccurate predictions and unreliable outcomes.

• Challenges:
 • Data Evaluation and Integration: Ensuring data is clean, well-integrated, and representative. For
example, a model trained to recognize faces needs a diverse dataset that reflects various ethnicities,
ages, and lighting conditions.

• Data Exploration and Governance: Implementing robust data governance to maintain the
integrity and usability of data over time.

• Transparency:

• Model Explainability: ML models, especially complex ones like deep neural networks, can act as
"black boxes," where it's unclear how decisions are made.

• Example: In a credit scoring model, it's crucial for regulatory and fairness reasons to explain why a
loan application was denied, which can be challenging with highly complex ML models.

• Manpower:

• Skill Requirement: Effective use of ML requires a combination of skills in data science, software
development, and domain expertise.

• Bias Avoidance: Having diverse teams is important to prevent biases in model development. •
Example: An organization implementing an ML solution for customer service might need experts in
natural language processing, software engineering, and customer interaction to develop a
comprehensive tool.

• Other Issues:

• Misapplication of Technology: ML is not suitable for every problem, and its misuse can lead to
wasted resources or poor decisions.

• Example: Employing deep learning for a simple data analysis task, where traditional statistical
methods would be more appropriate and less costly.

• Innovation Misuse: The hype around new ML techniques can lead to premature adoption without
proper understanding or necessity.

• Example: The early overuse of deep learning in situations where simpler models could suffice, like
predicting straightforward outcomes from small datasets.

• Traceability and Reproducibility: Ensuring that ML experiments are reproducible and that results
can be traced back to specific data and configuration settings.

• Example: A research team must be able to replicate an ML experiment's results using the same
datasets and parameters to verify findings and ensure reliability.