Fundamentals of Data Science

BCA – VI Semester NEP

By:

Rakshita Naik & Chitrashree K

Kamadhenu BCA College

Published by

Kamadhenu BCA College Karwar

Affiliated to Karnatak University Dharwad, Redg by Govt of Karnataka

Kamadhenu BCA College 1

 UNIT I - DATA MINING
INTRODUCTION

Data mining refers to filtering, sorting, and classifying data from larger datasets to reveal subtle
patterns and relationships, which helps enterprises identify and solve complex business
problems through data analysis. Data mining software tools and techniques allow
organizations to foresee future market trends and make business-critical decisions at crucial
times.

Data mining is an essential component of data science that employs advanced data analytics to
derive insightful information from large volumes of data. If we dig deeper, data mining is a
crucial ingredient of the knowledge discovery in databases (KDD) process, where data
gathering, processing, and analysis takes place at a fundamental level. Businesses rely heavily
on data mining to undertake analytics initiatives in the organizational setup. The analyzed data
sourced from data mining is used for varied analytics and business intelligence (BI) applications,
which consider real-time data analysis along with some historical pieces of information.

With top-notch data mining practices, enterprises can make several business strategies and
manage their operations better. This can entail refining customer-centric functions, including
advertising, marketing, sales, customer support, finance, HR, etc. Data mining also plays a vital
role in handling business-critical use cases such as cybersecurity planning, fraud detection, risk
management, and several others. Data mining finds applications across industry verticals such
as healthcare, scientific research, sports, governmental projects, etc.

Working:
Data mining is predominantly handled by a group of data scientists, skilled BI professionals,
analytics groups, business analysts, tech-savvy executives, and personnel having a solid
background and inclination toward data analytics.
Fundamentally, machine learning (ML), artificial intelligence (AI), statistical analysis, and data
management are crucial elements of data mining that are necessary to scrutinize, sort, and
prepare data for analysis. Top ML algorithms and AI tools have enabled the easy mining of
massive datasets, including customer data, transactional records, and even log files picked up
from sensors, actuators, IoT devices, mobile apps, and servers.

Key stages involved in the data mining process:

Kamadhenu BCA College 2

Data Mining Process

 Data gathering: Data mining begins with the data gathering step, where relevant
information is identified, collected, and organized for analysis. Data sources can
include data warehouses, data lakes, or any other source that contains raw data in a
structured or unstructured format.
 Data preparation: In the second step, fine-tuning the gathered data is the prime focus.
This involves several processes, such as data pre-processing, data profiling, and data
cleansing, to fix any data errors. These stages are essential to maintain data quality before
following up with the mining and analysis processes.
 Mining the data: In the third step, the data professional selects an appropriate data
mining technique once the desired quality of data is prepared. Here, a proper set of data
processing algorithms are identified where sample data is trained initially before running it
over the entire dataset.
 Data analysis and interpretation: In the last step, the results derived in the third
step are used to develop analytical models for making future business decisions. Moreover,
the data science team communicates the results to the concerned stakeholders via data
visualizations and other more straightforward techniques. The information is conveyed in
a manner that makes the content digestible for any non-expert working outside the field
of data science.

KDD vs Data Mining

KDD (Knowledge Discovery in Databases) is a field of computer science, which includes the
tools and theories to help humans in extracting useful and previously unknown information
(i.e., knowledge) from large collections of digitized data. KDD consists of several steps, and
Data Mining is one of them. Data Mining is the application of a specific algorithm to extract
patterns from data. Nonetheless, KDD and Data Mining are used interchangeably.

KDD

KDD is a computer science field specializing in extracting previously unknown and interesting
information from raw data. KDD is the whole process of trying to make sense of data by
developing appropriate methods or techniques. This process deals with low-level mapping data
into other forms that are more compact, abstract, and useful. This is achieved by creating short
reports, modelling the process of generating data, and developing predictive models that can
predict future cases.

Due to the exponential growth of data, especially in areas such as business, KDD has become a
very important process to convert this large wealth of data into business intelligence, as
manual extraction of patterns has become seemingly impossible in the past few decades.

For example, it is currently used for various applications such as social network analysis, fraud
detection, science, investment, manufacturing, telecommunications, data cleaning, sports,
information retrieval, and marketing. KDD is usually used to answer questions like what are the
main products that might help to obtain high-profit next year in V-Mart.

Kamadhenu BCA College 3

KDD Process Steps
Knowledge discovery in the database process includes the following steps, such as:

1. Goal identification: Develop and understand the application domain and the relevant
prior knowledge and identify the KDD process's goal from the customer perspective.

2. Creating a target data set: Selecting the data set or focusing on a set of variables or
data samples on which the discovery was made.

3. Data cleaning and preprocessing: Basic operations include removing noise if

appropriate, collecting the necessary information to model or account for noise,
deciding on strategies for handling missing data fields, and accounting for time
sequence information and known changes.

4. Data reduction and projection: Finding useful features to represent the data depending
on the purpose of the task. The effective number of variables under consideration may
be reduced through dimensionality reduction methods or conversion, or invariant
representations for the data can be found.

5. Matching process objectives: KDD with step 1 a method of mining particular. For
example, summarization, classification, regression, clustering, and others.

6. Modelling and exploratory analysis and hypothesis selection: Choosing the algorithms
or data mining and selecting the method or methods to search for data patterns. This
process includes deciding which model and parameters may be appropriate (e.g.,
definite data models are different models on the real vector) and the matching of data
mining methods, particularly with the general approach of the KDD process (for
example, the end-user might be more interested in understanding the model in its
predictive capabilities).

7. Data Mining: The search for patterns of interest in a particular representational form or
a set of these representations, including classification rules or trees, regression, and
clustering. The user can significantly aid the data mining method to carry out the
preceding steps properly.

8. Presentation and evaluation: Interpreting mined patterns, possibly returning to some

of the steps between steps 1 and 7 for additional iterations. This step may also involve
the visualization of the extracted patterns and models or visualization of the data given
the models drawn.

Kamadhenu BCA College 4

 9. Taking action on the discovered knowledge: Using the knowledge directly,
incorporating the knowledge in another system for further action, or simply
documenting and reporting to stakeholders. This process also includes checking and
resolving potential conflicts with previously believed knowledge (or extracted).

Data Mining
Data mining, also known as Knowledge Discovery in Databases, refers to the nontrivial extraction
of implicit, previously unknown, and potentially useful information from data stored in
databases.

Data Mining is only a step within the overall KDD process. There are two major Data Mining goals
defined by the application's goal: verification of discovery. Verification verifies the user's
hypothesis about data, while discovery automatically finds interesting patterns.

There are four major data mining tasks: clustering, classification, regression, and association
(summarization). Clustering is identifying similar groups from unstructured data. Classification
is learning rules that can be applied to new data. Regression is finding functions with minimal
error to model data. And the association looks for relationships between variables. Then, the
specific data mining algorithm needs to be selected. Different algorithms like linear regression,
logistic regression, decision trees, and Naive Bayes can be selected depending on the goal.
Then patterns of interest in one or more symbolic forms are searched. Finally, models are
evaluated either using predictive accuracy or understandability.

Need of Data Mining

The volume of information is increasing every day that we can handle from business transactions,
scientific data, sensor data, Pictures, videos, etc. So, we need a system that will be capable of
extracting the essence of information available and that can automatically generate reports,
views, or summaries of data for better decision-making.

Difference between KDD and Data Mining

Although the two terms KDD and Data Mining are heavily used interchangeably, they refer to two
related yet slightly different concepts.

KDD is the overall process of extracting knowledge from data, while Data Mining is a step inside
the KDD process, which deals with identifying patterns in data. And Data Mining is only the
application of a specific algorithm based on the overall goal of the KDD process.

KDD is an iterative process where evaluation measures can be enhanced, mining can be refined,
and new data can be integrated and transformed to get different and more appropriate results.

DBMS Vs Data Mining

DBMS (Database Management System) and Data Mining are two different concepts, but they are
related in the broader context of managing and utilizing data effectively. Here's a brief
overview of each:

Kamadhenu BCA College 5

1. Database Management System (DBMS):
1. A DBMS is software that allows users to define, create, maintain, and control access to
databases.
2. It provides functionalities for storing, retrieving, updating, and managing data in a
structured format.
3. DBMS ensures data integrity, security, and concurrency control.
4. Examples of DBMS include MySQL, Oracle Database, Microsoft SQL Server, PostgreSQL,
etc.
5. DBMS primarily focuses on storing and managing data efficiently for various applications
and users.

2. Data Mining:
1. Data Mining is the process of discovering patterns, correlations, anomalies, and insights
from large datasets.
2. It involves using various techniques from statistics, machine learning, and artificial
intelligence to analyze data and extract valuable information.
3. Data Mining helps in making predictions, identifying trends, segmenting data, and
making data-driven decisions.
4. It is often used for tasks such as customer segmentation, market basket analysis, fraud
detection, and predictive maintenance.
5. Data Mining is closely related to disciplines like machine learning, data analytics, and
business intelligence.

While DBMS focuses on the storage and management of data, Data Mining focuses on
analyzing and extracting useful information from that data. DBMS provides the infrastructure
and tools necessary for storing and accessing data efficiently, while Data Mining techniques
operate on top of this data to derive insights and knowledge. In many cases, Data Mining
utilizes data stored within a DBMS, making them complementary technologies in the broader
realm of data management and analysis.

Data Mining Techniques

Data mining includes the utilization of refined data analysis tools to find previously unknown,
valid patterns and relationships in huge data sets. These tools can incorporate statistical
models, machine learning techniques, and mathematical algorithms, such as neural networks
or decision trees. Thus, data mining incorporates analysis and prediction.

Depending on various methods and technologies from the intersection of machine learning,
database management, and statistics, professionals in data mining have devoted their careers
to better understanding how to process and make conclusions from the huge amount of data,
but what are the methods they use to make it happen?

In recent data mining projects, various major data mining techniques have been developed and
used, including association, classification, clustering, prediction, sequential patterns, and
regression.

Kamadhenu BCA College 6

1. Classification:
This technique is used to obtain important and relevant information about data and
metadata. This data mining technique helps to classify data in different classes.
Data mining techniques can be classified by different criteria, as follows:

1. Classification of Data mining frameworks as per the type of data sources mined: This
classification is as per the type of data handled. For example, multimedia, spatial data,
text data, time-series data, World Wide Web, and so on.
2. Classification of data mining frameworks as per the database involved: This
classification based on the data model involved. For example. Object-oriented database,
transactional database, relational database, and so on..
3. Classification of data mining frameworks as per the kind of knowledge discovered:
This classification depends on the types of knowledge discovered or data mining
functionalities. For example, discrimination, classification, clustering, characterization,
etc. some frameworks tend to be extensive frameworks offering a few data mining
functionalities together..
4. Classification of data mining frameworks according to data mining techniques used:
This classification is as per the data analysis approach utilized, such as neural networks,
machine learning, genetic algorithms, visualization, statistics, data warehouse-oriented
or database-oriented, etc.

The classification can also take into account, the level of user interaction involved in the data
mining procedure, such as query-driven systems, autonomous systems, or interactive
exploratory systems.

2. Clustering:
Clustering is a division of information into groups of connected objects. Describing the data
by a few clusters mainly loses certain confine details, but accomplishes improvement. It
models data by its clusters. Data modelling puts clustering from a historical point of view
rooted in statistics, mathematics, and numerical analysis. From a machine learning point of
view, clusters relate to hidden patterns, the search for clusters is unsupervised learning, and
the subsequent framework represents a data concept. From a practical point of view,
clustering plays an extraordinary job in data mining applications. For example, scientific data
exploration, text mining, information retrieval, spatial database applications, CRM, Web
analysis, computational biology, medical diagnostics, and much more.

Kamadhenu BCA College 7

 In other words, we can say that Clustering analysis is a data mining technique to identify
similar data. This technique helps to recognize the differences and similarities between the
data. Clustering is very similar to the classification, but it involves grouping chunks of data
together based on their similarities.

3. Regression:
Regression analysis is the data mining process is used to identify and analyze the relationship
between variables because of the presence of the other factor. It is used to define the
probability of the specific variable. Regression, primarily a form of planning and modelling.
For example, we might use it to project certain costs, depending on other factors such as
availability, consumer demand, and competition. Primarily it gives the exact relationship
between two or more variables in the given data set.

4. Association Rules:
This data mining technique helps to discover a link between two or more items. It finds a
hidden pattern in the data set.

Association rules are if-then statements that support to show the probability of interactions
between data items within large data sets in different types of databases. Association rule
mining has several applications and is commonly used to help sales correlations in data or
medical data sets.

The way the algorithm works is that you have various data, For example, a list of grocery
items that you have been buying for the last six months. It calculates a percentage of items
being purchased together.

These are three major measurements technique:

 Lift: This measurement technique measures the accuracy of the confidence over how
often item B is purchased.
(Confidence) / (item B)/ (Entire dataset)
 Support: This measurement technique measures how often multiple items are
purchased and compared it to the overall dataset.
(Item A + Item B) / (Entire dataset)
 Confidence: This measurement technique measures how often item B is purchased
when item A is purchased as well.
(Item A + Item B)/ (Item A)

5. Outer detection:
This type of data mining technique relates to the observation of data items in the data set,
which do not match an expected pattern or expected behavior. This technique may be used
in various domains like intrusion, detection, fraud detection, etc. It is also known as Outlier
Analysis or Outilier mining. The outlier is a data point that diverges too much from the rest of
the dataset. The majority of the real-world datasets have an outlier. Outlier detection plays a
significant role in the data mining field. Outlier detection is valuable in numerous fields like
network interruption identification, credit or debit card fraud detection, detecting outlying in
wireless sensor network data, etc.

Kamadhenu BCA College 8

6. Sequential Patterns:
The sequential pattern is a data mining technique specialized for evaluating sequential
data to discover sequential patterns. It comprises of finding interesting subsequences in a
set of sequences, where the stake of a sequence can be measured in terms of different
criteria like length, occurrence frequency, etc.

In other words, this technique of data mining helps to discover or recognize similar patterns
in transaction data over some time.

7. Prediction:
Prediction used a combination of other data mining techniques such as trends, clustering,
classification, etc. It analyzes past events or instances in the right sequence to predict a
future event.

Data mining, despite its immense potential for extracting valuable insights from data, faces
various problems, issues, and challenges:

1. Data Quality: Poor quality data can significantly impact the results of data mining efforts.
Issues such as missing values, outliers, inaccuracies, and inconsistencies can distort
patterns and lead to erroneous conclusions. Cleaning and preprocessing the data to
ensure its quality is a critical challenge.

2. Scalability: With the exponential growth of data volumes, scalability becomes a significant
concern. Traditional data mining algorithms may struggle to handle large datasets
efficiently. Developing scalable algorithms and leveraging distributed computing
frameworks are essential to process big data effectively.

3. Complexity of Data: Modern datasets are often complex, with high dimensionality,
heterogeneity, and varying structures. Dealing with such diverse data types, including
structured, semi-structured, and unstructured data, poses challenges in data
representation, integration, and analysis.

4. Privacy and Security: Data mining raises privacy and security concerns, particularly when
dealing with sensitive or personal information. Protecting the privacy of individuals and
ensuring data security are paramount, requiring robust encryption, access controls, and
anonymization techniques.

5. Interpretability and Explainability: Complex data mining models, such as deep neural
networks, often lack interpretability, making it challenging to understand how they arrive
at their predictions or decisions. Ensuring model transparency and explainability is crucial,
especially in applications with legal, ethical, or regulatory implications.

6. Bias and Fairness: Data mining models can inherit biases present in the training data,
leading to unfair or discriminatory outcomes. Addressing bias and ensuring fairness in data
mining processes require careful consideration of the data collection process, feature
selection, and model evaluation.

7. Overfitting and Generalization: Overfitting occurs when a model learns to memorize the
training data instead of generalizing from it, leading to poor performance on unseen data.

Kamadhenu BCA College 9

 Balancing model complexity, regularization techniques, and proper evaluation methods are
essential to mitigate overfitting and improve generalization.

8. Feature Selection and Engineering: Identifying relevant features and engineering

meaningful representations from raw data is crucial for the success of data mining tasks.
Feature selection and engineering require domain knowledge, creativity, and careful
experimentation to extract useful information from the data.

9. Computational Complexity: Some data mining algorithms exhibit high computational

complexity, especially when dealing with large datasets or complex models. Optimizing
algorithm efficiency and leveraging parallel and distributed computing architectures are
essential to reduce computation time and resource usage.

10. Ethical and Regulatory Compliance: Data mining activities must comply with ethical
guidelines and legal regulations governing data usage, privacy, and consent. Ensuring
ethical conduct, obtaining informed consent, and respecting data ownership rights are
essential considerations in data mining projects.

Addressing these problems, issues, and challenges requires interdisciplinary expertise, including
data science, computer science, statistics, domain knowledge, and ethical considerations.
Collaboration between data scientists, domain experts, policymakers, and ethicists is essential
to develop responsible and effective data mining solutions.

Data mining (DM) finds application across various industries and domains, helping organizations
derive valuable insights from their data. Here are some common applications of data mining:

1. Business Intelligence: Data mining is extensively used in business intelligence to analyze

historical data, identify trends, and make data-driven decisions. It helps businesses
understand customer behavior, optimize marketing strategies, improve sales forecasting,
and enhance overall operational efficiency.

2. Customer Relationship Management (CRM): Data mining plays a crucial role in CRM by
segmenting customers based on their preferences, purchase history, and behavior. It
enables personalized marketing campaigns, customer retention strategies, and targeted
product recommendations, ultimately leading to improved customer satisfaction and
loyalty.

3. Fraud Detection and Prevention: Data mining techniques are employed in fraud detection
across various industries, including finance, insurance, and e-commerce. By analyzing
transactional data and detecting anomalies or suspicious patterns, organizations can
identify fraudulent activities and take proactive measures to prevent financial losses.

4. Healthcare Analytics: In healthcare, data mining helps analyze electronic health records
(EHRs), medical imaging data, and clinical data to improve patient care, optimize
treatment plans, and predict disease outcomes. It facilitates medical diagnosis, drug
discovery, patient monitoring, and healthcare resource allocation.

5. Market Basket Analysis: Market basket analysis is a data mining technique used in retail
and e-commerce to understand the associations between products purchased by

Kamadhenu BCA College 10

 customers. By analyzing transactional data, organizations can identify product affinities,
optimize product placement, and design effective cross-selling and upselling strategies.

6. Supply Chain Management: Data mining aids supply chain management by analyzing
supply chain data, demand forecasts, inventory levels, and logistics information. It helps
optimize inventory management, streamline logistics operations, identify supply chain
risks, and improve overall supply chain efficiency.

7. Predictive Maintenance: Data mining enables predictive maintenance by analyzing

equipment sensor data, maintenance logs, and historical performance data. By identifying
patterns indicative of equipment failure or deterioration, organizations can schedule
preventive maintenance activities, minimize downtime, and reduce maintenance costs.

8. Social Media Analysis: Data mining techniques are applied in social media analytics to
extract insights from social media platforms such as Twitter, Facebook, and Instagram.
Organizations use sentiment analysis, topic modeling, and social network analysis to
understand customer sentiments, track brand perception, and identify influencers.

9. Text Mining and Natural Language Processing (NLP): Text mining and NLP techniques
analyze unstructured textual data from sources such as emails, customer reviews, and
documents. It facilitates tasks such as document categorization, sentiment analysis,
information extraction, and entity recognition, enabling organizations to derive actionable
insights from text data.

10. Environmental Monitoring: Data mining is used in environmental monitoring to analyze

data from sensors, satellites, weather stations, and environmental databases. It helps
monitor air quality, water quality, climate patterns, and ecological changes, supporting
environmental conservation efforts and decision-making.

These are just a few examples of the diverse applications of data mining across industries. As data
continues to grow in volume and complexity, the importance of data mining in extracting
meaningful insights and driving informed decision-making will continue to increase.

Data Warehouse
Data Warehouse is a relational database management
system (RDBMS) construct to meet the requirement of
transaction processing systems. It can be loosely
described as any centralized data repository which can be
queried for business benefits. It is a database that stores
information oriented to satisfy decision-making requests.
It is a group of decision support technologies, targets to
enabling the knowledge worker (executive, manager, and
analyst) to make superior and higher decisions. So, Data
Warehousing support architectures and tool for business
executives to systematically organize, understand and
use their information to make strategic decisions.
Kamadhenu BCA College 11
Data Warehouse environment contains an extraction, transportation, and loading (ETL)
solution, an online analytical processing (OLAP) engine, customer analysis tools, and other
applications that handle the process of gathering information and delivering it to business
users.

A Data Warehouse (DW) is a relational database that is designed for query and analysis
rather than transaction processing. It includes historical data derived from transaction data
from single and multiple sources.

A Data Warehouse provides integrated, enterprise-wide, historical data and focuses on

providing support for decision-makers for data modeling and analysis.

A Data Warehouse is a group of data specific to the entire organization, not only to a particular
group of users.It is not used for daily operations and transaction processing but used for
making decisions.

A Data Warehouse can be viewed as a data system with the following attributes:

 It is a database designed for investigative tasks, using data from various applications.
 It supports a relatively small number of clients with relatively long interactions.
 It includes current and historical data to provide a historical perspective of information.
 Its usage is read-intensive.
 It contains a few large tables.

"Data Warehouse is a subject-oriented, integrated, and time-variant store of information in

support of management's decisions."

Characteristics of Data Warehouse

Subject-Oriented

A data warehouse target on the modeling and analysis of data for decision-makers. Therefore,
data warehouses typically provide a concise and straightforward view around a particular
subject, such as customer, product, or sales, instead of the global organization's ongoing

Kamadhenu BCA College 12

operations. This is done by excluding data that are not useful concerning the subject and
including all data needed by the users to understand the subject.

Integrated

A data warehouse integrates various heterogeneous data sources like RDBMS, flat files, and
online transaction records. It requires performing data cleaning and integration during data
warehousing to ensure consistency in naming conventions, attributes types, etc., among
different data sources.

Time-Variant
Historical information is kept in a data warehouse. For example, one can retrieve files from 3
months, 6 months, 12 months, or even previous data from a data warehouse. These variations
with a transactions system, where often only the most current file is kept.

Kamadhenu BCA College 13

Non-Volatile

The data warehouse is a physically separate data storage, which is transformed from the
source operational RDBMS. The operational updates of data do not occur in the data
warehouse, i.e., update, insert, and delete operations are not performed. It usually requires
only two procedures in data accessing: Initial loading of data and access to data. Therefore, the
DW does not require transaction processing, recovery, and concurrency capabilities, which
allows for substantial speedup of data retrieval. Non-Volatile defines that once entered into
the warehouse, and data should not change.

History of Data Warehouse

The idea of data warehousing came to the late 1980's when IBM researchers Barry Devlin and
Paul Murphy established the "Business Data Warehouse."

In essence, the data warehousing idea was planned to support an architectural model for the
flow of information from the operational system to decisional support environments. The
concept attempt to address the various problems associated with the flow, mainly the high
costs associated with it.

In the absence of data warehousing architecture, a vast amount of space was required to
support multiple decision support environments. In large corporations, it was ordinary for
various decision support environments to operate independently.

Goals of Data Warehousing

 To help reporting as well as analysis
 Maintain the organization's historical information
 Be the foundation for decision making.

Need for Data Warehouse

Data Warehouse is needed for the following reasons:

Kamadhenu BCA College 14

1. Business User: Business users require a data warehouse to view summarized data from
the past. Since these people are non-technical, the data may be presented to them in an
elementary form.

2. Store historical data: Data Warehouse is required to store the time variable data from the
past. This input is made to be used for various purposes.

3. Make strategic decisions: Some strategies may be depending upon the data in the data
warehouse. So, data warehouse contributes to making strategic decisions.

4. For data consistency and quality: Bringing the data from different sources at a
commonplace, the user can effectively undertake to bring the uniformity and consistency
in data.

5. High response time: Data warehouse has to be ready for somewhat unexpected loads and
types of queries, which demands a significant degree of flexibility and quick response time.

Benefits of Data Warehouse

1. Understand business trends and make better forecasting decisions.
2. Data Warehouses are designed to perform well enormous amounts of data.
3. The structure of data warehouses is more accessible for end-users to navigate,
understand, and query.
4. Queries that would be complex in many normalized databases could be easier to build
and maintain in data warehouses.
5. Data warehousing is an efficient method to manage demand for lots of information from
lots of users.
6. Data warehousing provide the capabilities to analyze a large amount of historical data.

Multi Dimensional Data Model

The multi-Dimensional Data Model is a method which is used for ordering data in the
database along with good arrangement and assembling of the contents in the database.
The Multi Dimensional Data Model allows customers to interrogate analytical questions
associated with market or business trends, unlike relational databases which allow customers
to access data in the form of queries. They allow users to rapidly receive answers to the
requests which they made by creating and examining the data comparatively fast.
OLAP (online analytical processing) and data warehousing uses multi dimensional databases.
It is used to show multiple dimensions of the data to users.

Kamadhenu BCA College 15

It represents data in the form of data cubes. Data cubes allow to model and view the data
from many dimensions and perspectives. It is defined by dimensions and facts and is
represented by a fact table. Facts are numerical measures and fact tables contain measures
of the related dimensional tables or names of the facts.

Multidimensional Data Representation

Working on a Multidimensional Data Model
On the basis of the pre-decided steps, the Multidimensional Data Model works.

The following stages should be followed by every project for building a Multi Dimensional
Data Model :

Stage 1 :
Assembling data from the client : In first stage, a Multi Dimensional Data Model collects
correct data from the client. Mostly, software professionals provide simplicity to the client
about the range of data which can be gained with the selected technology and collect the
complete data in detail.

Stage 2 :
Grouping different segments of the system : In the second stage, the Multi Dimensional
Data Model recognizes and classifies all the data to the respective section they belong to and
also builds it problem-free to apply step by step.

Stage 3 :
Noticing the different proportions : In the third stage, it is the basis on which the design of
the system is based. In this stage, the main factors are recognized according to the user’s
point of view. These factors are also known as “Dimensions”.

Stage 4 :
Preparing the actual-time factors and their respective qualities : In the fourth stage, the
factors which are recognized in the previous step are used further for identifying the related
qualities. These qualities are also known as “attributes” in the database.

Kamadhenu BCA College 16

Stage 5 :
Finding the actuality of factors which are listed previously and their qualities : In the fifth
stage, A Multi Dimensional Data Model separates and differentiates the actuality from the
factors which are collected by it. These actually play a significant role in the arrangement of a
Multi Dimensional Data Model.

Stage 6 :
Building the Schema to place the data, with respect to the information collected from the
steps above : In the sixth stage, on the basis of the data which was collected previously, a
Schema is built.

For Example :
1. Let us take the example of a firm. The revenue cost of a firm can be recognized on the
basis of different factors such as geographical location of firm’s workplace, products of
the firm, advertisements done, time utilized to flourish a product, etc.

Example 1
2. Let us take the example of the data of a factory which sells products per quarter in
Bangalore. The data is represented in the table given below :

2D factory data
In the above given presentation, the factory’s sales for Bangalore are, for the time dimension,
which is organized into quarters and the dimension of items, which is sorted according to the
kind of item which is sold. The facts here are represented in rupees (in thousands).
Now, if we desire to view the data of the sales in a three-dimensional table, then it is
represented in the diagram given below. Here the data of the sales is represented as a

Kamadhenu BCA College 17

two dimensional table. Let us consider the data according to item, time and location (like
Kolkata, Delhi, Mumbai). Here is the table :

3D data representation as 2D
This data can be represented in the form of three dimensions conceptually, which is shown in
the image below :

3D data representation

Features of multidimensional data models:

1. Measures: Measures are numerical data that can be analyzed and compared, such as
sales or revenue. They are typically stored in fact tables in a multidimensional data model.

2. Dimensions: Dimensions are attributes that describe the measures, such as time,
location, or product. They are typically stored in dimension tables in a multidimensional
data model

3. Cubes: Cubes are structures that represent the multidimensional relationships between
measures and dimensions in a data model. They provide a fast and efficient way to
retrieve and analyze data.

4. Aggregation: Aggregation is the process of summarizing data across dimensions and

levels of detail. This is a key feature of multidimensional data models, as it enables users
to quickly analyze data at different levels of granularity.

5. Drill-down and roll-up: Drill-down is the process of moving from a higher-level

summary of data to a lower level of detail, while roll-up is the opposite process of moving
from a lower-level detail to a higher-level summary. These features enable users to
explore data in greater detail and gain insights into the underlying patterns.
Kamadhenu BCA College 18
 6. Hierarchies: Hierarchies are a way of organizing dimensions into levels of detail. For
example, a time dimension might be organized into years, quarters, months, and days.
Hierarchies provide a way to navigate the data and perform drill-down and roll-up
operations.

7. OLAP (Online Analytical Processing): OLAP is a type of multidimensional data

model that supports fast and efficient querying of large datasets. OLAP systems are
designed to handle complex queries and provide fast response times.

Advantages of Multi Dimensional Data Model

The following are the advantages of a multi-dimensional data model :
 A multi-dimensional data model is easy to handle.
 It is easy to maintain.
 Its performance is better than that of normal databases (e.g. relational databases).
 The representation of data is better than traditional databases. That is because the multi-
dimensional databases are multi-viewed and carry different types of factors.
 It is workable on complex systems and applications, contrary to the simple one-
dimensional database systems.
 The compatibility in this type of database is an upliftment for projects having lower
bandwidth for maintenance staff.

Disadvantages of Multi Dimensional Data Model

The following are the disadvantages of a Multi Dimensional Data Model :
 The multi-dimensional Data Model is slightly complicated in nature and it requires
professionals to recognize and examine the data in the database.
 During the work of a Multi-Dimensional Data Model, when the system caches, there is a
great effect on the working of the system.
 It is complicated in nature due to which the databases are generally dynamic in design.
 The path to achieving the end product is complicated most of the time.
 As the Multi Dimensional Data Model has complicated systems, databases have a large
number of databases due to which the system is very insecure when there is a security
break.

Data Cleaning
Data cleaning is a crucial aspect of data warehousing, ensuring that the data stored in the data
warehouse is accurate, consistent, and reliable. Here's how data cleaning is typically performed
in the context of data warehousing:

1. Identifying Data Quality Issues: The first step in data cleaning is to identify data quality
issues within the data warehouse. Common issues include missing values, incorrect data
types, duplicates, inconsistencies, and outliers.
2. Data Profiling: Data profiling involves analyzing the structure, content, and quality of data
within the data warehouse. It helps identify patterns, anomalies, and potential data quality
problems that need to be addressed during the cleaning process.

3. Data Standardization: Data standardization involves ensuring consistency in data formats,

units of measurement, and naming conventions across different datasets within the data

Kamadhenu BCA College 19

 warehouse. Standardizing data improves data quality and facilitates data integration and
analysis.

4. Handling Missing Values: Missing values are common in datasets and can adversely affect
data analysis and decision-making. Data cleaning techniques such as imputation (replacing
missing values with estimated values based on statistical methods) or deletion (removing
records with missing values) are used to handle missing data.

5. Removing Duplicates: Duplicate records can skew analysis results and waste storage space
in the data warehouse. Data cleaning involves identifying and removing duplicate records
based on key attributes or combinations of attributes.

6. Correcting Errors: Data cleaning may involve identifying and correcting errors in the data,
such as typos, inconsistencies, and inaccuracies. Techniques such as data validation rules,
referential integrity checks, and data cleansing tools are used to identify and rectify errors.

7. Data Transformation: Data transformation involves converting data from one format or
structure to another to ensure compatibility and consistency within the data warehouse.
This may include data normalization, aggregation, or restructuring to meet specific
requirements for analysis and reporting.

8. Addressing Inconsistencies: Data cleaning addresses inconsistencies in data values, formats,

and representations. This may involve resolving discrepancies between different datasets,
aligning data with defined business rules, and ensuring data integrity across the data
warehouse.

9. Automating Data Cleaning Processes: Data cleaning processes can be automated using
tools and software that streamline data profiling, validation, cleansing, and transformation
tasks. Automation reduces manual effort, accelerates data cleaning cycles, and improves
overall data quality.

10. Monitoring and Maintenance: Data cleaning is an ongoing process that requires regular
monitoring and maintenance to ensure data quality over time. Continuous monitoring of
data quality metrics, proactive identification of issues, and timely intervention are essential
for maintaining high-quality data in the data warehouse.

Data Integration
Data integration is the process of merging data from several disparate sources. While performing
data integration, you must work on data redundancy, inconsistency, duplicity, etc. In data
mining, data integration is a record pre-processing method that includes merging data from a
couple of the heterogeneous data sources into coherent data to retain and provide a unified
perspective of the data. These assets could also include several record cubes, databases, or flat
documents. The statistical integration strategy is formally stated as a triple (G, S, M) approach.
G represents the global schema, S represents the heterogeneous source of schema, and M
represents the mapping between source and global schema queries.

It has been an integral part of data operations because data can be obtained from several sources.
It is a strategy that integrates data from several sources to make it available to users in a
single uniform view that shows their status. There are communication sources between
Kamadhenu BCA College 20
systems that can include multiple databases, data cubes, or flat files. Data fusion merges data
from various diverse sources to produce meaningful results. The consolidated findings must
exclude inconsistencies, contradictions, redundancies, and inequities.

Data integration is important because it gives a uniform view of scattered data while also
maintaining data accuracy. It assists the data-mining program in meaningful mining
information, which in turn assists the executive and managers make strategic decisions for the
enterprise's benefit.

The data integration methods are formally characterized as a triple (G, S, M), where;

G represents the global schema,

S represents the heterogeneous source of schema,
M represents the mapping between source and global schema queries.

Data Integration Approaches

There are mainly two types of approaches for data integration. These are as follows:

Tight Coupling : It is the process of using ETL (Extraction, Transformation, and Loading) to
combine data from various sources into a single physical location.

Loose Coupling: Facts with loose coupling are most effectively kept in the actual source
databases. This approach provides an interface that gets a query from the user, changes it into
a format that the supply database may understand, and then sends the query to the source
databases without delay to obtain the result.

Issues in Data Integration :

When you integrate the data in Data Mining, you may face many issues.
There are some of those issues:

1. Entity Identification Problem

As you understand, the records are obtained from heterogeneous sources, and how can you
'match the real-world entities from the data'. For example, you were given client data from
specialized statistics sites. Customer identity is assigned to an entity from one statistics supply,
while a customer range is assigned to an entity from another statistics supply. Analyzing such
metadata statistics will prevent you from making errors during schema integration.

Structural integration is completed by guaranteeing that the functional dependency and

referential constraints of a character in the source machine match the functional dependency
and referential constraints of the identical character in the target machine. For example,
assume that the discount is applied to the entire order in one machine, but in every other
machine, the discount is applied to each item in the order. This distinction should be noted
before the information from those assets is included in the goal system.

2. Redundancy and Correlation Analysis

One of the major issues in the course of data integration is redundancy. Unimportant data that
are no longer required are referred to as redundant data. It may also appear due to attributes
created from the use of another property inside the information set. For example, if one truth
set contains the patronage and distinct data set as the purchaser's date of the beginning, then

Kamadhenu BCA College 21

age may be a redundant attribute because it can be deduced from the use of the beginning
date.

Inconsistencies further increase the level of redundancy within the characteristic. The use of
correlation analysis can be used to determine redundancy. The traits are examined to
determine their interdependence on each difference, consequently discovering the link
between them.

3. Tuple Duplication
Information integration has also handled duplicate tuples in addition to redundancy. Duplicate
tuples may also appear in the generated information if the denormalized table was utilized as a
deliverable for data integration.

4. Data warfare Detection and backbone

The data warfare technique of combining records from several sources is unhealthy. In the
same way, that characteristic values can vary, so can statistics units. The disparity may be
related to the fact that they are represented differently within the special data units. For
example, in one-of-a-kind towns, the price of an inn room might be expressed in a particular
currency. This type of issue is recognized and fixed during the data integration process.

Data Integration Techniques

There are various data integration techniques in data mining. Some of them are
as follows:

1. Manual Integration
This method avoids using automation during data integration. The data analyst collects,
cleans, and integrates the data to produce meaningful information. This strategy is suitable
for a mini organization with a limited data set. Although, it will be time-consuming for the
huge, sophisticated, and recurring integration. Because the entire process must be done
manually, it is a time-consuming operation.

2. Middleware Integration
The middleware software is used to take data from many sources, normalize it, and store it
in the resulting data set. When an enterprise needs to integrate data from legacy systems
to modern systems, this technique is used. Middleware software acts as a translator
between legacy and advanced systems. You may take an adapter that allows two systems
with different interfaces to be connected. It is only applicable to certain systems.

3. Application-based integration
It is using software applications to extract, transform, and load data from disparate sources.
This strategy saves time and effort, but it is a little more complicated because building such
an application necessitates technical understanding. This strategy saves time and effort,
but it is a little more complicated because building such an application necessitates
technical understanding.

4. Uniform Access Integration

This method combines data from a more disparate source. However, the data's position is
not altered in this scenario; the data stays in its original location. This technique merely

Kamadhenu BCA College 22

 generates a unified view of the integrated data. The integrated data does not need to be
stored separately because the end-user only sees the integrated view.

5. Data Warehousing
This technique is related to the uniform access integration technique in a roundabout way.
The unified view, on the other hand, is stored in a different location. It enables the data
analyst to deal with more sophisticated inquiries. Although it is a promising solution and
increased storage costs, the unified data's view or copy requires separate storage and
maintenance costs.

Integration tools
There are various integration tools in data mining. Some of them are as follows:

On-promise data integration tool

An on-premise data integration tool integrates data from local sources and connects legacy
databases using middleware software.

Open-source data integration tool

If you want to avoid pricey enterprise solutions, an open-source data integration tool is the
ideal alternative. Although, you will be responsible for the security and privacy of the data if
you're using the tool.

Cloud-based data integration tool

A cloud-based data integration tool may provide an 'integration platform as a service'.

Data Transformation in Data Mining

Raw data is difficult to trace or understand. That's why it needs to be preprocessed before
retrieving any information from it. Data transformation is a technique used to convert the
raw data into a suitable format that efficiently eases data mining and retrieves strategic
information. Data transformation includes data cleaning techniques and a data reduction
technique to convert the data into the appropriate form.

Data transformation is an essential data preprocessing technique that must be performed on

the data before data mining to provide patterns that are easier to understand.

Data transformation changes the format, structure, or values of the data and converts them
into clean, usable data. Data may be transformed at two stages of the data pipeline for data
analytics projects. Organizations that use on-premises data warehouses generally use an ETL
(extract, transform, and load) process, in which data transformation is the middle step. Today,
most organizations use cloud-based data warehouses to scale compute and storage resources
with latency measured in seconds or minutes. The scalability of the cloud platform lets
organizations skip preload transformations and load raw data into the data warehouse, then
transform it at query time.

Data integration, migration, data warehousing, data wrangling may all involve data
transformation. Data transformation increases the efficiency of business and analytic processes,
and it enables businesses to make better data-driven decisions. During the data transformation

Kamadhenu BCA College 23

process, an analyst will determine the structure of the data. This could mean that data
transformation may be:

o Constructive: The data transformation process adds, copies, or replicates data.

o Destructive: The system deletes fields or records.
o Aesthetic: The transformation standardizes the data to meet requirements or parameters.
o Structural: The database is reorganized by renaming, moving, or combining columns.

Data Transformation Techniques

There are several data transformation techniques that can help structure and clean up the data
before analysis or storage in a data warehouse. Let's study all techniques used for data
transformation, some of which we have already studied in data reduction and data cleaning.

1. Data Smoothing
Data smoothing is a process that is used to remove noise from the dataset using some
algorithms. It allows for highlighting important features present in the dataset. It helps in
predicting the patterns. When collecting data, it can be manipulated to eliminate or reduce
any variance or any other noise form.

The concept behind data smoothing is that it will be able to identify simple changes to help
predict different trends and patterns. This serves as a help to analysts or traders who need
to look at a lot of data which can often be difficult to digest for finding patterns that they
wouldn't see otherwise.

We have seen how the noise is removed from the data using the techniques such as
binning, regression, clustering.

o Binning: This method splits the sorted data into the number of bins and smoothens the
data values in each bin considering the neighborhood values around it.

o Regression: This method identifies the relation among two dependent attributes so
that if we have one attribute, it can be used to predict the other attribute.

o Clustering: This method groups similar data values and form a cluster. The values that
lie outside a cluster are known as outliers.

Kamadhenu BCA College 24

2. Attribute Construction
In the attribute construction method, the new attributes consult the existing attributes to
construct a new data set that eases data mining. New attributes are created and applied to
assist the mining process from the given attributes. This simplifies the original data and
makes the mining more efficient.

For example, suppose we have a data set referring to measurements of different plots, i.e., we
may have the height and width of each plot. So here, we can construct a new attribute
'area' from attributes 'height' and 'weight'. This also helps understand the relations among
the attributes in a data set.

3. Data Aggregation
Data collection or aggregation is the method of storing and presenting data in a summary
format. The data may be obtained from multiple data sources to integrate these data
sources into a data analysis description. This is a crucial step since the accuracy of data
analysis insights is highly dependent on the quantity and quality of the data used.

Gathering accurate data of high quality and a large enough quantity is necessary to produce
relevant results. The collection of data is useful for everything from decisions concerning
financing or business strategy of the product, pricing, operations, and marketing strategies.

For example, we have a data set of sales reports of an enterprise that has quarterly sales of
each year. We can aggregate the data to get the enterprise's annual sales report.

4. Data Normalization

Normalizing the data refers to scaling the data values to a much smaller range such as [-1, 1] or
[0.0, 1.0]. There are different methods to normalize the data, as discussed below.

Consider that we have a numeric attribute A and we have n number of observed values for
attribute A that are V1, V2, V3, ….Vn.
o Min-max normalization: This method implements a linear transformation on the
original data. Let us consider that we have minA and maxA as the minimum and
maximum value observed for attribute A and Viis the value for attribute A that has to be
normalized. The min-max normalization would map V i to the V'i in a new smaller range
[new_minA, new_maxA].
The formula for min-max normalization is given below:

Kamadhenu BCA College 25

 For example, we have $1200 and $9800 as the minimum, and maximum value for the
attribute income, and [0.0, 1.0] is the range in which we have to map a value of $73,600.
The value $73,600 would be transformed using min-max normalization as follows:

o Z-score normalization: This method normalizes the value for attribute A using the mean
and standard deviation. The following formula is used for Z-score normalization:

Here Ᾱ and σA are the mean and standard deviation for attribute A, respectively.
For example, we have a mean and standard deviation for attribute A as $54,000 and
$16,000. And we have to normalize the value $73,600 using z-score normalization.

o Decimal Scaling: This method normalizes the value of attribute A by moving the decimal
point in the value. This movement of a decimal point depends on the maximum
absolute value of A. The formula for the decimal scaling is given below:

Here j is the smallest integer such that max(|v'i|)<1

For example, the observed values for attribute A range from -986 to 917, and the
maximum absolute value for attribute A is 986. Here, to normalize each value of
attribute A using decimal scaling, we have to divide each value of attribute A by 1000,
i.e., j=3.
So, the value -986 would be normalized to -0.986, and 917 would be normalized to
0.917.
The normalization parameters such as mean, standard deviation, the maximum
absolute value must be preserved to normalize the future data uniformly.

5. Data Discretization
This is a process of converting continuous data into a set of data intervals. Continuous
attribute values are substituted by small interval labels. This makes the data easier to
study and analyze. If a data mining task handles a continuous attribute, then its discrete
values can be replaced by constant quality attributes. This improves the efficiency of the
task.

This method is also called a data reduction mechanism as it transforms a large dataset into
a set of categorical data. Discretization also uses decision tree-based algorithms to
produce short, compact, and accurate results when using discrete values.

Data discretization can be classified into two types: supervised discretization, where the
class information is used, and unsupervised discretization, which is based on which
direction the process proceeds, i.e., 'top-down splitting strategy' or 'bottom-up merging
strategy'.
Kamadhenu BCA College 26
 For example, the values for the age attribute can be replaced by the interval labels such as
(0-10, 11-20…) or (kid, youth, adult, senior).

6. Data Generalization
It converts low-level data attributes to high-level data attributes using concept hierarchy.
This conversion from a lower level to a higher conceptual level is useful to get a clearer
picture of the data. Data generalization can be divided into two approaches:
o Data cube process (OLAP) approach.
o Attribute-oriented induction (AOI) approach.

For example, age data can be in the form of (20, 30) in a dataset. It is transformed into a
higher conceptual level into a categorical value (young, old).

Data Transformation Process

The entire process for transforming data is known as ETL (Extract, Load, and Transform).
Through the ETL process, analysts can convert data to its desired format. Here are the steps
involved in the data transformation process:

1. Data Discovery: During the first stage, analysts work to understand and identify data in
its source format. To do this, they will use data profiling tools. This step helps analysts
decide what they need to do to get data into its desired format.
2. Data Mapping: During this phase, analysts perform data mapping to determine how
individual fields are modified, mapped, filtered, joined, and aggregated. Data mapping is
essential to many data processes, and one misstep can lead to incorrect analysis and ripple
through your entire organization.
3. Data Extraction: During this phase, analysts extract the data from its original source.
These may include structured sources such as databases or streaming sources such as
customer log files from web applications.
4. Code Generation and Execution: Once the data has been extracted, analysts need to
create a code to complete the transformation. Often, analysts generate codes with the help
of data transformation platforms or tools.

Kamadhenu BCA College 27

 5. Review: After transforming the data, analysts need to check it to ensure everything has
been formatted correctly.
6. Sending: The final step involves sending the data to its target destination. The target
might be a data warehouse or a database that handles both structured and unstructured
data.

Advantages of Data Transformation

Transforming data can help businesses in a variety of ways. Here are some of the essential
advantages of data transformation, such as:

1. Better Organization: Transformed data is easier for both humans and computers to use.
2. Improved Data Quality: There are many risks and costs associated with bad data. Data
transformation can help your organization eliminate quality issues such as missing values
and other inconsistencies.
3. Perform Faster Queries: You can quickly and easily retrieve transformed data thanks to it
being stored and standardized in a source location.
4. Better Data Management: Businesses are constantly generating data from more and more
sources. If there are inconsistencies in the metadata, it can be challenging to organize and
understand it. Data transformation refines your metadata, so it's easier to organize and
understand.
5. More Use Out of Data: While businesses may be collecting data constantly, a lot of that
data sits around unanalyzed. Transformation makes it easier to get the most out of your
data by standardizing it and making it more usable.

Disadvantages of Data Transformation

While data transformation comes with a lot of benefits, still there are some challenges to
transforming data effectively, such as:

1. Data transformation can be expensive. The cost is dependent on the specific infrastructure,
software, and tools used to process data. Expenses may include licensing, computing
resources, and hiring necessary personnel.
2. Data transformation processes can be resource-intensive. Performing transformations in an
on-premises data warehouse after loading or transforming data before feeding it into
applications can create a computational burden that slows down other operations. If you use

Kamadhenu BCA College 28

 a cloud-based data warehouse, you can do the transformations after loading because the
platform can scale up to meet demand.
3. Lack of expertise and carelessness can introduce problems during transformation. Data
analysts without appropriate subject matter expertise are less likely to notice incorrect data
because they are less familiar with the range of accurate and permissible values.
4. Enterprises can perform transformations that don't suit their needs. A business might
change information to a specific format for one application only to then revert the
information to its prior format for a different application.

Ways of Data Transformation

There are several different ways to transform data, such as:

1. Scripting: Data transformation through scripting involves Python or SQL to write the code
to extract and transform data. Python and SQL are scripting languages that allow you to
automate certain tasks in a program. They also allow you to extract information from data
sets. Scripting languages require less code than traditional programming languages.
Therefore, it is less intensive.
2. On-Premises ETL Tools: ETL tools take the required work to script the data
transformation by automating the process. On-premises ETL tools are hosted on company
servers. While these tools can help save you time, using them often requires extensive
expertise and significant infrastructure costs.
3. Cloud-Based ETL Tools: As the name suggests, cloud-based ETL tools are hosted in the
cloud. These tools are often the easiest for non-technical users to utilize. They allow you to
collect data from any cloud source and load it into your data warehouse. With cloud-based
ETL tools, you can decide how often you want to pull data from your source, and you can
monitor your usage.

Data Reduction in Data Mining

Data mining is applied to the selected data in a large amount database. When data analysis and
mining is done on a huge amount of data, then it takes a very long time to process, making it
impractical and infeasible.

Data reduction techniques ensure the integrity of data while reducing the data. Data reduction is a
process that reduces the volume of original data and represents it in a much smaller volume.
Data reduction techniques are used to obtain a reduced representation of the dataset that is
much smaller in volume by maintaining the integrity of the original data.
Kamadhenu BCA College 29
 By reducing the data, the efficiency of the data mining process is improved, which produces
the same analytical results.

Data reduction does not affect the result obtained from data mining. That means the result
obtained from data mining before and after data reduction is the same or almost the
same.Data reduction aims to define it more compactly. When the data size is smaller, it is
simpler to apply sophisticated and computationally high-priced algorithms. The reduction of
the data may be in terms of the number of rows (records) or terms of the number of columns
(dimensions).

Techniques of Data Reduction

Here are the following techniques or methods of data reduction in data mining, such as:

1. Dimensionality Reduction
Whenever we encounter weakly important data, we use the attribute required for our analysis.
Dimensionality reduction eliminates the attributes from the data set under consideration,
thereby reducing the volume of original data. It reduces data size as it eliminates outdated or
redundant features. Here are three methods of dimensionality reduction.

i. Wavelet Transform: In the wavelet transform, suppose a data vector A is transformed

into a numerically different data vector A' such that both A and A' vectors are of the
same length. Then how it is useful in reducing data because the data obtained from the
wavelet transform can be truncated. The compressed data is obtained by retaining the
smallest fragment of the strongest wavelet coefficients. Wavelet transform can be
applied to data cubes, sparse data, or skewed data.

ii. Principal Component Analysis: Suppose we have a data set to be analyzed that has
tuples with n attributes. The principal component analysis identifies k independent
tuples with n attributes that can represent the data set. In this way, the original data
can be cast on a much smaller space, and dimensionality reduction can be achieved.
Principal component analysis can be applied to sparse and skewed data.

iii. Attribute Subset Selection: The large data set has many attributes, some of which are
irrelevant to data mining or some are redundant. The core attribute subset selection
reduces the data volume and dimensionality. The attribute subset selection reduces the
volume of data by eliminating redundant and irrelevant attributes. The attribute subset
selection ensures that we get a good subset of original attributes even after eliminating
the unwanted attributes. The resulting probability of data distribution is as close as
possible to the original data distribution using all the attributes.

Kamadhenu BCA College 30

 2. Numerosity Reduction
The numerosity reduction reduces the original data volume and represents it in a much smaller form.
This technique includes two types parametric and non-parametric numerosity reduction.

i. Parametric: Parametric numerosity reduction incorporates storing only data

parameters instead of the original data. One method of parametric numerosity
reduction is the regression and log-linear method.
 Regression and Log-Linear: Linear regression models a relationship between the
two attributes by modeling a linear equation to the data set. Suppose we need
to model a linear function between two attributes. y = wx +b Here, y is the
response attribute, and x is the predictor attribute. If we discuss in terms of data
mining, attribute x and attribute y are the numeric database attributes, whereas
w and b are regression coefficients.
Multiple linear regressions let the response variable y model linear function
between two or more predictor variables. Log-linear model discovers the
relation between two or more discrete attributes in the database. Suppose we
have a set of tuples presented in n dimensional space. Then the log-linear model
is used to study the probability of each tuple in a multidimensional space.
Regression and log-linear methods can be used for sparse data and skewed data.
ii. Non-Parametric: A non-parametric numerosity reduction technique does not
assume any model. The non-Parametric technique results in a more uniform reduction,
irrespective of data size, but it may not achieve a high volume of data reduction like the
parametric. There are at least four types of Non-Parametric data reduction techniques,
Histogram, Clustering, Sampling, Data Cube Aggregation, and Data Compression.

 Histogram: A histogram is a graph that represents frequency distribution which

describes how often a value appears in the data. Histogram uses the binning
method to represent an attribute's data distribution. It uses a disjoint subset
which we call bin or buckets.
A histogram can represent a dense, sparse, uniform, or skewed data. Instead of only
one attribute, the histogram can be implemented for multiple attributes. It can
effectively represent up to five attributes.
 Clustering: Clustering techniques groups similar objects from the data so that the
objects in a cluster are similar to each other, but they are dissimilar to objects in
another cluster.
How much similar are the objects inside a cluster can be calculated using a distance
function. More is the similarity between the objects in a cluster closer they appear
in the cluster.
The quality of the cluster depends on the diameter of the cluster, i.e., the max
distance between any two objects in the cluster. The cluster representation
replaces the original data. This technique is more effective if the present data can
be classified into a distinct clustered.
 Sampling: One of the methods used for data reduction is sampling, as it can
reduce the large data set into a much smaller data sample. Below we will discuss
the different methods in which we can sample a large data set D containing N
tuples:

1. Simple random sample without replacement (SRSWOR) of size s: In this

s, some tuples are drawn from N tuples such that in the data set D (s<N).

Kamadhenu BCA College 31

 The probability of drawing any tuple from the data set D is 1/N. This means
all tuples have an equal probability of getting sampled.
2. Simple random sample with replacement (SRSWR) of size s: It is similar
to the SRSWOR, but the tuple is drawn from data set D, is recorded, and
then replaced into the data set D so that it can be drawn again.

3. Cluster sample: The tuples in data set D are clustered into M mutually
disjoint subsets. The data reduction can be applied by implementing
SRSWOR on these clusters. A simple random sample of size s could be
generated from these clusters where s<M.
4. Stratified sample: The large data set D is partitioned into mutually
disjoint sets called 'strata'. A simple random sample is taken from each
stratum to get stratified data. This method is effective for skewed data.

3. Data Cube Aggregation

This technique is used to aggregate data in a simpler

form. Data Cube Aggregation is a multidimensional
aggregation that uses aggregation at various levels
of a data cube to represent the original data set,
thus achieving data reduction.

For example, suppose you have the data of All

Electronics sales per quarter for the year 2018 to
the year 2022. If you want to get the annual sale per
year, you just have to aggregate the sales per
quarter for each year. In this way, aggregation provides you with the required data, which is much
smaller in size, and thereby we achieve data reduction even without losing any data.

The data cube aggregation is a multidimensional

aggregation that eases multidimensional
analysis. The data cube present precomputed
and summarized data which eases the data
mining into fast access.

4. Data Compression

Data compression employs modification, encoding,

or converting the structure of data in

Kamadhenu BCA College 32

a way that consumes less space. Data compression involves building a compact representation
of information by removing redundancy and representing data in binary form. Data that can be
restored successfully from its compressed form is called Lossless compression. In contrast, the
opposite where it is not possible to restore the original form from the compressed form is
Lossy compression. Dimensionality and numerosity reduction method are also used for data
compression.

This technique reduces the size of the files using different encoding mechanisms, such as
Huffman Encoding and run-length Encoding. We can divide it into two types based on their
compression techniques.

i. Lossless Compression: Encoding techniques (Run Length Encoding) allow a simple and
minimal data size reduction. Lossless data compression uses algorithms to restore the
precise original data from the compressed data.
ii. Lossy Compression: In lossy-data compression, the decompressed data may differ from
the original data but are useful enough to retrieve information from them. For example,
the JPEG image format is a lossy compression, but we can find the meaning equivalent
to the original image. Methods such as the Discrete Wavelet transform technique PCA
(principal component analysis) are examples of this compression.

5. Discretization Operation

The data discretization technique is used to divide the attributes of the continuous nature into
data with intervals. We replace many constant values of the attributes with labels of small
intervals. This means that mining results are shown in a concise and easily understandable way.

i. Top-down discretization: If you first consider one or a couple of points (so-called

breakpoints or split points) to divide the whole set of attributes and repeat this method
up to the end, then the process is known as top-down discretization, also known as
splitting.
ii. Bottom-up discretization: If you first consider all the constant values as split-points,
some are discarded through a combination of the neighborhood values in the interval.
That process is called bottom-up discretization.

Benefits of Data Reduction

The main benefit of data reduction is simple: the more data you can fit into a terabyte of disk
space, the less capacity you will need to purchase. Here are some benefits of data reduction,
such as:

o Data reduction can save energy.

o Data reduction can reduce your physical storage costs.
o And data reduction can decrease your data center track.

Data reduction greatly increases the efficiency of a storage system and directly impacts your
total spending on capacity.

Kamadhenu BCA College 33

Discretization in data mining
Data discretization refers to a method of converting a huge number of data values into smaller
ones so that the evaluation and management of data become easy. In other words, data
discretization is a method of converting attributes values of continuous data into a finite set of
intervals with minimum data loss. There are two forms of data discretization first is supervised
discretization, and the second is unsupervised discretization. Supervised discretization refers to
a method in which the class data is used. Unsupervised discretization refers to a method
depending upon the way which operation proceeds. It means it works on the top-down
splitting strategy and bottom-up merging strategy.

Now, we can understand this concept with the help of an example

Suppose we have an attribute of Age with the given values

Age 1,5,9,4,7,11,14,17,13,18, 19,31,33,36,42,44,46,70,74,78,77

Table before Discretization

Attribute Age Age Age Age

1,5,4,9,7 11,14,17,13,18,19 31,33,36,42,44,46 70,74,77,78
After Discretization Child Young Mature Old

Another example is analytics, where we gather the static data of website visitors. For example,
all visitors who visit the site with the IP address of India are shown under country level.

Some Famous techniques of data discretization

1. Histogram analysis
Histogram refers to a plot used to represent the underlying frequency distribution of a
continuous data set. Histogram assists the data inspection for data distribution. For
example, Outliers, skewness representation, normal distribution representation, etc.

2. Binning
Binning refers to a data smoothing technique that helps to group a huge number of
continuous values into smaller values. For data discretization and the development of idea
hierarchy, this technique can also be used.

3. Cluster Analysis
Cluster analysis is a form of data discretization. A clustering algorithm is executed by
dividing the values of x numbers into clusters to isolate a computational feature of x.

4. Data discretization using decision tree analysis

Data discretization refers to a decision tree analysis in which a top-down slicing technique
is used. It is done through a supervised procedure. In a numeric attribute discretization,
first, you need to select the attribute that has the least entropy, and then you need to run
it with the help of a recursive process. The recursive process divides it into various
discretized disjoint intervals, from top to bottom, using the same splitting criterion.

Kamadhenu BCA College 34

5. Data discretization using correlation analysis
Discretizing data by linear regression technique, you can get the best neighboring interval,
and then the large intervals are combined to develop a larger overlap to form the final 20
overlapping intervals. It is a supervised procedure.

Data discretization and concept hierarchy generation

The term hierarchy represents an organizational structure or mapping in which items are
ranked according to their levels of importance. In other words, we can say that a hierarchy
concept refers to a sequence of mappings with a set of more general concepts to complex
concepts. It means mapping is done from low-level concepts to high-level concepts. For
example, in computer science, there are different types of hierarchical systems. A document is
placed in a folder in windows at a specific place in the tree structure is the best example of a
computer hierarchical tree model. There are two types of hierarchy: top-down mapping and
the second one is bottom-up mapping.

Let's understand this concept hierarchy for the dimension location with the help of an example.
A particular city can map with the belonging country. For example, New Delhi can be mapped
to India, and India can be mapped to Asia.

Top-down mapping

Top-down mapping generally starts with the top with some general information and ends with
the bottom to the specialized information.

Bottom-up mapping

Bottom-up mapping generally starts with the bottom with some specialized information and
ends with the top to the generalized information.

Kamadhenu BCA College 35

Data discretization and binarization in data mining

Data discretization is a method of converting attributes values of continuous data into a finite
set of intervals with minimum data loss. In contrast, data binarization is used to transform the
continuous and discrete attributes into binary attributes.

Kamadhenu BCA College 36

 UNIT II - MINING FREQUENS PATTERNS
Frequent Pattern Mining

Finding recurrent patterns or item sets in huge datasets is the goal of frequent pattern mining,
a crucial data mining approach. It looks for groups of objects that regularly appear together in
order to expose underlying relationships and interdependence. Market basket analysis, web
usage mining, and bioinformatics are a few areas where this method is important.

It helps organizations comprehend client preferences, optimize cross−selling tactics, and

improve recommendation systems by revealing patterns of consumer behavior. By examining
user navigational habits and customizing the browsing experience, online use mining aids in
enhancing website performance. We'll examine frequent pattern mining in data mining in this
piece. Let's begin.

Basic Concepts in Frequent Pattern Mining

The technique of frequent pattern mining

is built upon a number of fundamental
ideas. The analysis is based on transaction
databases, which include records or
transactions that represent collections of
objects. Items inside these transactions
are grouped together as itemsets.

The importance of patterns is greatly

influenced by support and confidence
measurements. Support quantifies how
frequently an itemset appears in the
database, whereas confidence quantifies
how likely it is that a rule generated from
the itemset is accurate.

The Apriori algorithm, a popular method for finding recurrent patterns, takes a methodical
approach. In order to find no more frequent itemsets, it generates candidate itemsets, prunes
the infrequent ones, and then progressively grows the size of the itemsets. The patterns that
fulfill the required support criteria are successfully identified through this iterative approach.

Apriori Algorithm
Apriori algorithm refers to the algorithm which is used to calculate the association rules
between objects. It means how two or more objects are related to one another. In other words,
we can say that the apriori algorithm is an association rule leaning that analyzes that people
who bought product A also bought product B.

The primary objective of the apriori algorithm is to create the association rule between
different objects. The association rule describes how two or more objects are related to one
another. Apriori algorithm is also called frequent pattern mining. Generally, you operate the
Apriori algorithm on a database that consists of a huge number of transactions. Let's
Kamadhenu BCA College 37
 understand the apriori algorithm with the help of an example; suppose you go to Big Bazar and
buy different products. It helps the customers buy their products with ease and increases the
sales performance of the Big Bazar. In this tutorial, we will discuss the apriori algorithm with
examples.

Introduction

We take an example to understand the concept better. You must have noticed that the Pizza
shop seller makes a pizza, soft drink, and breadstick combo together. He also offers a discount
to their customers who buy these combos. Do you ever think why does he do so? He thinks
that customers who buy pizza also buy soft drinks and breadsticks. However, by making
combos, he makes it easy for the customers. At the same time, he also increases his sales
performance.

Similarly, you go to Big Bazar, and you will find biscuits, chips, and Chocolate bundled together.
It shows that the shopkeeper makes it comfortable for the customers to buy these products in
the same place.

The above two examples are the best examples of Association Rules in Data Mining. It helps us
to learn the concept of apriori algorithms.

Apriori algorithm refers to an algorithm that is used in mining frequent products sets and
relevant association rules. Generally, the apriori algorithm operates on a database containing a
huge number of transactions. For example, the items customers but at a Big Bazar.

Apriori algorithm helps the customers to buy their products with ease and increases the sales
performance of the particular store.

Components of Apriori algorithm

The given three components comprise the apriori algorithm.

1. Support
2. Confidence
3. Lift

Let's take an example to understand this concept.

We have already discussed above; you need a huge database containing a large no of
transactions. Suppose you have 4000 customers transactions in a Big Bazar. You have to
calculate the Support, Confidence, and Lift for two products, and you may say Biscuits and
Chocolate. This is because customers frequently buy these two items together. Out of 4000
transactions, 400 contain Biscuits, whereas 600 contain Chocolate, and these 600 transactions
include a 200 that includes Biscuits and chocolates. Using this data, we will find out the support,
confidence, and lift.

1. Support
Support refers to the default popularity of any product. You find the support as a quotient
of the division of the number of transactions comprising that product by the total number
of transactions. Hence, we get

Kamadhenu BCA College 38

 Support (Biscuits) = (Transactions relating biscuits) / (Total transactions)
= 400/4000 = 10 percent.

2. Confidence
Confidence refers to the possibility that the customers bought both biscuits and
chocolates together. So, you need to divide the number of transactions that comprise both
biscuits and chocolates by the total number of transactions to get the confidence.

Hence,
Confidence = (Transactions relating both biscuits and Chocolate) / (Total transactions
involving Biscuits)
= 200/400
= 50 percent.
It means that 50 percent of customers who bought biscuits bought chocolates also.

3. Lift
Consider the above example; lift refers to the increase in the ratio of the sale of chocolates
when you sell biscuits. The mathematical equations of lift are given below.

Lift = (Confidence (Biscuits - chocolates)/ (Support (Biscuits)

= 50/10 = 5

It means that the probability of people buying both biscuits and chocolates together is five
times more than that of purchasing the biscuits alone. If the lift value is below one, it
requires that the people are unlikely to buy both the items together. Larger the value, the
better is the combination.

Working of Apriori Algorithm

We will understand this algorithm with the help of an example Consider a Big Bazar scenario
where the product set is P = {Rice, Pulse, Oil, Milk, Apple}. The database comprises six
transactions where 1 represents the presence of the product and 0 represents the absence of
the product.

Transaction ID Rice Pulse Oil Milk Apple

t1 1 1 1 0 0

t2 0 1 1 1 0

t3 0 0 0 1 1

t4 1 1 0 1 0

t5 1 1 1 0 1

t6 1 1 1 1 1

The Apriori Algorithm makes the given assumptions

Kamadhenu BCA College 39
  All subsets of a frequent itemset must be frequent.
 The subsets of an infrequent item set must be infrequent.
 Fix a threshold support level. In our case, we have fixed it at 50 percent.

Step 1
Make a frequency table of all the products that appear in all the transactions. Now, short the
frequency table to add only those products with a threshold support level of over 50 percent.
We find the given frequency table.

Product Frequency (Number of

transactions)
Rice (R) 4
Pulse(P) 5
Oil(O) 4
Milk(M) 4

The above table indicated the products frequently bought by the customers.

Step 2
Create pairs of products such as RP, RO, RM, PO, PM, OM. You will get the given frequency
table.

Itemset Frequency (Number of

transactions)
RP
4
RO
3
RM
2
PO
4
PM
3
OM
2

Step 3
Implementing the same threshold support of 50 percent and consider the products that are
more than 50 percent. In our case, it is more than 3

Thus, we get RP, RO, PO, and PM

Step 4
Now, look for a set of three products that the customers buy together. We get the given
combination.
1. RP and RO give RPO

Kamadhenu BCA College 40

 2. PO and PM give POM

Step 5
Calculate the frequency of the two itemsets, and you will get the given frequency table.

Itemset Frequency (Number of

transactions)
RPO
4
POM
3

If you implement the threshold assumption, you can figure out that the customers' set of three
products is RPO.

We have considered an easy example to discuss the apriori algorithm in data mining. In reality, you
find thousands of such combinations.

There are various methods used for the efficiency of the Apriori algorithm

Hash-based itemset counting

In hash-based itemset counting, you need to exclude the k-itemset whose equivalent hashing
bucket count is least than the threshold is an infrequent itemset.

Transaction Reduction

In transaction reduction, a transaction not involving any frequent X itemset becomes not valuable
in subsequent scans.

Apriori Algorithm in data mining

We have already discussed an example of the apriori algorithm related to the frequent itemset
generation. Apriori algorithm has many applications in data mining.

The primary requirements to find the association rules in data mining are given below.

Use Brute Force

Analyze all the rules and find the support and confidence levels for the individual rule. Afterward,
eliminate the values which are less than the threshold support and confidence levels.

The two-step approaches

The two-step approach is a better option to find the associations rules than the Brute Force
method.

Step 1
we have already discussed how to create the frequency table and calculate itemsets having a
greater support value than that of the threshold support.

Kamadhenu BCA College 41

Step 2
To create association rules, you need to use a binary partition of the frequent itemsets. You
need to choose the ones having the highest confidence levels.

In the above example, you can see that the RPO combination was the frequent itemset. Now,
we find out all the rules using RPO.

RP-O, RO-P, PO-R, O-RP, P-RO, R-PO

You can see that there are six different combinations. Therefore, if you have n elements, there
will be 2n - 2 candidate association rules.

Advantages of Apriori Algorithm

 It is used to calculate large itemsets.
 Simple to understand and apply.

Disadvantages of Apriori Algorithms

 Apriori algorithm is an expensive method to find support since the calculation has to pass
through the whole database.
 Sometimes, you need a huge number of candidate rules, so it becomes computationally
more expensive.

FP Growth Algorithm in Data Mining

In Data Mining, finding frequent patterns in large databases is very important and has been
studied on a large scale in the past few years. Unfortunately, this task is computationally
expensive, especially when many patterns exist.

The FP-Growth Algorithm proposed by Han in. This is an efficient and scalable method for
mining the complete set of frequent patterns by pattern fragment growth, using an extended
prefix-tree structure for storing compressed and crucial information about frequent patterns
named frequent-pattern tree (FP-tree). In his study, Han proved that his method outperforms
other popular methods for mining frequent patterns, e.g. the Apriori Algorithm and the
TreeProjection. In some later works, it was proved that FP-Growth performs better than other
methods, including Eclat and Relim. The popularity and efficiency of the FP-Growth Algorithm
contribute to many studies that propose variations to improve its performance.

The FP-Growth Algorithm is an alternative way to find frequent item sets without using
candidate generations, thus improving performance. For so much, it uses a divide-and-conquer
strategy. The core of this method is the usage of a special data structure named frequent-
pattern tree (FP-tree), which retains the item set association information.

This algorithm works as follows:

 First, it compresses the input database creating an FP-tree instance to represent frequent
items.
 After this first step, it divides the compressed database into a set of conditional databases,
each associated with one frequent pattern.

Kamadhenu BCA College 42

  Finally, each such database is mined separately.

Using this strategy, the FP-Growth reduces the search costs by recursively looking for short
patterns and then concatenating them into the long frequent patterns.

In large databases, holding the FP tree in the main memory is impossible. A strategy to cope with
this problem is to partition the database into a set of smaller databases (called projected
databases) and then construct an FP-tree from each of these smaller databases.

FP-Tree
The frequent-pattern tree (FP-tree) is a compact data structure that stores quantitative
information about frequent patterns in a database. Each transaction is read and then mapped
onto a path in the FP-tree. This is done until all transactions have been read. Different
transactions with common subsets allow the tree to remain compact because their paths
overlap.

A frequent Pattern Tree is made with the initial item sets of the database. The purpose of the FP
tree is to mine the most frequent pattern. Each node of the FP tree represents an item of the
item set.

The root node represents null, while the lower nodes represent the item sets. The associations of
the nodes with the lower nodes, that is, the item sets with the other item sets, are maintained
while forming the tree.

Han defines the FP-tree as the tree structure given below:

1. One root is labelled as "null" with a set of item-prefix subtrees as children and a
frequent-item-header table.
2. Each node in the item-prefix subtree consists of three fields:
o Item-name: registers which item is represented by the node;
o Count: the number of transactions represented by the portion of the path
reaching the node;
o Node-link: links to the next node in the FP-tree carrying the same item name or
null if there is none.
3. Each entry in the frequent-item-header table consists of two fields:
o Item-name: as the same to the node;
o Head of node-link: a pointer to the first node in the FP-tree carrying the item
name.

Additionally, the frequent-item-header table can have the count support for an item. The below
diagram is an example of a best-case scenario that occurs when all transactions have the same
itemset; the size of the FP-tree will be only a single branch of nodes.

Kamadhenu BCA College 43

The worst-case scenario occurs when every transaction has a unique item set. So the space
needed to store the tree is greater than the space used to store the original data set because
the FP-tree requires additional space to store pointers between nodes and the counters for
each item. The diagram below shows how a worst-case scenario FP-tree might appear. As you
can see, the tree's complexity grows with each transaction's uniqueness.

Algorithm by Han
The original algorithm to construct the FP-Tree defined by Han is given below:

Algorithm 1: FP-tree construction

Input: A transaction database DB and a minimum support threshold?
Output: FP-tree, the frequent-pattern tree of DB.
Method: The FP-tree is constructed as follows.

1. The first step is to scan the database to find the occurrences of the itemsets in the
database. This step is the same as the first step of Apriori. The count of 1-itemsets in the
database is called support count or frequency of 1-itemset.
2. The second step is to construct the FP tree. For this, create the root of the tree. The root is
represented by null.
3. The next step is to scan the database again and examine the transactions. Examine the
first transaction and find out the itemset in it. The itemset with the max count is taken at

Kamadhenu BCA College 44

 the top, and then the next itemset with the lower count. It means that the branch of the tree
is constructed with transaction itemsets in descending order of count.
4. The next transaction in the database is examined. The itemsets are ordered in descending
order of count. If any itemset of this transaction is already present in another branch, then
this transaction branch would share a common prefix to the root. This means that the
common itemset is linked to the new node of another itemset in this transaction.
5. Also, the count of the itemset is incremented as it occurs in the transactions. The common
node and new node count are increased by 1 as they are created and linked according to
transactions.
6. The next step is to mine the created FP Tree. For this, the lowest node is examined first,
along with the links of the lowest nodes. The lowest node represents the frequency
pattern length 1. From this, traverse the path in the FP Tree. This path or paths is called a
conditional pattern base.
A conditional pattern base is a sub-database consisting of prefix paths in the FP tree occurring
with the lowest node (suffix).
7. Construct a Conditional FP Tree, formed by a count of itemsets in the path. The itemsets
meeting the threshold support are considered in the Conditional FP Tree.
8. Frequent Patterns are generated from the Conditional FP Tree.

Using this algorithm, the FP-tree is constructed in two database scans. The first scan collects and
sorts the set of frequent items, and the second constructs the FP-Tree.

Example

Support threshold=50%, Confidence= 60%

Table 1:

Transaction List of items

T1 I1,I2,I3

T2 I2,I3,I4

T3 I4,I5

T4 I1,I2,I4

T5 I1,I2,I3,I5

T6 I1,I2,I3,I4

Solution: Support threshold=50% => 0.5*6= 3 => min_sup=3

Table 2: Count of each item

Item Count

Kamadhenu BCA College 45

 I1 4

I2 5

I3 4

I4 4

I5 2

Table 3: Sort the itemset in descending order.

Item Count
I2 5

I1 4

I3 4

I4 4

Build FP Tree

Let's build the FP tree in the following steps, such as:

1. Considering the root node null.

2. The first scan of Transaction T1: I1, I2, I3 contains three items {I1:1}, {I2:1}, {I3:1}, where I2 is
linked as a child, I1 is linked to I2 and I3 is linked to I1.
3. T2: I2, I3, and I4 contain I2, I3, and I4, where I2 is linked to root, I3 is linked to I2 and I4 is
linked to I3. But this branch would share the I2 node as common as it is already used in T1.
4. Increment the count of I2 by 1, and I3 is linked as a child to I2, and I4 is linked as a child to I3.
The count is {I2:2}, {I3:1}, {I4:1}.
5. T3: I4, I5. Similarly, a new branch with I5 is linked to I4 as a child is created.
6. T4: I1, I2, I4. The sequence will be I2, I1, and I4. I2 is already linked to the root node. Hence it
will be incremented by 1. Similarly I1 will be incremented by 1 as it is already linked with I2 in
T1, thus {I2:3}, {I1:2}, {I4:1}.
7. T5:I1, I2, I3, I5. The sequence will be I2, I1, I3, and I5. Thus {I2:4}, {I1:3}, {I3:2}, {I5:1}.
8. T6: I1, I2, I3, I4. The sequence will be I2, I1, I3, and I4. Thus {I2:5}, {I1:4}, {I3:3}, {I4 1}.

Kamadhenu BCA College 46

Mining of FP-tree is summarized below:

1. The lowest node item, I5, is not considered as it does not have a min support count.
Hence it is deleted.
2. The next lower node is I4. I4 occurs in 2 branches , {I2,I1,I3:,I41},{I2,I3,I4:1}. Therefore
considering I4 as suffix the prefix paths will be {I2, I1, I3:1}, {I2, I3: 1} this forms the
conditional pattern base.
3. The conditional pattern base is considered a transaction database, and an FP tree is
constructed. This will contain {I2:2, I3:2}, I1 is not considered as it does not meet the min
support count.
4. This path will generate all combinations of frequent patterns : {I2,I4:2},{I3,I4:2},{I2,I3,I4:2}
5. For I3, the prefix path would be: {I2,I1:3},{I2:1}, this will generate a 2 node FP-tree : {I2:4,
I1:3} and frequent patterns are generated: {I2,I3:4}, {I1:I3:3}, {I2,I1,I3:3}.
6. For I1, the prefix path would be: {I2:4} this will generate a single node FP-tree: {I2:4} and
frequent patterns are generated: {I2, I1:4}.

Conditional Pattern Conditional FP- Frequent Patterns

Item
Base tree Generated
I4 {I2,I1,I3:1},{I2,I3:1} {I2:2, I3:2} {I2,I4:2},{I3,I4:2},{I2,I3,I4:2}

I3 {I2,I1:3},{I2:1} {I2:4, I1:3} {I2,I3:4}, {I1:I3:3}, {I2,I1,I3:3}

I1 {I2:4} {I2:4} {I2,I1:4}

The diagram given below depicts the conditional FP tree associated with the conditional node
I3.

Kamadhenu BCA College 47

FP-Growth Algorithm
After constructing the FP-Tree, it's possible to mine it to find the complete set of frequent
patterns. Han presents a group of lemmas and properties to do this job and then describes the
following FP-Growth Algorithm.

Algorithm 2: FP-Growth

Input: A database DB, represented by FP-tree constructed according to Algorithm 1, and a

minimum support threshold?

Output: The complete set of frequent patterns.

Method: Call FP-growth (FP-tree, null).

Procedure FP-growth(Tree, a)
{
If the tree contains a single prefix path, then.
{
// Mining single prefix-path FP-tree
let P be the single prefix-path part of the tree;
let Q be the multipath part with the top branching node replaced by a null root;

generate pattern ß ∪ a with support = minimum support of nodes in ß;

for each combination (denoted as ß) of the nodes in the path, P do

let freq pattern set(P) be the set of patterns so generated;

}
else let Q be Tree;
for each item ai in Q, do
{

generate pattern ß = ai ∪ a with support = ai .support;

// Mining multipath FP-tree

construct ß's conditional pattern-based, and then ß's conditional FP-tree Tree ß;
if Tree ß ≠ Ø then
call FP-growth(Tree ß, ß);
let freq pattern set(Q) be the set of patterns so generated;

return(freq pattern set(P) ∪ freq pattern set(Q) ∪ (freq pattern set(P) × freq pattern set(Q)))
}

When the FP-tree contains a single prefix path, the complete set of frequent patterns can be
generated in three parts:
Kamadhenu BCA College 48
 1. The single prefix-path P,
2. The multipath Q,
3. And their combinations (lines 01 to 03 and 14).

The resulting patterns for a single prefix path are the enumerations of its subpaths with
minimum support. After that, the multipath Q is defined, and the resulting patterns are
processed. Finally, the combined results are returned as the frequent patterns found.

Advantages of FP Growth Algorithm

Here are the following advantages of the FP growth algorithm, such as:
 This algorithm needs to scan the database twice when compared to Apriori, which scans
the transactions for each iteration.
 The pairing of items is not done in this algorithm, making it faster.
 The database is stored in a compact version in memory.
 It is efficient and scalable for mining both long and short frequent patterns.

Disadvantages of FP-Growth Algorithm

This algorithm also has some disadvantages, such as:
 FP Tree is more cumbersome and difficult to build than Apriori.
 It may be expensive.
 The algorithm may not fit in the shared memory when the database is large.

Difference between Apriori and FP Growth Algorithm

Apriori and FP-Growth algorithms are the most basic FIM algorithms. There are some basic
differences between these algorithms, such as:

Apriori FP Growth

Apriori generates frequent patterns by making FP Growth generates an FP-Tree for making
the itemsets using pairings such as single item frequent patterns.
set, double itemset, and triple itemset.
Apriori uses candidate generation where FP-growth generates a conditional FP-Tree
frequent subsets are extended one item at a for every item in the data.
time.
Since apriori scans the database in each step, FP-tree requires only one database scan in
it becomes time-consuming for data where its beginning steps, so it consumes less
the number of items is larger. time.
A converted version of the database is saved A set of conditional FP-tree for every item
in the memory is saved in the memory

It uses a breadth-first search It uses a depth-first search.

Kamadhenu BCA College 49

Association Rule Learning
Association rule learning is a type of unsupervised learning technique that checks for the
dependency of one data item on another data item and maps accordingly so that it can be
more profitable. It tries to find some interesting relations or associations among the variables
of dataset. It is based on different rules to discover the interesting relations between variables
in the database.

The association rule learning is one of the very important concepts of machine learning, and it
is employed in Market Basket analysis, Web usage mining, continuous production, etc. Here
market basket analysis is a technique used by the various big retailer to discover the
associations between items. We can understand it by taking an example of a supermarket, as
in a supermarket, all products that are purchased together are put together.

For example, if a customer buys bread, he most likely can also buy butter, eggs, or milk, so
these products are stored within a shelf or mostly nearby. Consider the below diagram:

Association rule learning can be divided into three types of algorithms:

1. Apriori
2. Eclat
3. F-P Growth Algorithm
We will understand these algorithms in later chapters.
Working :
Association rule learning works on the concept of If and Else Statement, such as if A then B.

Here the If element is called antecedent, and then statement is called as Consequent. These
types of relationships where we can find out some association or relation between two items is
known as single cardinality. It is all about creating rules, and if the number of items increases,

Kamadhenu BCA College 50

then cardinality also increases accordingly. So, to measure the associations between thousands
of data items, there are several metrics. These metrics are given below:

1. Support
2. Confidence
3. Lift

Let's understand each of them:

Support
Support is the frequency of A or how frequently an item appears in the dataset. It is defined as
the fraction of the transaction T that contains the itemset X. If there are X datasets, then for
transactions T, it can be written as:

Confidence
Confidence indicates how often the rule has been found to be true. Or how often the items X
and Y occur together in the dataset when the occurrence of X is already given. It is the ratio of
the transaction that contains X and Y to the number of records that contain X.

Lift
It is the strength of any rule, which can be defined as below formula:

It is the ratio of the observed support measure and expected support if X and Y are
independent of each other. It has three possible values:

o If Lift= 1: The probability of occurrence of antecedent and consequent is independent

of each other.
o Lift>1: It determines the degree to which the two itemsets are dependent to each other.
o Lift<1: It tells us that one item is a substitute for other items, which means one item has
a negative effect on another.

Types of Association Rule Lerning

Association rule learning can be divided into three algorithms:

Apriori Algorithm
This algorithm uses frequent datasets to generate association rules. It is designed to work on
the databases that contain transactions. This algorithm uses a breadth-first search and Hash
Tree to calculate the itemset efficiently.
Kamadhenu BCA College 51
It is mainly used for market basket analysis and helps to understand the products that can be
bought together. It can also be used in the healthcare field to find drug reactions for patients.

Eclat Algorithm
Eclat algorithm stands for Equivalence Class Transformation. This algorithm uses a depth-
first search technique to find frequent itemsets in a transaction database. It performs faster
execution than Apriori Algorithm.

F-P Growth Algorithm

The F-P growth algorithm stands for Frequent Pattern, and it is the improved version of the
Apriori Algorithm. It represents the database in the form of a tree structure that is known as a
frequent pattern or tree. The purpose of this frequent tree is to extract the most frequent
patterns.

Applications of Association Rule Learning

It has various applications in machine learning and data mining. Below are some popular
applications of association rule learning:

 Market Basket Analysis: It is one of the popular examples and applications of

association rule mining. This technique is commonly used by big retailers to determine
the association between items.
 Medical Diagnosis: With the help of association rules, patients can be cured easily, as
it helps in identifying the probability of illness for a particular disease.
 Protein Sequence: The association rules help in determining the synthesis of artificial
Protein.
 It is also used for the Catalog Design and Loss-leader Analysis and many more other
applications.

Kamadhenu BCA College 52

 Classification
Classification:

It is a data analysis task, i.e. the process of finding a model that describes and distinguishes data
classes and concepts. Classification is the problem of identifying to which of a set of categories
(subpopulations), a new observation belongs to, on the basis of a training set of data
containing observations and whose categories membership is known.
Example: Before starting any project, we need to check its feasibility. In this case, a classifier is
required to predict class labels such as ‘Safe’ and ‘Risky’ for adopting the Project and to further
approve it. It is a two-step process such as:
1. Learning Step (Training Phase): Construction of Classification Model
Different Algorithms are used to build a classifier by making the model learn using the
training set available. The model has to be trained for the prediction of accurate results.
2. Classification Step: Model used to predict class labels and testing the constructed
model on test data and hence estimate the accuracy of the classification rules.

Test data are used to estimate the accuracy of the classification rule
Training and Testing:

Suppose there is a person who is sitting under a fan and the fan starts falling on him, he should get
aside in order not to get hurt. So, this is his training part to move away. While Testing if the
person sees any heavy object coming towards him or falling on him and moves aside then the
system is tested positively and if the person does not move aside then the system is negatively
tested.The same is the case with the data, it should be trained in order to get the accurate and
best results.

There are certain data types associated with data mining that actually tells us the format of the file
(whether it is in text format or in numerical format).

Attributes – Represents different features of an object. Different types of attributes are:

Kamadhenu BCA College 53

 1. Binary: Possesses only two values i.e. True or False
Example: Suppose there is a survey evaluating some products. We need to check
whether it’s useful or not. So, the Customer has to answer it in Yes or No.

Product usefulness: Yes / No

 Symmetric: Both values are equally important in all aspects

 Asymmetric: When both the values may not be important.
2. Nominal: When more than two outcomes are possible. It is in Alphabet form rather
than being in Integer form.
Example: One needs to choose some material but of different colors. So, the color
might be Yellow, Green,Black,Red.
Different Colors: Red, Green, Black, Yellow
 Ordinal: Values that must have some meaningful order.
Example: Suppose there are grade sheets of few students which might contain
different grades as per their performance such as A, B, C, D Grades: A, B, C, D
 Continuous: May have an infinite number of values, it is in float type
Example: Measuring the weight of few Students in a sequence or orderly
manner i.e. 50, 51, 52, 53 Weight: 50, 51, 52, 53
 Discrete: Finite number of values.Example: Marks of a Student in a few subjects:
65, 70, 75, 80, 90 Marks: 65, 70, 75, 80, 90
Syntax:

 Mathematical Notation: Classification is based on building a function taking input

feature vector “X” and predicting its outcome “Y” (Qualitative response taking values in
set C)
 Here Classifier (or model) is used which is a Supervised function, can be designed
manually based on the expert’s knowledge. It has been constructed to predict class
labels (Example: Label – “Yes” or “No” for the approval of some event).
Classifiers can be categorized into two major types:
1. Discriminative: It is a very basic classifier and determines just one class for each row of
data. It tries to model just by depending on the observed data, depends heavily on the
quality of data rather than on distributions. Example: Logistic Regression
2. Generative: It models the distribution of individual classes and tries to learn the model
that generates the data behind the scenes by estimating assumptions and distributions
of the model. Used to predict the unseen data. Example: Naive Bayes Classifier
Detecting Spam emails by looking at the previous data. Suppose 100 emails and that
too divided in 1:4 i.e. Class A: 25%(Spam emails) and Class B: 75%(Non-Spam emails).
Now if a user wants to check that if an email contains the word cheap, then that may be
termed as Spam. It seems to be that in Class A(i.e. in 25% of data), 20 out of 25 emails
are spam and rest not. And in Class B(i.e. in 75% of data), 70 out of 75 emails are not
spam and rest are spam. So, if the email contains the word cheap, what is the
probability of it being spam ?? (= 80%)
Classifiers Of Machine Learning:

1. Decision Trees
2. Bayesian Classifiers
3. Neural Networks

Kamadhenu BCA College 54

 4. K-Nearest Neighbour
5. Support Vector Machines
6. Linear Regression
7. Logistic Regression
Associated Tools and Languages: Used to mine/ extract useful information from raw data.
 Main Languages used: R, SAS, Python, SQL
 Major Tools used: RapidMiner, Orange, KNIME, Spark, Weka
 Libraries used: Jupyter, NumPy, Matplotlib, Pandas, ScikitLearn, NLTK, TensorFlow,
Seaborn, Basemap, etc.
Real–Life Examples :

 Market Basket Analysis:It is a modeling technique that has been associated with
frequent transactions of buying some combination of items.
Example: Amazon and many other Retailers use this technique. While viewing some
products, certain suggestions for the commodities are shown that some people have
bought in the past.
 Weather Forecasting:Changing Patterns in weather conditions needs to be observed
based on parameters such as temperature, humidity, wind direction. This keen
observation also requires the use of previous records in order to predict it accurately.
Advantages:

 Mining Based Methods are cost-effective and efficient

 Helps in identifying criminal suspects
 Helps in predicting the risk of diseases
 Helps Banks and Financial Institutions to identify defaulters so that they may approve
Cards, Loan, etc.
Disadvantages:
Privacy: When the data is either are chances that a company may give some information about
their customers to other vendors or use this information for their profit. Accuracy Problem:
Selection of Accurate model must be there in order to get the best accuracy and result.
APPLICATIONS:

 Marketing and Retailing

 Manufacturing
 Telecommunication Industry
 Intrusion Detection
 Education System
 Fraud Detection
Issues of classification

Classification in data mining involves assigning predefined labels or categories to instances based
on their features or attributes. While classification algorithms have proven to be powerful
tools, they are not without challenges. Here are some common issues associated with
classification in data mining:
1. Imbalanced Datasets:
Kamadhenu BCA College 55
  When the distribution of classes in the dataset is uneven, classification
algorithms may be biased toward the majority class. This can lead to poor
performance in predicting minority classes.
2. Overfitting:
 Overfitting occurs when a model learns the training data too well, capturing
noise or random fluctuations rather than the underlying pattern. This can result
in poor generalization to new, unseen data.
3. Underfitting:
 In contrast to overfitting, underfitting occurs when a model is too simple to
capture the underlying structure of the data. This leads to poor performance on
both the training and test datasets.
4. Noise in Data:
 Noisy data, which includes errors or irrelevant information, can negatively
impact the performance of classification algorithms. Noisy features may mislead
the model and result in incorrect predictions.
5. Feature Selection and Dimensionality:
 The curse of dimensionality can affect the performance of classification
algorithms when dealing with a large number of features. Feature selection or
dimensionality reduction techniques are often needed to mitigate this issue.
6. Categorical and Missing Data:
 Many classification algorithms are designed for numerical data. Handling
categorical variables or dealing with missing data requires preprocessing and
may impact the model's performance.
7. Computational Complexity:
 Some classification algorithms can be computationally intensive, especially
when dealing with large datasets. Training complex models may require
significant computational resources.
8. Interpretable Models:
 Complex models, such as deep neural networks, might lack interpretability,
making it challenging to understand the decision-making process. Interpretable
models are often preferred in scenarios where transparency is crucial.
9. Concept Drift:
 The underlying relationships in the data may change over time, leading to
concept drift. Classification models trained on historical data may become less
accurate when applied to current data.
10. Scalability:
 Some algorithms may not scale well with the size of the dataset. Training large
models or handling big data can be challenging for certain classification
techniques.
11. Evaluation Metrics:
 The choice of evaluation metrics is crucial and may depend on the nature of the
problem. Using inappropriate metrics can lead to a misinterpretation of the
model's performance.

Kamadhenu BCA College 56

 12. Ethical and Bias Concerns:
 Bias in the training data or algorithmic decisions may lead to unfair or
discriminatory outcomes. Ensuring fairness and addressing bias is a critical
consideration in classification tasks, especially when the data reflects societal
inequalities.
Addressing these issues requires a combination of careful preprocessing, appropriate algorithm
selection, and ongoing monitoring and refinement of models. It's essential to tailor the
approach to the specific characteristics of the data and the goals of the classification task.

Decision tree induction

Decision Tree is a supervised learning method used in data mining for classification and regression
methods. It is a tree that helps us in decision-making purposes. The decision tree creates
classification or regression models as a tree structure. It separates a data set into smaller
subsets, and at the same time, the decision tree is steadily developed. The final tree is a tree
with the decision nodes and leaf nodes. A decision node has at least two branches. The leaf
nodes show a classification or decision. We can't accomplish more split on leaf nodes-The
uppermost decision node in a tree that relates to the best predictor called the root node.
Decision trees can deal with both categorical and numerical data.
Key factors:

Entropy:

Entropy refers to a common way to measure impurity. In the decision tree, it measures the
randomness or impurity in data sets.

Information Gain:

Information Gain refers to the decline in entropy after the dataset is split. It is also called
Entropy Reduction. Building a decision tree is all about discovering attributes that return the
highest data gain.

Kamadhenu BCA College 57

In short, a decision tree is just like a flow chart diagram with the terminal nodes showing
decisions. Starting with the dataset, we can measure the entropy to find a way to segment the
set until the data belongs to the same class.
Why are decision trees useful?

 It enables us to analyze the possible consequences of a decision thoroughly.

 It provides us a framework to measure the values of outcomes and the
probability of accomplishing them.
 It helps us to make the best decisions based on existing data and best
speculations.
In other words, we can say that a decision tree is a hierarchical tree structure that can be used
to split an extensive collection of records into smaller sets of the class by implementing a
sequence of simple decision rules. A decision tree model comprises a set of rules for portioning
a huge heterogeneous population into smaller, more homogeneous, or mutually exclusive
classes. The attributes of the classes can be any variables from nominal, ordinal, binary, and
quantitative values, in contrast, the classes must be a qualitative type, such as categorical or
ordinal or binary. In brief, the given data of attributes together with its class, a decision tree
creates a set of rules that can be used to identify the class. One rule is implemented after
another, resulting in a hierarchy of segments within a segment. The hierarchy is known as
the tree, and each segment is called a node. With each progressive division, the members from
the subsequent sets become more and more similar to each other. Hence, the algorithm used
to build a decision tree is referred to as recursive partitioning. The algorithm is known
as CART (Classification and Regression Trees).
Consider the given example of a factory where

Expanding factor costs $3 million, the probability of a good economy is 0.6 (60%), which leads
to $8 million profit, and the probability of a bad economy is 0.4 (40%), which leads to $6
million profit.
Not expanding factor with 0$ cost, the probability of a good economy is 0.6(60%), which leads
to $4 million profit, and the probability of a bad economy is 0.4, which leads to $2 million profit.
The management teams need to take a data-driven decision to expand or not based on the
given data.

Net Expand = ( 0.6 *8 + 0.4*6 ) - 3 = $4.2M

Net Not Expand = (0.6*4 + 0.4*2) - 0 = $3M
$4.2M > $3M,therefore the factory should be expanded.
Kamadhenu BCA College 58
Bayesian classification in data mining is a statistical technique used to classify data based on
probabilistic reasoning. It is a type of probabilistic classification that uses Bayes' theorem to
predict the probability of a data point belonging to a certain class. The Bayesian classification is
a powerful technique for probabilistic inference and decision-making and is widely used in
various applications such as medical diagnosis, spam classification, fraud detection, etc.
Introduction to Bayesian Classification in Data Mining

Bayesian classification in data mining is a statistical approach to data classification that uses
Bayes' theorem to make predictions about a class of a data point based on observed data. It is
a popular data mining and machine learning technique for modelling the probability of certain
outcomes and making predictions based on that probability.
The basic idea behind Bayesian classification in data mining is to assign a class label to a new
data instance based on the probability that it belongs to a particular class, given the observed
data. Bayes' theorem provides a way to compute this probability by multiplying the prior
probability of the class (based on previous knowledge or assumptions) by the likelihood of the
observed data given that class (conditional probability).

Several types of Bayesian classifiers exist, such as naive Bayes, Bayesian network classifiers,
Bayesian logistic regression, etc. Bayesian classification is preferred in many applications
because it allows for the incorporation of new data (just by updating the prior probabilities)
and can update the probabilities of class labels accordingly.
This is important when new data is constantly being collected, or the underlying distribution
may change over time. In contrast, other classification techniques, such as decision trees or
support vector machines, do not easily accommodate new data and may require re-training of
the entire model to incorporate new information. This can be computationally expensive and
time-consuming.
Bayesian classification is a powerful tool for data mining and machine learning and is widely
used in many applications, such as spam filtering, text classification, and medical diagnosis. Its
ability to incorporate prior knowledge and uncertainty makes it well-suited for real-world
problems where data is incomplete or noisy and accurate predictions are critical.
Bayes’ Theorem in Data Mining

Bayes' theorem is used in Bayesian classification in data mining, which is a technique for
predicting the class label of a new instance based on the probabilities of different class labels
and the observed features of the instance. In data mining, Bayes' theorem is used to compute
the probability of a hypothesis (such as a class label or a pattern in the data) given some
observed event (such as a set of features or attributes). It is named after Reverend Thomas
Bayes, an 18th-century British mathematician who first formulated it.
Bayes' theorem states that the probability of a hypothesis H given some observed event E is
proportional to the likelihood of the evidence given the hypothesis, multiplied by the prior
probability of the hypothesis, as shown below -

Kamadhenu BCA College 59

Rule-based classification

Rule-based classification in data mining is a technique in which class decisions are taken based
on various “if...then… else” rules. Thus, we define it as a classification type governed by a set of
IF-THEN rules. We write an IF-THEN rule as:

“IF condition THEN conclusion.”

IF-THEN Rule

To define the IF-THEN rule, we can split it into two parts:

 Rule Antecedent: This is the “if condition” part of the rule. This part is present in the
LHS(Left Hand Side). The antecedent can have one or more attributes as conditions,
with logic AND operator.
 Rule Consequent: This is present in the rule's RHS(Right Hand Side). The rule
consequent consists of the class prediction.

Example:

R1: IF tutor = codingNinja AND student = yes

THEN happyLearning = true

Assessment of Rule

In rule-based classification in data mining, there are two factors based on which we can access
the rules. These are:

 Coverage of Rule: The fraction of the records which satisfy the antecedent
conditions of a particular rule is called the coverage of that rule.We can calculate
this by dividing the number of records satisfying the rule(n1) by the total number of
records(n). Coverage(R) = n1/n
 Accuracy of a rule: The fraction of the records that satisfy the antecedent
conditions and meet the consequent values of a rule is called the accuracy of that
rule. We can calculate this by dividing the number of records satisfying the
consequent values(n2) by the number of records satisfying the rule(n1). Accuracy(R)
= n2/n1
Generally, we convert them into percentages by multiplying them by 100. We do so to make it
easy for the layman to understand these terms and their values.
Properties of Rule-Based Classifiers
There are two significant properties of rule-based classification in data mining. They are:
Kamadhenu BCA College 60
  Rules may not be mutually exclusive
 Rules may not be exhaustive
Rules may not be mutually exclusive in nature

Many different rules are generated for the dataset, so it is possible and likely that many of
them satisfy the same data record. This condition makes the rules not mutually exclusive.

Since the rules are not mutually exclusive, we cannot decide on classes that cover different
parts of data on different rules. But this was our main objective. So, to solve this problem, we
have two ways:
 The first way is using an ordered set of rules. By ordering the rules, we set priority
orders. Thus, this ordered rule set is called a decision list. So the class with the highest
priority rule is taken as the final class.

 The second solution can be assigning votes for each class depending on their weights.
So, in this, the set of rules remains unordered.
lazy learner

A "lazy learner" in the context of machine learning refers to an algorithm that postpones the
processing or learning phase until it receives a query for predictions. Instead of building a
model during the training phase, a lazy learner stores instances of the training data and uses
them to make predictions when new, unseen instances are presented. Lazy learners are also
known as instance-based learners or memory-based learners.

One of the most common examples of a lazy learner is the k-Nearest Neighbors (k-NN)
algorithm. Here's an overview of lazy learner classification, using k-NN as an example:
k- Nearest Neighbors (k-NN):

1. Training Phase:
 In the training phase, a k-NN classifier doesn't build a model in the traditional
sense.
 Instead, it memorizes the training instances and their corresponding class labels.

2. Prediction Phase:
 When a new instance needs to be classified, the algorithm identifies the k-
nearest neighbors to the new instance from the training data based on a
distance metric (e.g., Euclidean distance).
 The majority class among the k-nearest neighbors is assigned to the new
instance as its predicted class.
3. Characteristics of Lazy Learners:
 No Model Building: Lazy learners don't create an explicit model during the
training phase. They store the training instances and use them for predictions.
 Adaptability: Lazy learners are adaptive to changes in the dataset, as they can
easily incorporate new instances without retraining the entire model.
 Computationally Intensive: The prediction phase can be computationally
expensive, especially in high-dimensional spaces or with large datasets, as it
requires calculating distances for each query.
4. Parameters in k-NN:
Kamadhenu BCA College 61
  k (Number of Neighbors): The choice of the value for k impacts the algorithm's
sensitivity to noise and its ability to capture the underlying structure of the data.
5. Advantages:
 Simple Implementation: Lazy learners are easy to implement and understand.
 Adaptive to Local Patterns: They can adapt well to local patterns in the data.

6. Disadvantages:
 High Computation Cost: The prediction phase can be computationally expensive,
especially with large datasets.
 Sensitivity to Noise: Lazy learners can be sensitive to noise or irrelevant
features in the dataset.
7. Use Cases:
 Non-Stationary Data: Lazy learners are suitable for scenarios where the data
distribution is non-stationary, and the model needs to adapt to changes over
time.
 Local Patterns: When the decision boundaries are complex and vary in different
parts of the feature space.
Lazy learners, like k-NN, are well-suited for certain types of problems, particularly in situations
where the decision boundaries are complex and the relationships within the data are nonlinear
and varied. However, their computational cost can be a drawback in high-dimensional or large
datasets. The choice of whether to use a lazy learner depends on the specific characteristics of
the problem at hand.
Nearest Neighbor

o K-Nearest Neighbor is one of the simplest Machine Learning algorithms based on

Supervised Learning technique.
o K-NN algorithm assumes the similarity between the new case/data and available cases
and put the new case into the category that is most similar to the available categories.
o K-NN algorithm stores all the available data and classifies a new data point based on the
similarity. This means when new data appears then it can be easily classified into a well
suite category by using K- NN algorithm.
o K-NN algorithm can be used for Regression as well as for Classification but mostly it is
used for the Classification problems.
o K-NN is a non-parametric algorithm, which means it does not make any assumption on
underlying data.
o It is also called a lazy learner algorithm because it does not learn from the training set
immediately instead it stores the dataset and at the time of classification, it performs
an action on the dataset.
o KNN algorithm at the training phase just stores the dataset and when it gets new data,
then it classifies that data into a category that is much similar to the new data.
Example: Suppose, we have an image of a creature that looks similar to cat and dog,
but we want to know either it is a cat or dog. So for this identification, we can use the
KNN algorithm, as it works on a similarity measure. Our KNN model will find the similar
features of the new data set to the cats and dogs images and based on the most similar
features it will put it in either cat or dog category.

Kamadhenu BCA College 62

Why do we need a K-NN Algorithm?
Suppose there are two categories, i.e., Category A and Category B, and we have a new data
point x1, so this data point will lie in which of these categories. To solve this type of problem,
we need a K-NN algorithm. With the help of K-NN, we can easily identify the category or class
of a particular dataset. Consider the below diagram:

How does K-NN work?

The K-NN working can be explained on the basis of the below algorithm:

o Step-1: Select the number K of the neighbors

o Step-2: Calculate the Euclidean distance of K number of neighbors
o Step-3: Take the K nearest neighbors as per the calculated Euclidean distance.
o Step-4: Among these k neighbors, count the number of the data points in each category.
o Step-5: Assign the new data points to that category for which the number of the
neighbor is maximum.
o Step-6: Our model is ready.
Suppose we have a new data point and we need to put it in the required category. Consider
the below image:

Kamadhenu BCA College 63

 o Firstly, we will choose the number of neighbors, so we will choose the k=5.
o Next, we will calculate the Euclidean distance between the data points. The Euclidean
distance is the distance between two points, which we have already studied in
geometry. It can be calculated as:

o By calculating the Euclidean distance we got the nearest neighbors, as three nearest
neighbors in category A and two nearest neighbors in category B. Consider the below
image:

o As we can see the 3 nearest neighbors are from category A, hence this new data point
must belong to category A.
How to select the value of K in the K-NN Algorithm?

Below are some points to remember while selecting the value of K in the K-NN algorithm:

o There is no particular way to determine the best value for "K", so we need to try some
values to find the best out of them. The most preferred value for K is 5.
o A very low value for K such as K=1 or K=2, can be noisy and lead to the effects of
outliers in the model.
o Large values for K are good, but it may find some difficulties.
Kamadhenu BCA College 64
Advantages of KNN Algorithm:

o It is simple to implement.
o It is robust to the noisy training data
o It can be more effective if the training data is large.

Disadvantages of KNN Algorithm:

o Always needs to determine the value of K which may be complex some time.
o The computation cost is high because of calculating the distance between the data
points for all the training samples.
Prediction

In data mining, prediction involves using models built from historical or existing data to make
informed predictions or estimations about future or unseen data. The goal is to identify
patterns, relationships, or trends within the data that can be generalized to make predictions
on new instances. Predictive modeling is a key aspect of data mining and machine learning.
Here's an overview of the prediction process in data mining:
1. Data Collection and Preprocessing:
 The first step is to collect relevant data for the problem at hand. This data is
then preprocessed to clean it, handle missing values, and transform features
into a suitable format for modeling.
2. Feature Selection and Engineering:
 Features (attributes or variables) that are most relevant to the prediction task
are selected or engineered to enhance the performance of the predictive model.
3. Training a Predictive Model:
 A predictive model is trained using a portion of the available data (training
dataset). The model learns patterns and relationships between input features
and the target variable (the variable to be predicted).
4. Model Evaluation:
 The trained model is evaluated using a separate dataset (validation or test
dataset) that the model has not seen during training. Evaluation metrics, such as
accuracy, precision, recall, F1 score, or others, are used to assess the model's
performance.
5. Model Tuning:
 If the model's performance is not satisfactory, adjustments are made. This may
involve tuning hyper parameters, changing the model architecture, or modifying
the feature set.
6. Prediction on New Data:
 Once the model is trained and validated, it can be used to make predictions on
new, unseen data. These predictions are based on the patterns learned during
the training phase.
7. Deployment:
 If the predictive model meets the desired performance criteria, it can be
deployed for use in a real-world environment. This could involve integrating the
model into a software application or a decision-making system.
8. Monitoring and Updating:
Kamadhenu BCA College 65
  Predictive models need to be monitored for performance degradation over time,
especially if the data distribution changes. Models may need to be periodically
updated or retrained to maintain their accuracy.
9. Types of Predictive Models:
 Various types of predictive models can be used, including linear regression,
decision trees, support vector machines, neural networks, and ensemble
methods like random forests or gradient boosting.
10. Applications of Prediction in Data Mining:
 Predictive modeling is applied in numerous fields, including finance (credit
scoring), healthcare (disease prediction), marketing (customer churn prediction),
and many others.
The prediction process involves a combination of statistical and machine learning techniques,
and the success of the prediction task depends on the quality of the data, the choice of
features, and the appropriateness of the modeling technique for the specific problem.
Accuracy, precision, and recall are common evaluation metrics used in data mining and
machine learning to assess the performance of predictive models. These metrics provide
insights into different aspects of a model's performance, especially in binary or multiclass
classification tasks.
1. Accuracy:
 Definition: Accuracy is the ratio of correctly predicted instances to the total
number of instances in the dataset.

 Components:
 TP (True Positives): Instances correctly predicted as positive.
 TN (True Negatives): Instances correctly predicted as negative.
 FP (False Positives): Instances incorrectly predicted as positive.
 FN (False Negatives): Instances incorrectly predicted as negative.

 Interpretation: Accuracy provides an overall measure of how well a model is

performing across all classes. However, it may not be suitable for imbalanced
datasets, where one class significantly outweighs the other.
2. Precision:
 Definition: Precision is the ratio of correctly predicted positive instances to the
total predicted positive instances.


 Interpretation: Precision focuses on the accuracy of positive predictions. It is
useful when the cost of false positives is high. A high precision indicates a low
false-positive rate.
3. Recall (Sensitivity or True Positive Rate):
 Definition: Recall is the ratio of correctly predicted positive instances to the
total actual positive instances.

Kamadhenu BCA College 66

  Interpretation: Recall measures the ability of a model to identify all relevant
instances of a positive class. It is important when the cost of false negatives is
high, as it indicates the model's sensitivity to positive instances.
4. F1 Score:
 Definition: The F1 score is the harmonic mean of precision and recall, providing
a balance between the two metrics.

 Interpretation: F1 score
combines precision and recall into a single metric. It is particularly useful when
there is an uneven class distribution.
5. Receiver Operating Characteristic (ROC) Curve and Area Under the Curve (AUC):
 Definition: ROC curve plots the true positive rate (recall) against the false
positive rate at various classification thresholds. AUC quantifies the area under
the ROC curve.
 Interpretation: ROC and AUC provide insights into a model's ability to
discriminate between classes across different thresholds. A higher AUC
generally indicates better model performance.
When interpreting these metrics, it's crucial to consider the specific context of the problem and
the associated costs of false positives and false negatives. The choice of evaluation metric
depends on the goals and requirements of the task at hand.

Kamadhenu BCA College 67

 UNIT IV - CLUSTERING
INTRODUCTION
Clustering is a technique in machine learning and data analysis that involves grouping similar
data points together based on certain features or characteristics. The goal of clustering is to
partition a set of data points into subsets or clusters in such a way that data points within the
same cluster are more similar to each other than to those in other clusters. Clustering is an
unsupervised learning approach, meaning that it doesn't rely on labeled training data and the
algorithm identifies patterns and structures in the data on its own.
In clustering, a group of different data objects is classified as similar objects. One group means
a cluster of data. Data sets are divided into different groups in the cluster analysis, which is
based on the similarity of the data. After the classification of data into various groups, a label is
assigned to the group. It helps in adapting to the changes by doing the classification.
The aim is to partition a set of data points into distinct, non-overlapping groups called clusters
such that data points within a cluster have high similarity but are quite dissimilar from points in
other clusters
Cluster Analysis

Cluster Analysis in Data Mining means that to find out the group of objects which are similar to
each other in the group but are different from the object in other groups. In the process
of clustering in data analytics, the sets of data are divided into groups or classes based on data
similarity. Then each of these classes is labelled according to their data types. Going
through clustering in data mining example can help you understand the analysis more
extensively.
Cluster data mining analysis is an unsupervised learning technique that groups data points
based on their similarities. The goal is to create clusters where points within a cluster are more
similar than points in other clusters. Some key aspects of cluster analysis:
 It can be used to discover patterns in data without prior knowledge of class labels. The
algorithm explores the data and groups similar points together.
 Many clustering methods in data mining analysis algorithms include k-means,
hierarchical clustering, density-based clustering, etc. Each has its approach for defining
clusters.
 The number of classification and clustering in data mining must be determined
beforehand for some algorithms like k-means. Other algorithms, like hierarchical
clustering, can infer the number of clusters from the data.
 Choosing the appropriate clustering algorithm and tuning its parameters, like several
clusters, is key for obtaining meaningful clusters from the data.
 Cluster evaluation in data mining analysis is commonly used for exploratory data
analysis, market segmentation, social network analysis, and image segmentation.
 Cluster evaluation in data mining results requires domain knowledge and analyzing
cluster characteristics like tightness, separation, etc. External evaluation measures can
also be used if class labels are available.
Kamadhenu BCA College 68
Overall view of contents in clustering:
Two main approaches of clustering:

1. Partitioning clustering
2. Hierarchical clustering
1. Partition algorithms are:

1. PAM
2. CLARA
3. CLARANS
2. Hierarchical algorithms are

1. DBSCAN
2. CURE
3. BIRCH
Categorical data mining using clustering involves following algorithms

1. STIRR
2. ROCK
3. CACTUS

Partition methods

Partitioning Method: This clustering method classifies the information into multiple groups based
on the characteristics and similarity of the data. Its the data analysts to specify the number of
clusters that has to be generated for the clustering methods. In the partitioning method when
database(D) that contains multiple(N) objects then the partitioning method constructs user-
specified(K) partitions of the data in which each partition represents a cluster and a particular
region. There are many algorithms that come under partitioning method some of the popular
ones are K-Mean, PAM(K-Medoids), CLARA algorithm (Clustering Large Applications) etc. In this
article, we will be seeing the working of K Mean algorithm in detail.
K-Mean (A centroid based Technique): The K means algorithm takes the input parameter K from
the user and partitions the dataset containing N objects into K clusters so that resulting
similarity among the data objects inside the group (intracluster) is high but the similarity of
data objects with the data objects from outside the cluster is low (intercluster). The similarity
of the cluster is determined with respect to the mean value of the cluster. It is a type of square
error algorithm. At the start randomly k objects from the dataset are chosen in which each of
the objects represents a cluster mean(centre). For the rest of the data objects, they are
assigned to the nearest cluster based on their distance from the cluster mean

K-mean clustering:

Unsupervised Machine Learning is the process of teaching a computer to use unlabeled,

unclassified data and enabling the algorithm to operate on that data without supervision.
Without any previous data training, the machine’s job in this case is to organize unsorted data
according to parallels, patterns, and variations.
Kamadhenu BCA College 69
What is the objective of k-means clustering?

The goal of clustering is to divide the population or set of data points into a number of groups
so that the data points within each group are more comparable to one another and different
from the data points within the other groups. It is essentially a grouping of things based on
how similar and different they are to one another.
How k-means clustering works?

We are given a data set of items, with certain features, and values for these features (like a
vector). The task is to categorize those items into groups. To achieve this, we will use the K-
means algorithm, an unsupervised learning algorithm. ‘K’ in the name of the algorithm
represents the number of groups/clusters we want to classify our items into.
(It will help if you think of items as points in an n-dimensional space). The algorithm will
categorize the items into k groups or clusters of similarity. To calculate that similarity, we will
use the Euclidean distance as a measurement.

The algorithm works as follows:

1. First, we randomly initialize k points, called means or cluster centroids.
2. We categorize each item to its closest mean, and we update the mean’s coordinates,
which are the averages of the items categorized in that cluster so far.
3. We repeat the process for a given number of iterations and at the end, we have our
clusters.
The “points” mentioned above are called means because they are the mean values of the items
categorized in them. To initialize these means, we have a lot of options. An intuitive method is
to initialize the means at random items in the data set. Another method is to initialize the
means at random values between the boundaries of the data set (if for a feature x, the items
have values in [0,3], we will initialize the means with values for x at [0,3]).
K-Medoids:

Medoid: A Medoid is a point in the cluster from which the sum of distances to other data
points is minimal.

(or) A Medoid is a point in the cluster from which dissimilarities with all the other points in the
clusters are minimal.
Instead of centroids as reference points in K-Means algorithms, the K-Medoids algorithm takes
a Medoid as a reference point.
There are three types of algorithms for K-Medoids Clustering:

1. PAM (Partitioning Around Clustering)

2. CLARA (Clustering Large Applications)
3. CLARANS (Randomized Clustering Large ApplicationS
4.
1. PAM algorithm
The Partitioning Around Medoids (PAM) algorithm is a clustering algorithm that belongs to
the family of partitioning methods, similar to the more commonly known K-Means

Kamadhenu BCA College 70

 algorithm. PAM, however, is designed to be more robust to noise and outliers than K-
Means. It was introduced by Kaufman and Rousseeuw in 1987 as an improvement over K-
Means, especially when dealing with data points that might not conform well to the mean-
based approach of K-Means.

Here are the key features and steps of the PAM algorithm:
Objective:

 Like K-Means, PAM aims to partition a dataset into k clusters, where each cluster is
represented by one data point called a "medoid."
 A medoid is the most centrally located point in a cluster, minimizing the sum of
dissimilarities (or distances) between it and the other points in the cluster.
Steps:

1. Initialization:
 Choose k initial medoids randomly from the dataset.
2. Assignment:
 Assign each data point to the nearest medoid based on a dissimilarity metric
(commonly using Euclidean distance or other distance measures).
3. Update Medoids:
 For each cluster, consider swapping a non-medoid point with another non-medoid
point to minimize the total dissimilarity within the cluster.
 Repeat this process until no further improvement can be made.
4. Repeat:
 Iterate steps 2 and 3 until convergence, where convergence occurs when no more
medoid swaps lead to a reduction in total dissimilarity.

Dissimilarity Measure:

 PAM can use different dissimilarity measures depending on the nature of the data, but
commonly it uses distance measures such as Euclidean distance.
Advantages of PAM:

 More robust to noise and outliers compared to K-Means, as it uses medoids that are
less sensitive to extreme values.
 Provides a more interpretable cluster center (medoid) than K-Means, especially when t
he mean might not be a representative point.
Limitations:

 Computationally more expensive than K-Means, especially for large datasets.

 The choice of the initial medoids can impact the algorithm's performance.

Implementation:

 PAM is available in the R programming language through the cluster package, but it
might not be as commonly implemented in other languages as K-Means.
2. CLARA
CLARA (Clustering Large Applications) is an extension of the Partitioning Around Medoids
(PAM) clustering algorithm designed to handle large datasets that might not fit into
Kamadhenu BCA College 71
 memory. It was proposed by Kaufman and Rousseeuw in 1990 as a scalable version of
PAM for large datasets. CLARA uses a sampling approach to select representative subsets
of the data, applies PAM to these subsets, and then combines the results to provide an
overall clustering solution.

Here are the key features and steps of the CLARA algorithm:
Objective:

 Like PAM, CLARA aims to partition a dataset into k clusters using medoids.
 CLARA is specifically designed to handle large datasets that do not fit into memory by
using random samples.
Steps:

1. Sampling:
 Randomly select multiple samples (subsets) of the data, each of size sampsize.
 The samples are chosen without replacement, and each data point can appear
in multiple samples.
2. Apply PAM to Each Sample:
 Apply the PAM algorithm to each sampled subset. This involves selecting initial
medoids, assigning points to clusters, and updating the medoids within each
subset.
3. Evaluate Quality:
 Evaluate the quality of each clustering solution by calculating the total
dissimilarity across all points in each cluster.
4. Choose Final Clustering:
 Select the clustering solution with the lowest overall dissimilarity as the final
result.
Advantages of CLARA:

 Handles large datasets that cannot fit into memory by using random samples.
 Provides a more robust clustering solution by considering multiple samples.
Limitations:

 Computationally more intensive than standard PAM due to the need to apply PAM to
multiple samples.
 The choice of the sample size (sampsize) can impact the quality of the clustering
solution.
Implementation:

 CLARA is often implemented in the R programming language through the cluster

package, which provides functions for both PAM and CLARA.
In summary, CLARA is an extension of the PAM algorithm designed to handle large datasets by
using random samples. It provides a more scalable solution for clustering when the entire
dataset cannot be loaded into memory.
3. CLARANS (Clustering Large Applications based on Randomized Search)

Kamadhenu BCA College 72

Objectives: CLARANS is a clustering algorithm designed for large datasets. Its primary
objectives include:

1. Scalability: To efficiently handle large datasets without a substantial increase in

computational complexity.
2. Robustness: To be robust against outliers and noise in the data by employing a
randomized search strategy.
3. Flexibility: To provide flexibility in handling various types of data and clusters.
Steps: The CLARANS algorithm follows a randomized search strategy and involves the following steps:

1. Input Parameters:
 Number of clusters (k)
 Maximum number of neighbors examined (max-neighbors)
 Number of local minima (num-local)
2. Initialization:
 Randomly select k data points as initial medoids.
3. Main Loop:
 Generate a random number of neighbors for each medoid.
 Swap medoids with neighboring non-medoid points and compute the cost of the
new configuration.
 If the new configuration reduces the cost, update the medoid; otherwise, revert
the swap.
 Repeat the above steps until local minima are reached.
4. Repeat Main Loop:
 Repeat the main loop for a specified number of local minima.
5. Output:
 Return the clustering solution with the lowest cost.

Limitations:

 Sensitivity to Parameters: The performance of CLARANS is sensitive to the choice of

parameters such as max-neighbors and num-local. Suboptimal parameter values may
lead to less effective clustering.
 Not Suitable for All Datasets: CLARANS may not perform well on datasets with irregular
shapes or clusters with varying densities.
 Randomization Impact: The randomization strategy makes the algorithm non-
deterministic, and different runs may yield different results.
 Computationally Intensive: While designed for large datasets, CLARANS can still be
computationally intensive, especially for a large number of clusters or complex datasets.
Implementation:

 CLARANS can be implemented in various programming languages.

 Pseudo-code or algorithmic descriptions are available in research papers describing its
implementation details.
 Existing libraries or packages in programming languages like Python, R, or Java may
provide ready-to-use implementations.

Kamadhenu BCA College 73

 Hierarchical methods:

Hierarchical clustering is a method used in cluster analysis to build a hierarchy of clusters. It

creates a tree-like structure, called a dendrogram, that represents the relationships between
data points. There are two main types of hierarchical clustering: Agglomerative (bottom-up)
and Divisive (top-down).
1. Agglomerative Hierarchical Clustering:

 Bottom-Up Approach: It starts with each data point as a single cluster and successively
merges clusters until only one cluster, containing all data points, remains.
 Steps:
1. Treat each data point as a single cluster.
2. Find the closest (most similar) pair of clusters and merge them into a new
cluster.
3. Repeat step 2 until only one cluster remains.
 Linkage Methods: The choice of linkage method determines how the distance between
clusters is measured during the merging process. Common linkage methods include:
 Single Linkage: Distance between the closest pair of points in different clusters.
 Complete Linkage: Distance between the farthest pair of points in different
clusters.
 Average Linkage: Average distance between all pairs of points in different
clusters.
 Dendrogram: The result is a dendrogram, a tree-like diagram that illustrates the
hierarchy of clusters.

2. Divisive Hierarchical Clustering:

 Top-Down Approach: It starts with all data points in a single cluster and recursively
divides clusters until each data point is in its own cluster.
 Steps:
1. Treat all data points as a single cluster.
2. Find a cluster to split into two based on a criterion.
3. Repeat step 2 recursively until each data point is in its own cluster.
 Divisive Approach: Divisive hierarchical clustering involves recursively dividing clusters
based on certain criteria.
Advantages of Hierarchical Clustering:

 Provides an intuitive visual representation of data relationships in the form of a

dendrogram.
 No need to specify the number of clusters in advance.
 Can be used with various distance metrics and linkage methods.
Limitations of Hierarchical Clustering:

 Computationally expensive, especially for large datasets.

 The choice of linkage method and distance metric can significantly impact results.
 Once a decision is made at a certain level, it cannot be undone in agglomerative
clustering.

Kamadhenu BCA College 74

Implementation:

 Hierarchical clustering algorithms are implemented in various programming languages,

including Python, R, and MATLAB.
 Popular libraries like SciPy in Python provide functions for hierarchical clustering.
 Implementation involves choosing a linkage method, a distance metric, and interpreting
the dendrogram to determine the number of clusters or the clustering structure.
DBSCAN

Density-Based Clustering refers to one of the most popular unsupervised learning methodologies
used in model building and machine learning algorithms. The data points in the region
separated by two clusters of low point density are considered as noise. The surroundings with
a radius ε of a given object are known as the ε neighborhood of the object. If the ε
neighborhood of the object comprises at least a minimum number, MinPts of objects, then it is
called a core object.
Density-Based Clustering - Background

There are two different parameters to calculate the density-based clustering

EPS: It is considered as the maximum radius of the neighborhood.

MinPts: MinPts refers to the minimum number of points in an Eps neighborhood of that point.

NEps (i) : { k belongs to D and dist (i,k) < = Eps}

Directly density reachable:

A point i is considered as the directly density reachable from a point k with respect to Eps,
MinPts if

i belongs to NEps(k) Core

point condition:

NEps (k) >= MinPts

Density reachable:

A point denoted by i is a density reachable from a point j with respect to Eps, MinPts if there is a
sequence chain of a point i1,…., in, i1 = j, pn = i such that i i + 1 is directly density reachable from
ii.

Kamadhenu BCA College 75

Density connected:

A point i refers to density connected to a point j with respect to Eps, MinPts if there is a point o
such that both i and j are considered as density reachable from o with respect to Eps and
MinPts.

Working of Density-Based Clustering

Suppose a set of objects is denoted by D', we can say that an object I is directly density
reachable form the object j only if it is located within the ε neighborhood of j, and j is a core
object.

An object i is density reachable form the object j with respect to ε and MinPts in a given set of
objects, D' only if there is a sequence of object chains point i1,…., in, i1 = j, pn = i such that ii + 1
is directly density reachable from ii with respect to ε and MinPts.
An object i is density connected object j with respect to ε and MinPts in a given set of objects,
D' only if there is an object o belongs to D such that both point i and j are density reachable
from o with respect to ε and MinPts.

DBSCAN stands for Density-Based Spatial Clustering of Applications with Noise. It depends on a
density-based notion of cluster. It also identifies clusters of arbitrary size in the spatial
database with outliers.

Grid based clustering

Grid-based clustering is an approach to clustering data points based on a grid structure

imposed on the dataset. It involves partitioning the data space into a grid and grouping data

Kamadhenu BCA College 76

points that fall within the same grid cell. This method is particularly suitable for datasets with a
spatial or geometric nature. One common algorithm for grid-based clustering is the Density-
Based Spatial Clustering of Applications with Noise (DBSCAN), which can be adapted for grid-
based clustering.

Here are the key steps and characteristics of grid-based clustering:

1. Grid Partitioning:
 The data space is divided into a grid of cells. The size of the cells can be determined
based on the characteristics of the dataset.
 Each cell in the grid is identified by a unique identifier.
2. Data Point Assignment:
 Each data point is assigned to the grid cell corresponding to its location in the data
space.
 The assignment is based on the spatial coordinates of the data points.
3. Cluster Formation:
 Clusters are formed by grouping together data points that belong to the same grid cell.
 Adjacent grid cells with non-empty data points can also be considered for cluster
formation.
4. Grid-Based Density Estimation:
 Grid-based clustering often involves assessing the density of data points within each
grid cell.
 Cells with a sufficiently high density may be considered as part of a cluster.
5. Outlier Detection:
 Cells with low data point density can be labeled as outliers or noise.

Advantages of Grid-Based Clustering:

 Efficient for spatial datasets with a grid-like structure.

 Naturally handles datasets with varying densities.
 Suitable for identifying clusters of different shapes and sizes.
Limitations of Grid-Based Clustering:

 The choice of grid size can impact the clustering results; an inappropriate grid size may
lead to oversmoothing or undersmoothing.
 Struggles with clusters of irregular shapes or clusters with varying densities.
 Sensitive to the spatial distribution of data points.
DBSCAN as a Grid-Based Clustering Algorithm:

 DBSCAN, although not exclusively grid-based, can be adapted for grid-based clustering
by using a grid structure to efficiently compute spatial relationships.
 The algorithm defines clusters as dense regions separated by areas of lower point
density.
Implementation:

 Grid-based clustering can be implemented in various programming languages, including

Python and R.

Kamadhenu BCA College 77

  Libraries like scikit-learn in Python provide functions for clustering algorithms, including
DBSCAN.
 Custom implementations may involve defining a grid structure, assigning data points to
grid cells, and determining cluster membership based on grid-based density estimation.
Evaluation of clustering

Evaluating the performance of a clustering algorithm is crucial to understanding how well it has
grouped similar data points and how meaningful the resulting clusters are. Several metrics and
methods are commonly used to evaluate the quality of clustering. Here are some key aspects
and methods for evaluating clustering results:
1. Internal Evaluation Metrics:
 Silhouette Score: Measures how similar an object is to its own cluster compared
to other clusters. A higher silhouette score indicates better-defined clusters.
 Davies-Bouldin Index: Computes the average similarity ratio of each cluster with
the cluster that is most similar to it. Lower values indicate better clustering.
2. External Evaluation Metrics:
 Adjusted Rand Index (ARI): Measures the similarity between true labels and
predicted labels, adjusted for chance. ARI ranges from -1 to 1, where higher
values indicate better agreement.
 Normalized Mutual Information (NMI): Measures the mutual information
between true labels and predicted labels, normalized by the entropy of the
labels. Higher NMI values indicate better clustering.
3. Visual Inspection:
 Dendrogram and Scatter Plots: For hierarchical clustering, dendrograms can
provide insights into the hierarchical structure. Scatter plots of clustered data
points can visually reveal the separation of clusters.
 TSNE (t-distributed Stochastic Neighbor Embedding) or PCA (Principal
Component Analysis): Dimensionality reduction techniques can help visualize
the data in lower dimensions and inspect the separation between clusters.
4. Cluster Purity:
 Cluster Purity: Measures the extent to which each cluster contains
predominantly a single class. Higher purity values indicate better clustering
performance.
5. Fowlkes-Mallows Index:
 Fowlkes-Mallows Index: A metric that assesses the similarity between true and
predicted clusters. It considers precision and recall.
6. Computational Efficiency:
 Runtime and Memory Usage: Depending on the application, the computational
efficiency of the clustering algorithm may be an important consideration.
7. Domain-Specific Metrics:
 Metrics tailored to the specific goals of the clustering task. For example, if the
goal is to identify disease subtypes, domain-specific metrics related to disease
characteristics may be relevant.

Kamadhenu BCA College 78

 8. Robustness:
 Assessing how well the clustering results hold up under perturbations, such as
noise or variations in input parameters.
When evaluating clustering results, it's often advisable to use a combination of these metrics to
gain a comprehensive understanding of the algorithm's performance. The choice of evaluation
metric may also depend on the nature of the data and the goals of the clustering task.

Kamadhenu BCA College 79

 IMPORTANT QUESTIONS

Data Mining: Introduction and Concepts

1. What is Data Mining?

Answer:Data Mining is the process of analyzing large datasets to identify patterns, trends, and
relationships that are not immediately apparent. It uses algorithms, statistical models, and
machine learning techniques to explore hidden structures in data. The goal is to extract useful
information from raw data, which can be used to make better decisions, predict future trends,
and understand underlying structures within the data. For example, in retail, Data Mining can
help identify buying patterns of customers and suggest new products based on those patterns.

2. What are the main differences between KDD (Knowledge Discovery in Databases) and
Data Mining?

Answer:Knowledge Discovery in Databases (KDD) is the overall process of discovering

knowledge from data, which includes multiple steps such as data cleaning, integration,
transformation, mining, evaluation, and presentation of the results. Data Mining, on the other
hand, refers specifically to the process of applying algorithms to extract patterns or knowledge
from data. KDD is the broader process, and Data Mining is a crucial step within that process.
For example, in a KDD project, you would first gather data, clean it, transform it, and then
apply Data Mining techniques to extract knowledge.

3. How does Data Mining differ from DBMS (Database Management Systems)?

Answer:A Database Management System (DBMS) is a software system designed to store,

manage, and retrieve data from databases. It focuses on ensuring efficient data storage,
retrieval, and management. Data Mining, in contrast, is focused on analyzing the data stored in
a DBMS to find patterns and trends that can lead to actionable insights. While DBMS provides
tools for managing data, Data Mining uses statistical, machine learning, and computational
techniques to analyze and interpret that data. In a business context, a DBMS stores customer
data, and Data Mining analyzes that data to uncover customer purchasing behavior.

4. What are some of the key techniques used in Data Mining?

Answer:
Some of the key techniques used in Data Mining include:

1. Classification: Involves categorizing data into predefined classes or categories based on

a training dataset.
2. Clustering: Groups data points based on similarity without predefined labels.
3. Regression: Used for predicting continuous values based on historical data, such as
predicting sales revenue based on past data.
4. Association Rule Mining: Identifies relationships between variables in large datasets. A
classic example is the discovery of product associations in market basket analysis.
5. Anomaly Detection: Identifies rare or abnormal instances in the data, which can
indicate fraud, errors, or novel occurrences.
6. Sequential Pattern Mining: Discovers patterns that occur in sequences over time, such
as customer purchase behavior.

Kamadhenu BCA College 80

 5. What are the challenges associated with Data Mining?

Answer:
Data Mining comes with several challenges:

1. Data Quality: Inconsistent, incomplete, or noisy data can significantly hinder the
data mining process. Cleaning the data is often a time-consuming and essential step.
2. Scalability: As the volume of data increases, it becomes harder to apply traditional Data
Mining techniques efficiently. Handling large datasets with millions of records can
require sophisticated techniques and high computational resources.
3. Interpretability of Models: Data Mining algorithms can generate complex models that
are hard for humans to interpret, which can be a barrier to their practical use.
4. Privacy Concerns: Mining sensitive data, such as personal or financial information, can
raise privacy issues, especially in domains like healthcare or finance.
5. Data Overfitting: Overfitting occurs when a model is too closely fit to the training data
and fails to generalize well to unseen data. This is especially problematic in predictive
modeling.
6. Class Imbalance: In many datasets, some classes are underrepresented, making
it difficult to build accUrate models, especially for rare events or minority classes.

Data Warehouse: Concepts and Operations

6. What is a Data Warehouse?

Answer:A Data Warehouse is a specialized database system that stores historical data from
multiple sources in a centralized repository. Unlike operational databases that store real-time
transaction data, a Data Warehouse is designed for querying and analysis. Data Warehouses
consolidate data from various sources (such as transactional systems, flat files, or external
databases) and present it in a structured, easy-to-analyze format. They support business
intelligence activities like reporting, dashboards, and data analysis by providing a
comprehensive view of an organization's data.

7. What is the purpose of a Data Warehouse?

Answer:The primary purpose of a Data Warehouse is to enable decision-makers to analyze large

volumes of historical data in a meaningful way. By integrating data from various sources, Data
Warehouses allow users to perform complex queries and gain insights into business trends,
customer behavior, and market conditions. For instance, businesses use Data Warehouses to
track sales performance over time, monitor customer satisfaction, or assess the impact of
marketing campaigns. Data Warehouses provide the historical foundation for creating business
intelligence reports, dashboards, and predictive models.

8. Explain the concept of Multidimensional Data Model in Data Warehousing.

Answer:The Multidimensional Data Model is a way of organizing data in a Data Warehouse using
multiple dimensions to represent various aspects of the data. Data is organized in a structure
known as a cube, where each axis represents a different dimension (such as time, geography, and
product category). The intersection of dimensions holds the actual data values, such as sales
numbers or profit margins. This model allows users to analyze data from different perspectives,
such as viewing sales by month (time), by region (geography), or by product

Kamadhenu BCA College 81

category. This model is particularly useful for OLAP (Online Analytical Processing) systems,
where complex queries are needed to analyze data.

9. What is Data Cleaning, and why is it important in Data Warehousing?

Answer:Data Cleaning refers to the process of detecting and correcting (or removing) errors,
inconsistencies, and inaccuracies in the data. In the context of Data Warehousing, cleaning is
crucial because Data Warehouses often consolidate data from multiple sources, which may have
different formats, structures, and levels of quality. Cleaning ensures that the data is accurate,
complete, and consistent, which in turn enhances the quality of any analyses performed on it.
Without proper data cleaning, reports and business intelligence insights generated from the data
can be misleading or incorrect, leading to poor decision-making.

10. What is Data Integration and Transformation in Data Warehousing?

Answer:Data Integration is the process of combining data from various sources into a unified
format suitable for analysis. In Data Warehousing, this involves extracting data from disparate
systems, transforming it into a common format, and loading it into the Data Warehouse (ETL
process: Extract, Transform, Load). Data Transformation involves modifying the data to meet
the specific needs of the Data Warehouse, which may include normalizing data, converting
currencies, or dealing with missing values. Transformation ensures that the data is structured in
a way that makes it easy to analyze and query.

11. What is Data Reduction, and why is it used in Data Warehousing?

Answer:Data Reduction is a technique used to reduce the size of the data stored in a Data
Warehouse while preserving the essential patterns and relationships. Large volumes of data can
slow down query performance and complicate analysis. By applying methods like aggregation,
sampling, or dimensionality reduction, Data Reduction techniques help to streamline data
processing without sacrificing important information. For example, aggregating monthly sales
data into quarterly data reduces the amount of data while still retaining essential trends.

12. Explain Discretization in the context of Data Warehousing.

Answer:Discretization refers to the process of converting continuous data into discrete intervals
or categories. In Data Warehousing, this can be useful when working with large sets of
continuous data such as age, income, or temperature. By converting these continuous values
into discrete categories (e.g., age groups like 20-30, 30-40, etc.), it becomes easier to analyze
the data and perform aggregations. Discretization helps simplify the data and can improve the
performance of Data Mining algorithms, as many algorithms perform better with categorical
data.

Mining Frequent Patterns and Association Rules

13. What is Frequent Item Set Mining?

Answer:Frequent Item Set Mining involves discovering sets of items that frequently appear
together in transactional datasets. This is most commonly used in market basket analysis to
understand which products are often purchased together. For example, in a grocery store,
frequent item sets might reveal that customers often buy bread and butter together. The key
objective is to identify these frequently co-occurring items to help businesses with decisions like
cross-selling, promotions, and inventory management. Techniques like the A-Priori Algorithm
and FP-Growth are widely used to mine frequent item sets.
Kamadhenu BCA College 82
14. What is the A-Priori Algorithm, and how does it work?

Answer:The A-Priori Algorithm is an algorithm for mining frequent item sets. It works by
iteratively generating candidate item sets of increasing size, starting from individual items, and
checking their frequency in the dataset. The key idea behind A-Priori is that any subset of a
frequent item set must also be frequent. This property allows the algorithm to prune the search
space by eliminating infrequent item sets early in the process. A-Priori uses a breadth-first
search approach, first finding frequent individual items, then pairs of items, triples, and so on,
until no more frequent item sets can be found.

15. What is the FP-Growth Algorithm, and how does it differ from A-Priori?

Answer:The FP-Growth (Frequent Pattern Growth) algorithm is an efficient alternative to the

A-Priori algorithm for mining frequent item sets. It overcomes the limitations of A-Priori by
avoiding candidate generation, which can be computationally expensive. Instead, FP-Growth
uses a compact structure called an FP-tree (Frequent Pattern tree), which stores compressed
information about the frequent item sets. FP-Growth then extracts frequent item sets from the
FP-tree by recursively finding frequent patterns. The main advantage of FP-Growth over A-
Priori is that it performs faster and scales better for large datasets since it reduces the number of
passes over the data.

16. What are Association Rules, and how are they mined?

Answer:Association Rules are used to identify relationships between different variables in large
datasets. These rules are typically written in the form of "If X, then Y," where X and Y are
items or attributes. For example, an association rule could be "If a customer buys milk, they are
likely to buy bread." To mine Association Rules, Data Mining algorithms like A-Priori and FP-
Growth are used to find frequent item sets, and then rules are generated from those item sets
based on metrics like support, confidence, and lift. Support measures the frequency of the rule,
confidence measures the reliability of the rule, and lift measures the strength of the rule
compared to random chance.

Classification: Concepts and Algorithms

17. What is Classification in Data Mining, and why is it important?

Answer:Classification is a supervised learning technique in Data Mining where the goal is to

predict the class or category of a given instance based on a training dataset. Each training
instance is labeled with a class, and the algorithm learns to assign new, unlabeled instances to
one of the predefined classes. Classification is important because it can be applied in various
fields such as spam detection, customer segmentation, fraud detection, and medical diagnosis.
For example, a classification model can be used to predict whether an email is spam or not,
based on features like the sender, subject, and content.

18. What are the key issues in Classification?

Answer:Some of the key issues in classification include:

1. Overfitting: Overfitting occurs when a model is too complex and starts to capture noise or
irrelevant patterns in the training data. This can result in poor generalization to new, unseen
data.

Kamadhenu BCA College 83

 2. Class Imbalance: When one class is underrepresented, classifiers may be biased towards
predicting the majority class. Techniques like resampling and adjusting class weights can help
mitigate this issue.
3. Feature Selection: Selecting the right features is crucial for building accurate classification
models. Irrelevant or redundant features can reduce the model’s performance.
4. Model Complexity: There is often a trade-off between simplicity and complexity in
classification models. Simpler models may underperform, while complex models may overfit.
5. Performance Metrics: Choosing appropriate performance metrics (accuracy, precision, recall,
F1 score) depends on the classification problem at hand. For imbalanced datasets, accuracy
alone may not be sufficient.

19. What is Decision Tree Induction, and how does it work?

Answer:Decision Tree Induction is a popular classification algorithm that builds a tree-like

structure where each node represents a decision based on a feature, and each branch represents
an outcome or decision. The process begins with the root node, which represents the entire
dataset, and recursively splits it into smaller subsets based on the feature that provides the best
split (usually measured by metrics like Information Gain or Gini Impurity). The splitting
continues until the subsets are pure enough, meaning that they predominantly belong to a single
class. Decision Trees are intuitive, easy to interpret, and can handle both numerical and
categorical data. However, they are prone to overfitting if not pruned properly.

20. What is the Naive Bayes classifier, and how does it work?

Answer:The Naive Bayes classifier is a probabilistic classification algorithm based on Bayes'

Theorem, which calculates the probability of each class given the observed features. The
"naive" part of the name comes from the assumption that all features are conditionally
independent given the class label. Despite this assumption, Naive Bayes often performs
surprisingly well, especially for text classification tasks such as spam detection or sentiment
analysis.

Classification (Continued)

21. What is Rule-Based Classification?

Answer:Rule-Based Classification involves using a set of "if-then" rules to classify instances.

These rules are created based on the relationships between attributes in the training data and the
target class. A rule consists of a condition (if part) and an action (then part). For example, a
rule might say: "If age > 40 and income < 50k, then classify as 'low risk'." The main advantage
of rule-based classification is its interpretability. These models are easy to understand and
provide transparent decision-making, which is valuable in domains where explainability is
essential, such as healthcare and finance.

22. What are Lazy Learners (or Learning from your Neighbors) in Classification?

Answer:Lazy Learners, also known as Instance-Based Learning, are algorithms that do not
explicitly build a model during the training phase. Instead, they store all the training instances
and perform classification at prediction time by comparing a new instance to stored examples.
The most common Lazy Learner algorithm is the k-Nearest Neighbors (k-NN) algorithm, where
the class of a new instance is determined by the majority class of its 'k' nearest neighbors in the
training set. Lazy Learners are simple, non-parametric, and can work well for many
classification problems, especially when the decision boundaries are irregular. However, they
Kamadhenu BCA College 84
can be computationally expensive during prediction, particularly when the dataset is large.

Kamadhenu BCA College 85

 23. What is the k-Nearest Neighbors (k-NN) algorithm?

Answer:The k-Nearest Neighbors (k-NN) algorithm is a simple and widely used classification
technique where the class of a data point is determined by the majority vote of its 'k' closest
training data points. The distance between data points is usually measured using metrics like
Euclidean distance. In practice, to classify a new instance, the algorithm finds the 'k' training
examples that are closest to the instance and predicts the class that occurs most frequently
among them. The main advantage of k-NN is its simplicity and effectiveness for non-linear
data. However, it can become inefficient for large datasets, as it requires computing distances to
all points in the training set.

24. What are Precision and Recall in classification models?

Answer:Precision and Recall are two important evaluation metrics used for
classification models, particularly when dealing with imbalanced classes.

1. Precision measures the accuracy of positive predictions. It is defined as the ratio of true
positive instances to the total predicted positives. A high precision indicates that the
model does a good job of only predicting positives when it is confident that they are
correct.

1. Recall (also known as Sensitivity or True Positive Rate) measures how well the model
identifies positive instances. It is defined as the ratio of true positive instances to the
total actual positives. A high recall indicates that the model correctly identifies most of
the actual positive instances.

Both precision and recall provide valuable insights into a model's performance,
especially in cases where the data is imbalanced (e.g., rare events like fraud detection).

25. What is the F1-Score, and how is it calculated?

Answer:The F1-Score is the harmonic mean of precision and recall, providing a single
measure of a model's performance, particularly in cases where there is a class imbalance. It
balances the trade-off between precision and recall.

The F1-Score ranges from 0 (worst performance) to 1 (best performance). It is a useful metric
when both false positives and false negatives are important, such as in medical diagnostics or
fraud detection.

Clustering: Concepts and Algorithms

26. What is Cluster Analysis in Data Mining?

Answer:Cluster Analysis is an unsupervised learning technique in Data Mining that groups

similar data points into clusters. The goal is to partition data in such a way that objects within
each cluster are more similar to each other than to those in other clusters. Cluster analysis is
widely used in exploratory data analysis, customer segmentation, image recognition, and
anomaly detection. For instance, in marketing, clustering can group customers with similar
purchasing behaviors, allowing businesses to tailor their marketing strategies. Clustering can be
performed using various algorithms, including K-Means, DBSCAN, and hierarchical clustering.

27. What are the different types of Clustering Methods?

Kamadhenu BCA College 86
Answer:
There are several types of clustering methods:

1. Partitioning Methods: These methods divide the dataset into a predefined number of
clusters. K-Means is the most popular partitioning algorithm. It works by selecting 'k'
initial centroids and iteratively updating them to minimize the distance between data
points and their corresponding centroids.
2. Hierarchical Methods: These methods build a tree-like structure (dendrogram) of
clusters, which can be either agglomerative (bottom-up) or divisive (top-down).
Agglomerative clustering starts by treating each data point as its own cluster and then
merges the closest clusters. Divisive clustering starts with all data points in one cluster
and iteratively splits it into smaller clusters.
3. Density-Based Methods: These methods, such as DBSCAN (Density-Based Spatial
Clustering of Applications with Noise), group points that are closely packed together
and separate regions with sparse data. DBSCAN can handle clusters of arbitrary
shapes and can also identify noise points (outliers).
4. Grid-Based Methods: These methods divide the data space into a finite number of cells
and perform clustering based on the grid structure. They are particularly useful for
large datasets with high dimensionality.

28. What is the K-Means clustering algorithm, and how does it work?

Answer:The K-Means algorithm is a partitioning clustering algorithm that divides a dataset into
'k' clusters. It works by first selecting 'k' initial centroids (either randomly or based on some
heuristic) and then assigning each data point to the nearest centroid. Once all points are assigned,
the centroids are recalculated as the mean of all points in their respective clusters. This process
is repeated iteratively until the centroids no longer change, meaning the algorithm has converged.
K-Means is efficient and easy to implement but has limitations, such as sensitivity to the initial
choice of centroids and difficulty handling non-spherical clusters.

29. What are the advantages and disadvantages of the K-Means algorithm?

Answer:

1. Advantages:

1. Simple to implement and computationally efficient.

2. Works well when clusters are well-separated and spherical.
3. Scales well to large datasets.

2. Disadvantages:

1. Sensitive to the initial choice of centroids and can get stuck in local minima.
2. Requires the user to specify the number of clusters ('k') in advance.
3. Struggles with clusters of varying sizes, densities, and shapes.
4. Sensitive to noise and outliers, which can distort the clustering results.

30. What is DBSCAN (Density-Based Spatial Clustering of Applications with Noise)?

Kamadhenu BCA College 87

 Answer:DBSCAN is a density-based clustering algorithm that groups together points that are
closely packed and separates regions with sparse data. Unlike K-Means, DBSCAN does not
require the user to specify the number of clusters in advance. It relies on two key parameters: the
radius around each point and the minimum number of points required to form a dense region
(MinPts). DBSCAN is particularly effective for clustering data with irregular shapes and can
also identify noise points, which do not belong to any cluster. However, DBSCAN can struggle
with clusters of varying densities.

31. What are the evaluation metrics for clustering?

Answer:
There are several evaluation metrics used to assess the quality of clustering results:

1. Silhouette Score: Measures how similar each point is to its own cluster compared to
other clusters. A high silhouette score indicates that the points are well-clustered.
2. Dunn Index: Measures the compactness and separation of clusters. A higher Dunn
Index indicates better clustering.
3. Davies-Bouldin Index: A lower Davies-Bouldin score indicates better clustering, as it
measures the average similarity between each cluster and its most similar cluster.
4. Rand Index: Measures the agreement between two clusterings, by comparing pairs of
points and counting the number of pairs that are either in the same cluster or different
clusters in both clusterings.
5. Adjusted Rand Index (ARI): Adjusts the Rand Index to account for chance, making
it more reliable for comparing clustering results.

32. Apriori Algorithm Problem

Problem 1:

Consider the following transaction dataset:

Transaction ID Items
Purchased T1 A, B, D
T2 B, C, D
T3 A, B, C, D
T4 A, B, C
T5 B, D
T6 A, C, D

We are given the minimum support threshold as 50% (0.5), and the minimum confidence
threshold as 70% (0.7).

Solution:

Step 1: Generate Candidate 1-itemsets The frequency of individual items in the transactions
are:

1. A: 3 times
2. B: 4 times

Kamadhenu BCA College 88

 3. C: 4 times
4. D: 4 times

The support count for each item:

1. Support(A) = 3/6 = 0.5

2. Support(B) = 4/6 = 0.67
3. Support(C) = 4/6 = 0.67
4. Support(D) = 4/6 = 0.67

Frequent 1-itemsets (support >= 50%):

1. A: 0.5, B: 0.67, C: 0.67, D: 0.67

Step 2: Generate Candidate 2-itemsets The 2-itemsets are:

1. (A, B) appears in transactions T1, T3, and T4 → 3/6 = 0.5

2. (A, C) appears in transactions T3, T4, and T6 → 3/6 = 0.5
3. (A, D) appears in transactions T1, T3, and T6 → 3/6 = 0.5
4. (B, C) appears in transactions T2, T3, T4, and T5 → 4/6 = 0.67
5. (B, D) appears in transactions T1, T2, T5, and T6 → 4/6 = 0.67
6. (C, D) appears in transactions T2, T3, T4, and T6 → 4/6 = 0.67

Frequent 2-itemsets (support >= 50%):

1. (A, B): 0.5, (A, C): 0.5, (A, D): 0.5, (B, C): 0.67, (B, D): 0.67, (C, D): 0.67

Step 3: Generate Candidate 3-itemsets The 3-itemsets are:

1. (A, B, C) appears in T3 and T4 → 2/6 = 0.33

2. (A, B, D) appears in T1 and T3 → 2/6 = 0.33
3. (A, C, D) appears in T3 and T6 → 2/6 = 0.33
4. (B, C, D) appears in T2, T3, T4, and T6 → 4/6 = 0.67

Frequent 3-itemsets (support >= 50%):

1. (B, C, D): 0.67

Step 4: Generate Association Rules For the frequent itemsets, we generate rules and
calculate their confidence:

1. Rule 1: (B, C) → D, Confidence = Support(B, C, D) / Support(B, C) = 0.67 / 0.67 = 1.0

(Confidence = 100% → satisfies threshold)
2. Rule 2: (B, D) → C, Confidence = Support(B, C, D) / Support(B, D) = 0.67 / 0.67 =
1.0 (Confidence = 100% → satisfies threshold)

Thus, the frequent itemsets and their corresponding association rules have been mined.

33. FP-Growth Algorithm Problem

Kamadhenu BCA College 89
 Problem 2:

Consider the following transaction dataset:

Transaction ID Items
Purchased T1 A, B, D
T2 B, C, D
T3 A, B, C, D
T4 A, B, C
T5 B, D
T6 A, C, D

We are given a minimum support threshold of 50% (0.5), and the goal is to mine frequent
itemsets using the FP-Growth algorithm.

Solution:

Step 1: Construct the Frequency Table The frequency of individual items is calculated:

1. A: 3 times
2. B: 4 times
3. C: 4 times
4. D: 4 times

The support for each item is:

1. Support(A) = 3/6 = 0.5

2. Support(B) = 4/6 = 0.67
3. Support(C) = 4/6 = 0.67
4. Support(D) = 4/6 = 0.67

The frequent items (support >= 50%) are A, B, C, and D.

Step 2: Sort Items in Transaction Data After sorting the items in each transaction based on
their frequency (from most frequent to least frequent), the transaction dataset looks like this:

Transaction ID Sorted
Items T1 B, A, D
T2 B, D, C
T3 B, A, D, C
T4 B, A, C
T5 B, D
T6 A, C, D

Step 3: Build the FP-tree Construct the FP-tree by iterating through the sorted transactions and
inserting them into the tree:

Kamadhenu BCA College 90

 1. Start with an empty tree.
2. Add items based on their frequency and count the occurrences of itemsets in the path.
3. Branch out as necessary for frequent itemsets.

After inserting all transactions, the FP-tree looks like this:

mathematica
CopyEdit
Root
|
B (5)
/ | \
A D C (4)
/\ \ |D C C D (3)
|
A (2)

Step 4: Mine Frequent Itemsets To mine the frequent itemsets, perform a conditional pattern
base mining:

1. For item D, the conditional pattern base contains: {(B, A), (B, C), (A, C)}.
2. From this, the frequent itemsets for D are {B, D}, {A, D}, and {B, A, D}.

Repeat the process for other items like B, C, and A.

The final frequent itemsets are:

1. {A, B}
2. {B, D}
3. {A, D}
4. {A, B, D}
5. {C, D}

Step 5: Generate Association Rules From the frequent itemsets, generate association rules and
calculate their confidence:

1. Rule 1: {B} → {D}, Confidence = Support(B, D) / Support(B) = 0.67 / 0.67 = 1.0

2. Rule 2: {A} → {D}, Confidence = Support(A, D) / Support(A) = 0.67 / 0.5 = 1.34
(confidence exceeds threshold)
3. Rule 3: {B, A} → {D}, Confidence = Support(A, B, D) / Support(A, B) = 0.5 / 0.5 = 1.0

Thus, the frequent itemsets and association rules have been mined using the FP-Growth
algorithm.

Kamadhenu BCA College 91