BIG DATA & ANALYTICS (ELECTIVE)

Unit -III

Introduction to Classification and Prediction

Classification and prediction are two essential forms of data analysis in data mining.
Classification deals with categorical labels (discrete, unordered), while prediction handles
continuous-valued functions. For example, classification can be used to categorize bank loan
applications as safe or risky, while prediction can forecast customer expenditures based on their
income and occupation.
The main difference between classification and prediction:
 Classification: Predicts discrete, categorical labels
 Prediction: Models continuous-valued functions or ordered values
Issues Regarding Classification
Several key preprocessing steps are crucial for effective classification:
1. Data Cleaning
 Removes or reduces noise using smoothing techniques
 Handles missing values by replacing them with most common or statistically probable
values
 Helps reduce confusion during the learning process
2. Relevance Analysis
 Identifies and removes redundant attributes through correlation analysis
 Performs attribute subset selection to maintain the original probability distribution
 Improves classification efficiency and scalability
3. Data Transformation and Reduction
 Involves normalization of values (especially for neural networks)
 Generalizes data to higher-level concepts using concept hierarchies
 Can use techniques like wavelet transformation, principle components analysis, and
discretization
Classification using Decision Trees
Decision tree classification is one of the most intuitive and widely used classification
methods in data mining. A decision tree is structured like a flowchart, where internal nodes
represent tests on attributes, branches represent the outcomes of these tests, and leaf nodes
represent class labels. The classification process starts at the root node and follows the
appropriate branches based on the attribute values of the instance being classified until
reaching a leaf node that provides the predicted class.
How Decision Trees Work
The construction of a decision tree follows a top-down approach, starting with the entire
dataset and recursively partitioning it into smaller subsets. At each step, the algorithm selects
the "best" attribute to split the data based on measures like information gain or Gini index.
This process continues until all instances in a node belong to the same class or no further
splitting is possible.
Example: Loan Approval Decision Tree
Consider a bank making loan approval decisions based on customer data:

In this example:
 First node checks income level
 Second level examines debt or credit score depending on income
 Leaf nodes provide the final decision (Approve/Reject)
 Each path from root to leaf represents a classification rule
Bayesian Classification
Bayesian classification is based on Bayes' theorem of probability and assumes that the effect
of an attribute value on a given class is independent of the values of other attributes. This
approach is particularly effective when dealing with large datasets and can handle missing
values by ignoring them during probability estimates.
How Bayesian Classification Works
The Naive Bayesian classifier calculates the probability of an instance belonging to each
possible class and selects the class with the highest probability. The probability calculation
uses Bayes' theorem:
P(Class|Data) = P(Data|Class) × P(Class) / P(Data)
Example: Email Spam Classification
Consider a simple email spam classification:
Words in Email: "win", "money", "free"

Calculate probability of spam vs. legitimate:

P(Spam|"win,money,free") ∝ P("win"|Spam) × P("money"|Spam) × P("free"|Spam) ×
P(Spam)
P(Legitimate|"win,money,free") ∝ P("win"|Legitimate) × P("money"|Legitimate) × P("free"|
Legitimate) × P(Legitimate)

Given historical probabilities:

P("win"|Spam) = 0.6 P("win"|Legitimate) = 0.05
P("money"|Spam) = 0.8 P("money"|Legitimate) = 0.10
P("free"|Spam) = 0.7 P("free"|Legitimate) = 0.15
P(Spam) = 0.3 P(Legitimate) = 0.7

Calculating:
P(Spam|words) ∝ 0.6 × 0.8 × 0.7 × 0.3 = 0.1008
P(Legitimate|words) ∝ 0.05 × 0.10 × 0.15 × 0.7 = 0.000525
Since 0.1008 > 0.000525, classify as Spam

Classification by Backpropagation
Backpropagation is a neural network learning algorithm that learns by iteratively processing a
dataset of training tuples, comparing the network's prediction for each tuple with the actual
known target value. The network adjusts its weights after each prediction to minimize the
error in its predictions.
How Neural Networks Work
A typical neural network consists of:
1. Input Layer: Receives the initial data
2. Hidden Layer(s): Processes the data through weighted connections
3. Output Layer: Produces the final classification
The learning process involves:
1. Forward propagation of input
2. Calculation of error at output
3. Backward propagation of error to adjust weights
4. Repeated iterations until convergence
Example: Customer Purchase Prediction
Consider a neural network predicting whether a customer will purchase a product:
Input Layer:
- Age (normalized to 0-1)
- Income (normalized to 0-1)
- Previous purchases (0-1)

Hidden Layer: 4 neurons

Output Layer: 1 neuron (Purchase: Yes/No)

Sample calculation for one neuron:

Input values: [0.35, 0.78, 0.62]
Weights: [0.4, 0.3, 0.5]
Bias: 0.2

Net input = (0.35 × 0.4) + (0.78 × 0.3) + (0.62 × 0.5) + 0.2 = 0.764
Output = sigmoid(0.764) = 0.683

Prediction Classification
Prediction classification deals with continuous-valued functions and often uses regression
analysis. Unlike standard classification that predicts discrete classes, prediction classification
estimates numerical values.
Types of Regression
1. Linear Regression: Models relationship between one dependent and one independent
variable
 2. Multiple Regression: Involves multiple independent variables
3. Nonlinear Regression: For relationships that aren't linear
Example: House Price Prediction
Using multiple linear regression:
Price = β₀ + β₁(Square_Footage) + β₂(Num_Bedrooms) + β₃(Age_of_House)
Given data:
Square_Footage = 2000
Num_Bedrooms = 3
Age_of_House = 15
If coefficients are:
β₀ = 50,000
β₁ = 100
β₂ = 15,000
β₃ = -1,000

Price = 50,000 + (100 × 2000) + (15,000 × 3) + (-1,000 × 15)

= 50,000 + 200,000 + 45,000 - 15,000
= $280,000

Best Practices for Classification

1. Data Preparation:
• Clean the data (remove noise and handle missing values)
• Normalize numerical attributes when needed
• Encode categorical attributes appropriately
2. Model Selection:
• Use decision trees when interpretability is important
• Choose Naive Bayes for text classification and when attributes are
independent
• Use neural networks for complex patterns and continuous values
• Apply regression for numerical predictions
3. Evaluation:
• Use cross-validation to assess model performance
• Consider multiple metrics (accuracy, precision, recall, F1-score)
• Test models on independent test sets
 • Monitor for overfitting
4. Implementation:
• Start with simpler models and gradually increase complexity if needed
• Document assumptions and limitations
• Regularly update models with new data
• Consider computational resources and time constraints

Introduction to Clustering
Clustering is a fundamental data mining technique that focuses on grouping similar objects
together while ensuring dissimilar objects remain in different groups. In essence, clustering
creates meaningful groups of data where the objects within each cluster share common
characteristics or patterns. This technique is particularly valuable in data analysis as it helps
identify natural groupings within data without prior knowledge of the groups' characteristics.
Clustering serves multiple purposes, including data compression (by representing many data
points with fewer cluster centers) and pattern recognition (by identifying recurring patterns in
data).
Types of Clustering Methods
1. Partitioning Methods
Partitioning methods divide data into k non-overlapping partitions where each partition
represents a cluster. The k-means algorithm is the most well-known partitioning method,
which iteratively assigns data points to the nearest cluster center and updates these centers
based on the mean of all points in each cluster. K-medoids, another partitioning method, is
more robust to outliers as it uses actual data points as cluster centers instead of mean values.
2. Hierarchical Methods
Hierarchical clustering creates a tree-like structure of clusters, offering multiple levels of
granularity. There are two approaches: agglomerative (bottom-up), which starts with
individual objects and progressively merges them into clusters, and divisive (top-down),
which begins with all objects in one cluster and recursively divides them. This method is
particularly useful when you need to understand the hierarchical relationships between data
points.
3. Density-Based Methods
Density-based clustering methods identify clusters as dense regions separated by regions of
lower object density. These methods are particularly effective at finding clusters of arbitrary
shapes and can naturally handle outliers. DBSCAN (Density-Based Spatial Clustering of
Applications with Noise) is a popular density-based algorithm that can discover clusters of
various shapes and sizes.
4. Grid-Based Methods
Grid-based clustering methods divide the data space into a grid structure of cells. Clustering
operations are performed on this grid structure, making these methods particularly efficient
for very large datasets. The processing time typically depends on the number of cells in the
grid rather than the number of data objects.
5. Model-Based Methods
Model-based clustering methods assume that the data is generated from a mixture of
probability distributions. Each cluster corresponds to a different probability distribution.
These methods can automatically determine the number of clusters and handle noise in the
data.
Spatial Mining
Spatial data mining is a specialized branch of data mining that focuses on extracting
knowledge from spatial data, which includes geographic locations, geometric spaces, and
spatial relationships. This field combines traditional data mining techniques with spatial
analysis methods to discover patterns and relationships that are influenced by geographic
proximity and spatial arrangement.
Spatial mining involves complex analyses that consider both spatial and non-spatial
attributes. For example, when analyzing retail store performance, spatial mining might
consider not just sales figures (non-spatial) but also store location, proximity to competitors,
and local demographics (spatial attributes). The technique incorporates specialized distance
metrics and spatial statistics to account for geographic relationships.
Applications of Spatial Mining
Spatial mining finds extensive applications in various fields:
 Geographic Information Systems (GIS) use spatial mining for mapping and spatial
analysis
 Urban planners employ it to optimize city layouts and transportation networks
 Resource management benefits from spatial mining in identifying optimal locations
for facilities
 Environmental studies use it to track and predict patterns in climate change and
pollution
 Location-based services rely on spatial mining for providing context-aware
recommendations
Web Mining
Web mining is the application of data mining techniques to discover patterns from the World
Wide Web. It encompasses three distinct categories, each focusing on different aspects of
web data analysis:
Web Content Mining
Web content mining focuses on extracting useful information from web page contents. This
includes analyzing text, images, audio, video, metadata, and hyperlinks. The process involves
techniques from text mining, image processing, and natural language processing to
understand and categorize web content. For example, search engines use content mining to
index web pages and understand their relevance to search queries.
Web Structure Mining
Web structure mining analyzes the hyperlink structure of the web to understand relationships
between websites. This involves studying how web pages are connected through hyperlinks
and identifying important or influential pages. PageRank, used by Google, is a famous
example of web structure mining that determines page importance based on its connection
patterns.
Web Usage Mining
Web usage mining focuses on analyzing how users interact with websites. This includes
studying web server logs, click streams, and user session data to understand navigation
patterns and behavior. The insights gained are valuable for improving website design,
personalizing content, and optimizing user experience. E-commerce sites particularly benefit
from this to understand customer behavior and improve conversion rates.
Text Mining
Text mining, also known as text analytics, is the process of deriving high-quality information
from text. It involves analyzing large collections of text documents to discover patterns and
trends. Text mining combines techniques from linguistics, statistics, and machine learning to
transform unstructured text into structured data that can be analyzed.
Key Processes in Text Mining
Text mining encompasses several key processes:
 Document classification systematically organizes documents into predefined
categories
 Document clustering groups similar documents together without predefined categories
 Information extraction identifies and extracts specific facts and relationships from text
 Topic modeling discovers abstract topics that occur in a collection of documents
  Sentiment analysis determines the emotional tone and opinions expressed in text
Applications of Text Mining
Text mining has numerous practical applications:
 Email filtering systems use text mining to identify spam and categorize messages
 Document organization systems automatically classify and organize large document
collections
 Content recommendation systems analyze text to suggest relevant content to users
 Market intelligence applications monitor and analyze market trends through text
analysis
 Customer feedback analysis helps companies understand customer sentiment and
preferences

Each of these applications builds on text mining's ability to process and analyze large
volumes of unstructured text data, making it an invaluable tool in today's data-rich
environment.