1.

Explain three classes of algorithms and broad generalizations in

machine learning.

Three Classes of Machine Learning Algorithms

Machine learning algorithms can be broadly categorized into three main
learning approaches:
1. Supervised Learning: Imagine a teacher showing you labeled
examples. In supervised Learning, the algorithm is trained on data
that has already been labelled with the desired Output. The
algorithm learns the relaƟonship between the input data (features)
and the Labelled output (target variable). This allows the model to
make predicƟons on new, unseen Data. Common supervised
learning tasks include:
O ClassificaƟon: Classifying emails as spam or not spam.
O Regression: PredicƟng house prices based on size and locaƟon.

2. Unsupervised Learning: Unlike supervised learning, unsupervised

learning deals With unlabeled data. The algorithm tries to find
hidden patterns or structures within The data itself. This can be
useful for tasks like:
O Clustering: Grouping similar customers together based on their
purchase history.
O Dimensionality ReducƟon: Reducing the number of features in a dataset
while Preserving important informaƟon.

3. Reinforcement Learning: This is inspired by how animals learn

through trial and Error. The algorithm interacts with an environment
and receives rewards or penalties For its actions. Over time, the
algorithm learns to take actions that maximize the Reward.
Reinforcement learning is used in applications like:
O Training a self-driving car to navigate roads.
O Training an AI agent to play complex games.

Broad GeneralizaƟons in Machine Learning

Here are some key concepts that apply broadly across machine learning
algorithms:
 Training and TesƟng: Machine learning models are built by training them
on a large dataset. This dataset is then split into training and tesƟng sets.
The model is trained on the training Data and its performance is evaluated
on the unseen tesƟng data. This helps assess how well The model
generalizes to new data.
 Overfiƫng and Underfiƫng: A well-trained model should be able to make
accurate predicƟons on both the training data and unseen data. Overfiƫng
occurs when a model memorizes the training data too well and performs
poorly on unseen data. Underfiƫng occurs when a model is too simple and
cannot learn the underlying paƩerns in the data. Techniques like
regularizaƟon are used to prevent these issues.
 Feature Engineering: The features you choose to represent your data
can significantly impact The performance of your model. Feature
engineering involves selecƟng, transforming, and creaƟng new features
that best capture the relevant informaƟon for the task at hand.
2. Explain linear regression technique with an example and its evaluation
metric.

Linear regression is a statistical method used to model the relationship

between a
dependent variable (target) and one or more independent variables
(predictors). The goal is to
find the best-fitting straight line (or hyperplane in higher dimensions) that
minimizes the
difference between the predicted values and the actual values.
Example
Problem Statement: Let's consider a simple example where we want to
predict the annual
salary of employees based on their years of experience.
Dataset: Suppose we have the following data:
Years of Experience Salary ($)
1 40,000
2 45,000
3 50,000
4 55,000
5 60,000
Steps:
1. Plot the Data:
o Plot a scatter plot of the data points.
2. Fit the Linear Model:
o The linear regression model can be represented as: y=β0+β1xy = \
beta_0 +
\beta_1 xy=β0+β1x where yyy is the salary, xxx is the years of
experience,
β0\beta_0β0 is the y-intercept, and β1\beta_1β1 is the slope of the line.
3. Estimate the Coefficients:
o Using methods such as Ordinary Least Squares (OLS), we estimate
β0\beta_0β0 and β1\beta_1β1.
o For our dataset, we might find: β0=35,000(intercept)\beta_0 = 35,000 \
quad
\text{(intercept)}β0=35,000(intercept) β1=5,000(slope)\beta_1 = 5,000 \
quad
\text{(slope)}β1=5,000(slope)
4. Predict Salary:
Using the model, we can predict the salary for a given number of years of
experience. For example, for 6 years of experience:
Predicted Salary=35,000+5,000×6=65,000\text{Predicted Salary} =
35,000 +
5,000 \times 6 = 65,000Predicted Salary=35,000+5,000×6=65,000
Evaluation Metric
The most common evaluation metric for linear regression is the Mean
Squared Error
(MSE), but other metrics such as R-squared are also widely used.
1. Mean Squared Error (MSE):
o MSE measures the average squared difference between the actual
values and
the predicted values. It is calculated as:
where n is the number of data points, yi_ is the actual value, and
y^i\hat{y}_iy^i is the predicted value.
o For our example, assuming the actual and predicted values are as
follows:
Years of Experience Actual Salary ($) Predicted Salary ($)
1 40,000 40,000
2 45,000 45,000
3 50,000 50,000
4 55,000 55,000
5 60,000 60,000
o The MSE would be:
MSE=15((40,000−40,000)2+(45,000−45,000)2+(50,000−50,000)2+(55,0
00−5
5,000)2+(60,000−60,000)2)=0\text{MSE} = \frac{1}{5} \left( (40,000 -
40,000)^2 + (45,000 - 45,000)^2 + (50,000 - 50,000)^2 + (55,000 -
55,000)^2
+ (60,000 - 60,000)^2 \right) = 0MSE=51
((40,000−40,000)2+(45,000−45,000)2+(50,000−50,000)2+(55,000−55,
000)2+
(60,000−60,000)2)=0
2. R-squared (Coefficient of Determination):
o R-squared measures the proportion of the variance in the dependent
variable
that is predictable from the independent variable(s). It is calculated as:
where yˉ\bar{y}yˉ is the mean of the actual values.

3. Explain KNN technique with an example and its similarity metric.

K-Nearest Neighbors (KNN) is a simple, non-parametric, and instance-

based
learning algorithm used for both classification and regression tasks. In
KNN, predictions for a
new data point are made based on the majority label (classification) or
average value
(regression) of the kkk-nearest data points from the training set.
Example
Problem Statement: Suppose we want to classify a new data point into
one of two
categories based on its features. Let's use a simple 2D example where we
want to classify
flowers based on petal length and petal width.
Dataset: Imagine we have a small dataset with the following points:
Petal Length Petal Width Flower Type
1.0 0.5 A
2.0 1.0 A
3.0 1.5 B
4.0 2.0 B
5.0 2.5 B
Steps:
1. Choose the Value of kkk:
o Select the number of nearest neighbors to consider (e.g., k=3k = 3k=3).
2. Calculate the Distance:
o For a new data point (e.g., Petal Length = 3.5, Petal Width = 1.8),
calculate the
distance to all points in the training set.
3. Find the Nearest Neighbors:
o Identify the kkk-nearest neighbors based on the distance metric.
4. Make the Prediction:
o For classification, assign the new point to the majority class among the
kkk-
nearest neighbors.
o For our new point, if the 3 nearest neighbors are (3.0, 1.5), (4.0, 2.0),
and (5.0,
2.5), which belong to classes B, B, and B respectively, the new point is
classified as B.
Similarity Metric
The most common distance metric used in KNN is the Euclidean distance,
but other metrics
such as Manhattan distance, Minkowski distance, and cosine similarity can
also be used
depending on the nature of the data and problem.

4 Explain K Means technique with an example.

K-Means is an unsupervised learning algorithm used to partition a dataset

into
KKK distinct, non-overlapping subsets (clusters). Each data point is
assigned to the cluster With the nearest mean, serving as a prototype of
the cluster.
Example
Problem Statement: Suppose we want to group a set of points into K=2K
= 2K=2 clusters Based on their coordinates in a 2D space.
Dataset: Imagine we have the following points:
Point X Y
A12
B23
C34
D56
E88
Steps:
1. Choose the Number of Clusters KKK:
O Select K=2K = 2K=2 for this example.
2. Initialize Centroids:
O Randomly select KKK points as initial centroids. For instance, let’s
choose
Points A (1,2) and E (8,8).
3. Assign Points to the Nearest Centroid:
O Calculate the distance of each point from the centroids and assign each
point
To the nearest centroid.
O Using the Euclidean distance:
 Distance from A to (1,2): 0 (closest)
 Distance from A to (8,8): (1−8)2+(2−8)2=49+36=85\sqrt{(1-8)^2 + (2-
8)^2} = \sqrt{49 + 36} = \sqrt{85}(1−8)2+(2−8)2=49+36=85

 Continue this for all points.

4. Update Centroids:
O Calculate the new centroids as the mean of all points assigned to each
cluster.
O Suppose after initial assignment, points A, B, and C are closer to
centroid (1,2) And points D and E are closer to centroid (8,8).
5. Reassign Points Based on New Centroids:
O Repeat the process of assigning points to the nearest centroid with the
updated Centroids.
6. Iterate Until Convergence:
O Repeat steps 3 and 4 until the centroids no longer change significantly
or the
Assignments remain the same.

5 Explain user retention

User Retention refers to the ability of a company or product to keep its

users over a specified period of time. It is a key metric for businesses,
especially in the context of digital products and services, as it directly
impacts revenue, growth, and overall success. Higher retention rates
typically indicate that users find value in the product or service, leading to
long-term engagement and loyalty.
Importance of User Retention
1. Revenue Growth: Retained users are more likely to make repeat
purchases or
subscriptions, driving sustained revenue.
2. Customer Lifetime Value (CLV): Long-term users contribute more to
the business over their lifetime.
3. Word of Mouth: Satisfied users are more likely to recommend the
product or service to others.
4. Cost Efficiency: Retaining existing users is often less expensive than
acquiring new ones.
Key Metrics to Measure User Retention
1. Retention Rate: The percentage of users who return to the product or
service after a specific period.
2. Churn Rate: The percentage of users who stop using the product or
service over a
given period.
3. Customer Lifetime Value (CLV): The total revenue expected from a user
over their
entire relationship with the company.
4. Daily/Weekly/Monthly Active Users (DAU/WAU/MAU): Metrics that track
the
number of unique users who engage with the product within a given
timeframe.
Strategies to Improve Retention:
Here are some general strategies to keep users engaged:
 Provide Value: Ensure your product solves a real problem or fulfills a
genuine need
for users.
 Great User Experience: Make your product user-friendly, intuitive, and
enjoyable to
interact with.
 Onboarding & Tutorials: Guide new users through your product's features
and
benefits.
 Ongoing Engagement: Offer fresh content, features, or rewards to keep
users coming back.
 Personalization: Tailor the user experience to individual needs and
preferences.
 Community Building: Foster a sense of community among users to
increase
stickiness.
Example of User Retention
Scenario: A Mobile Fitness App
1. Onboarding:
o The app provides a personalized fitness plan based on the user's goals
and
fitness level.
o Interactive tutorials show how to use the app's features effectively.
2. Engagement:
o Daily reminders and motivational messages encourage users to stick to
their
fitness routines.
o Regular updates introduce new workouts and features.
3. Customer Support:
o Users have access to live chat support and a comprehensive help center
with
workout guides and troubleshooting tips.
4. User Feedback:
O The app surveys users regularly to gather feedback on workouts and
features.
O Feedback is analyzed and used to make continuous improvements.
5. Loyalty Programs:
O Users earn points for completing workouts, which can be redeemed for
Discounts on premium subscriptions or fitness gear.
6. Community Building:
O The app hosts online challenges where users can compete and share
their
Progress on social media.
O A dedicated forum allows users to connect, share tips, and support each
other.

6. Explain feature extraction and selection.

Feature Extraction and Feature Selection are key processes in the

preprocessing phase of machine learning. Both aim to reduce the
dimensionality of the dataset while retaining the most important
information, but they achieve this in different ways.
Feature Extraction
Definition: Feature extraction involves transforming the data from its
original high-
dimensional space into a lower-dimensional space. The goal is to create
new features that are a combination or transformation of the original
features, retaining as much relevant information as possible.
Techniques:
1. Principal Component Analysis (PCA):
PCA reduces the dimensionality of data by transforming it into a set of
orthogonal (uncorrelated) components, ordered by the amount of variance
they
explain in the data.
o The first few principal components usually capture most of the variance,
allowing for dimensionality reduction while preserving the essential
structure.
2. Linear Discriminant Analysis (LDA):
o LDA is a supervised technique that finds a linear combination of features
that
best separates two or more classes. It projects the data in a way that
maximizes
the separation between classes.
3. Independent Component Analysis (ICA):
o ICA separates a multivariate signal into additive, independent
components. It
is often used for applications such as signal processing and image
analysis.
4. t-Distributed Stochastic Neighbor Embedding (t-SNE):
o t-SNE is a non-linear technique used primarily for data visualization. It
reduces dimensionality while preserving the local structure of the data.
5. Autoencoders:
o Autoencoders are a type of neural network used to learn efficient
codings of
input data. They can be used for dimensionality reduction by training the
network to compress the input into a lower-dimensional representation
and
then reconstruct the original input from this representation.

Feature Selection
Definition: Feature selection involves selecting a subset of the original
features based on certain criteria, aiming to keep the most relevant
features while discarding the redundant or irrelevant ones. This can
improve the performance of machine learning models by reducing
overfitting, improving accuracy, and decreasing computational cost.
Techniques:
1. Filter Methods:
o These methods use statistical techniques to evaluate the relevance of
each
feature independently of any machine learning model. Examples include:
 Correlation Coefficient: Measures the linear correlation between
features and the target variable.
 Chi-Squared Test: Evaluates the independence of features from the
target variable.
 ANOVA F-test: Used for feature selection with continuous features
and categorical targets.
2. Wrapper Methods:
o These methods use a predictive model to evaluate the combination of
features
and select the best subset based on model performance. Examples
include:
 Recursive Feature Elimination (RFE): Iteratively builds a model and
removes the least important features.
Forward/Backward Selection: Starts with an empty set of features
and adds/removes features based on model performance.
3. Embedded Methods:
o These methods perform feature selection during the model training
process.
Examples include:
 Lasso Regression (L1 Regularization): Adds a penalty to the
regression model that can shrink some coefficients to zero, effectively
performing feature selection.
 Tree-based Methods: Decision trees and random forests provide
feature importance scores that can be used to select features.

7. Explain filters wrappers and embedded methods.

1. Filter Methods
Definition: Filter methods evaluate the relevance of features
independently of any machine learning model. They use statistical
techniques to assess the relationship between each feature and the target
variable, ranking features based on their scores.
Techniques:
 Correlation Coefficient: Measures the linear relationship between
features and the
target variable. Features with high correlation with the target are
considered more
relevant.
 Chi-Squared Test: Evaluates the independence of categorical features
from the target variable. Features with lower p-values are more relevant.
 ANOVA F-test: Used for continuous features and categorical targets. It
measures the variance between groups and within groups.
 Mutual Information: Measures the amount of information one feature
provides
about the target variable. Higher values indicate more relevant features.
Pros:
 Computationally efficient and fast.
 Simple to understand and implement.
 Suitable for very high-dimensional datasets.
Cons:
 Ignores interactions between features.
 Can select redundant features that are individually relevant but
collectively less
informative.
2. Wrapper Methods
Definition: Wrapper methods evaluate subsets of features based on the
performance of a specific machine learning model. They search through
the feature space to find the combination of features that maximizes
model performance, often using cross-validation to avoid overfitting.
Techniques:
 Recursive Feature Elimination (RFE): Iteratively trains a model, ranks
features by
importance, and removes the least important features.
 Forward Selection: Starts with an empty set of features and adds the
most significant feature at each step, based on model performance.
 Backward Elimination: Starts with all features and removes the least
significant
feature at each step, based on model performance.
Pros:
 Considers interactions between features.
 Generally provides better feature subsets for a specific model.
Cons:
 Computationally expensive and slow, especially with large datasets.
 Prone to overfitting if not properly validated.
3. Embedded Methods
Definition: Embedded methods perform feature selection as part of the
model training process. They incorporate feature selection into the
construction of the model, often using regularization techniques to
penalize irrelevant features.
Techniques:
 Lasso Regression (L1 Regularization): Adds a penalty term to the
regression model
that can shrink some coefficients to zero, effectively performing feature
selection.
 Ridge Regression (L2 Regularization): Adds a penalty term to the
regression model
that can shrink coefficients but does not set them to zero. Often combined
with L1
regularization in Elastic Net.
 Decision Trees and Random Forests: Provide feature importance scores
based on
how often and how effectively a feature is used to split the data.
Pros:
 Feature selection is integrated into the model training, making it more
efficient.
 Takes into account feature interactions and model specifics.
 Often provides better generalization to new data.
Cons:
 More complex to understand and implement compared to filter methods.
 The choice of regularization parameters can be crucial and requires
tuning.

8. Explain wrappers and its two aspects.

In machine learning, specifically feature selection, wrappers are a

technique that leverages a model's performance to identify the optimal
set of features for that particular model. They essentially "wrap" around
the machine learning model and iteratively evaluate different feature
combinations to find the best fit. Here's a breakdown of wrappers and
their two key aspects:
The Core Idea of Wrappers:
 Imagine you're a data scientist training a model to predict customer
churn (when a
customer stops using a service). You have a ton of data about customer
behavior, but not all of it might be relevant for predicting churn.
 Wrappers don't pre-judge which features are important. Instead, they
treat the chosen machine learning model as a "black box" and evaluate its
performance based on different feature subsets.
 By feeding the model various combinations of features and analyzing its
output
(accuracy, error rate, etc.), wrappers identify the subset of features that
leads to the
best model performance for your specific prediction task.
Two Key Aspects of Wrappers:
1. Search Strategy: This determines how wrappers explore different
feature
combinations. Here are two common approaches:
o Sequential Search: This is like climbing a hill, taking one step at a time.
Wrappers start with an empty set of features and iteratively add or
remove
features based on their impact on the model's performance. They stop
when
adding or removing features no longer improves the model. This can be
either:
 Forward selection: Starts with no features and keeps adding the most
beneficial feature at each step.
 Backward elimination: Starts with all features and removes the least
beneficial feature at each step.
o Exhaustive Search: This is like checking every single path on a
mountain. It
evaluates all possible feature combinations, which can be computationally
expensive for datasets with many features. However, it guarantees finding
the
absolute best subset (assuming your model complexity allows it).
2. Evaluation Metric: This is the yardstick used to measure the model's
performance on different feature subsets. The choice depends on your
specific problem. Here are some common examples:
o Classification: Accuracy, precision, recall, F1-score (depending on the
class
imbalance).
o Regression: Mean squared error (MSE), R-squared.

9. Explain the concept of random forest and its construction in detail.

10. Explain the concept of decision tree and its process

Concept of a Decision Tree

1. Structure:
o Root Node: The topmost node in the tree that represents the
entire dataset and the first decision to be made.
o Internal Nodes: These nodes represent decisions based on the
values of
features. Each internal node splits the dataset into two or more
subsets based
on a certain criterion.
o Branches: The outcomes of a decision, leading to either another
decision node or a leaf node.
o Leaf Nodes: The terminal nodes that represent the final outcome
or
prediction.
2. Types of Decision Trees:
o Classification Trees: Used when the target variable is categorical.
The leaves represent class labels, and the branches represent
conjunctions of features that lead to those class labels.
o Regression Trees: Used when the target variable is continuous. The
leaves
represent continuous values.
Process of Building a Decision Tree
1. Data Splitting:
o The process starts at the root node with the entire dataset.
o The algorithm searches for the best feature to split the data based
on a certain criterion (e.g., Gini impurity, information gain for
classification, or mean squared error for regression).

2. Choosing the Best Split:

o For each feature, the algorithm evaluates all possible splits.
o It calculates a measure of the quality of the split. Common
measures include:
 Gini Impurity: Measures the impurity of a split (used in
classification).
 Information Gain (Entropy): Measures the reduction in entropy
after
the split (used in classification).
 Mean Squared Error (MSE): Measures the variance of the values
within each subset (used in regression).
o The feature and split point that result in the best (lowest) impurity
or error are chosen.
3. Recursion:
o The data is split into subsets based on the best split.
o The process is repeated recursively for each subset (internal
node), treating
each subset as a new dataset.
o This recursive process continues until a stopping criterion is met,
such as:
 A maximum tree depth is reached.
 The number of samples in a node is below a certain threshold.
  All samples in a node belong to the same class (for classification)
or
have similar values (for regression).
4. Pruning:
o Pruning is a technique to reduce the size of the tree and prevent
overfitting.
o There are two main types of pruning:
 Pre-pruning (early stopping): The tree growth stops early based on
certain conditions (e.g., maximum depth, minimum samples per
node).
 Post-pruning: The tree is fully grown and then nodes are removed
Based on certain criteria to simplify the model without significantly
Reducing accuracy.

11. Explain concept of dimensionality reduction.

Dimensionality reduction is a technique used in machine learning and

data analysis to reduce the number of features (variables) in a dataset
while aiming to capture the most important information. Imagine you have
a giant room with hundreds of light switches, but only a few actually
control the lights. Dimensionality reduction helps you identify those
essential few switches from the overwhelming number.
Here's a breakdown of the concept and its benefits:
Why dimensionality reduction matters:
 High-dimensional data can be cumbersome: Datasets with many
features can be
difficult to store, visualize, and analyze.
 The curse of dimensionality: As the number of features increases, some
machine
learning algorithms can suffer from the "curse of dimensionality," where
performance
deteriorates.
 Focus on what matters: By reducing dimensionality, you can focus on
the most
relevant features that have the strongest impact on your analysis or
model's
predictions.
Approaches to dimensionality reduction:
There are two main approaches to dimensionality reduction:
1. Feature Selection: This involves selecting a subset of the original
features that are
most informative and contribute most to your prediction task. It's like
choosing only
the essential light switches from the room. Here are some common
feature selection
techniques:
o Filter methods: Rank features based on a statistical measure of their
importance (e.g., correlation with the target variable).
o Wrapper methods: Use a machine learning model to evaluate the
performance of different feature subsets and select the one that leads to
the
best results.
2. Feature Extraction: This involves creating a new set of features by
combining or
transforming the existing features. It's like creating entirely new switches
based on
combinations of the existing ones, potentially revealing hidden patterns.
Here are
some common feature extraction techniques:
o Principal Component Analysis (PCA): Creates new features (principal
components) that capture the maximum variance in the data.
o Linear Discriminant Analysis (LDA): Creates new features that best
discriminate between different classes in a classification task.

12. Explain SVD and the important properties of it

Singular Value DecomposiƟon (SVD) is a powerful matrix factorizaƟon

technique widely used in various fields such as data compression, signal
processing, image processing, and machine learning. It decomposes a
matrix into three simpler matrices and provides insights into the
underlying structure of the original matrix. Here’s an explanaƟon of SVD

and its important properƟes:

Important Properties of SVD:
1. Uniqueness: For a given matrix, there exists a unique SVD
decomposition (U, Σ,
VT). This makes it a reliable and consistent way to analyze a matrix.
2. Dimensionality Reduction: The singular values in Σ tell us how much
information
each basis vector in U captures. We can discard basis vectors with
negligible singular values, effectively reducing the dimensionality of the
data while preserving the most important information. This is particularly
useful for high-dimensional datasets.
3. Data Compression: By keeping only the most significant singular values
and their
corresponding basis vectors, SVD can be used for data compression. This
can be
helpful for storing and transmitting large datasets.
4. Low-Rank Approximation: Any matrix can be approximated by a sum of
a smaller
number of outer products of the columns of U and the rows of VT,
weighted by the
corresponding singular values in Σ. This approximation captures the
essence of the
data with fewer dimensions.
5. Geometric Interpretation: In some cases, SVD can be interpreted
geometrically.
The basis vectors in U represent the principal axes of variation in the data,
and the
singular values reflect the lengths of those axes.

13. Explain the concept of PCA.

Principal Component Analysis (PCA) is a widely used technique for

dimensionality reduction
in data analysis and machine learning. It transforms a dataset of possibly
correlated variables
into a set of linearly uncorrelated variables called principal components.
These components
capture the maximum variance present in the data, allowing for a
reduction in the
dimensionality of the dataset while preserving essential features.
Key Concepts of PCA:
1. Variance and Covariance:
o PCA is based on the idea of capturing the directions (principal
components)
along which the data varies the most.
o It utilizes the covariance matrix of the data to determine these
directions.
2. Principal Components (PCs):
o Principal components are new variables that are linear combinations of
the
original variables.
o The first principal component captures the maximum variance in the
data, the
second principal component captures the second highest variance, and so
on.
o Each principal component is orthogonal (uncorrelated) to the others.
3. Dimensionality Reduction:
o PCA transforms a dataset with d dimensions into a dataset with fewer
dimensions, k, where k<d.
o This reduction is achieved by selecting the top k principal components
that
explain most of the variance in the data.
4. Eigenvalues and Eigenvectors:
o PCA finds the principal components by computing the eigenvectors and
eigenvalues of the covariance matrix of the data.
o Eigenvectors represent the directions or axes of maximum variance, and
eigenvalues indicate the magnitude of variance in those directions.
5. Application in Data Analysis:
o PCA is often used for exploratory data analysis and visualization to
understand the inherent structure of the data.
o It helps in identifying patterns, reducing noise, and selecting important
features for building predictive models.
Steps in PCA:
1. Standardization:
o Standardize the data to have mean 000 and variance 111. This step
ensures
that all variables contribute equally to the principal components.
2. Compute the Covariance Matrix:
o Calculate the covariance matrix Σ of the standardized data.
3. Eigenvalue Decomposition:
o Perform eigenvalue decomposition on Σ to obtain eigenvectors v1,v2,
…,vd
and corresponding eigenvalues λ1,λ2,…,λd .
4. Select Principal Components:
o Sort the eigenvectors based on their corresponding eigenvalues in
descending
order.
o Choose the top k eigenvectors to form the matrix Vk .
5. Transform the Data:
o Project the standardized data onto the new subspace spanned by the
selected k
eigenvectors.
o The transformed data Y is given by Y=XVk, where X is the standardized
data
matrix.

15 Sample r code for random forest.

SOL: # we will be using the diamonds data from ggplot

require(ggplot2)
# load and view the diamonds data
data(diamonds)
head(diamonds)
# plot a histogram with a line marking $12,000
ggplot(diamonds) + geom_histogram(aes(x=price)) +
geom_vline(xintercept=12000)
# build a TRUE/FALSE variable indicating if the price is above
#our threshold
diamonds$Expensive <- ifelse(diamonds$price >= 12000, 1, 0)
head(diamonds)
# get rid of the price column
diamonds$price <- NULL
## glmnet
install.packages("glmnet")
require(glmnet)
# build the predictor matrix, we are leaving out the last
#column which is our response
x <- model.matrix(~., diamonds[, -ncol(diamonds)])
# build the response vector
y <- as.matrix(diamonds$Expensive)
# run the glmnet
system.time(modGlmnet <- glmnet(x=x, y=y, family="binomial"))
# plot the coefficient path
plot(modGlmnet, label=TRUE)
# this illustrates that setting a seed allows you to recreate
# random results, run them both a few times
set.seed(48872)
sample(1:10)
## decision tree
require(rpart)
# fit a simple decision tree
modTree <- rpart(Expensive ~ ., data=diamonds)
# plot the splits
plot(modTree)
text(modTree)
## bagging (or bootstrap aggregating)
require(boot)
mean(diamonds$carat)
sd(diamonds$carat)
# function for bootstrapping the mean
boot.mean <- function(x, i)
{
mean(x[i])
}
# allows us to find the variability of the mean
boot(data=diamonds$carat, statistic=boot.mean, R=120)
install.packages("adabag")
require(adabag)
modBag <- bagging(formula=Species ~ ., iris, mfinal=10)
## boosting
install.packages("mboost")
require(mboost)
system.time(modglmBoost <- glmboost(as.factor(Expensive) ~ .,
data=diamonds, family=Binomial(link="logit")))
summary(modglmBoost)
#clear environment
rm(list= ls())
#clear all plots
dev.off(dev.list()["RStudioGD"])
## random forests
install.packages("randomForest") # For implementing random forest
algorithm
require(randomForest)
system.time(modForest <- randomForest(Species ~ ., data=iris,
importance=TRUE, proximity=TRUE))
modForest
plot(modForest)