Scribd
Upload
8 views4 pages

Machine Learning-Lecture 3(Student)

The document discusses Linear Discriminant Analysis (LDA) for classification, detailing its application using Bayes' Theorem in both one-dimensional and multivariate settings. It includes the mathematical formulation for normal distributions, assumptions about shared variance, and the classification process based on posterior probabilities. Additionally, it provides a practical example using R code to fit an LDA model to stock market data and evaluate its predictions.

Uploaded by

hubertkuo418
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
8 views4 pages

Machine Learning-Lecture 3(Student)

The document discusses Linear Discriminant Analysis (LDA) for classification, detailing its application using Bayes' Theorem in both one-dimensional and multivariate settings. It includes the mathematical formulation for normal distributions, assumptions about shared variance, and the classification process based on posterior probabilities. Additionally, it provides a practical example using R code to fit an LDA model to stock market data and evaluate its predictions.

Uploaded by

hubertkuo418
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

Lecture 3: Classification II

Linear Discriminant Analysis


⚫ Using Bayes’ Theorem for Classification
➢ Linear Discriminant Analysis for p = 1
Suppose we assume that fk(x) is normal or Gaussian. In the one
dimensional setting, the normal density takes the form

where  k and  k2 are the mean and variance parameters for the kth class.

For now, let us further assume that 12 = =  K2 : that is, there is a shared

variance term across all K classes, which for simplicity we can denote by
 2 . We find that

.
It is not hard to show that this is equivalent to assigning the observation to
the class for which
is largest.
➢ For instance, if K = 2 and π1 = π2, then the Bayes classifier assigns an
observation to class 1 if , and to class 2
otherwise. In this case, the Bayes decision boundary corresponds to the
point where

1
➢ The estimates

The LDA classifier

➢ Linear Discriminant Analysis for p >1


We assume that X = (X1,X2, . . .,Xp) is drawn from a multivariate Gaussian
(or multivariate normal) distribution, with a class-specific mean vector and
a common covariance matrix. The multivariate Gaussian density
is defined as

.
The Bayes classifier assigns an observation X = x to the class for which

is largest.

2
Computer Session
library(ISLR2)

library(MASS)

lda.fit = lda(Direction ~ Lag1 + Lag2, data = Smarket, subset = train)


lda.fit

## Call:
## lda(Direction ~ Lag1 + Lag2, data = Smarket, subset = train)
##
## Prior probabilities of groups:
## Down Up
## 0.491984 0.508016
##
## Group means:
## Lag1 Lag2
## Down 0.04279022 0.03389409
## Up -0.03954635 -0.03132544
##
## Coefficients of linear discriminants:
## LD1
## Lag1 -0.6420190
## Lag2 -0.5135293

plot(lda.fit)

3
lda.pred = predict(lda.fit, Smarket.2005)
names(lda.pred)
## [1] "class" "posterior" "x"

lda.class = lda.pred$class
table(lda.class, Direction.2005)

## Direction.2005
## lda.class Down Up
## Down 35 35
## Up 76 106

mean(lda.class == Direction.2005)

## [1] 0.5595238

sum(lda.pred$posterior[, 1] >= .5)

## [1] 70

sum(lda.pred$posterior[, 1] < .5)

## [1] 182

lda.pred$posterior[1:20, 1]
lda.class[1:20]

sum(lda.pred$posterior[, 1] > .9)

## [1] 0

You might also like