Scribd
Upload
40 views4 pages

Assignment 11-17-15: Michael Petzold November 19, 2015

This document loads libraries and data to perform support vector machine (SVM) classification on handwritten digit recognition. It reads in image and label data, preprocesses it, and splits it into training and test sets. It trains SVMs with different kernels on the data and evaluates the prediction accuracy. The best performing model uses a radial kernel and achieves a prediction accuracy of around 90% on the test set. Parameter tuning is done to find the optimal regularization parameter C value.

Uploaded by

mikey p
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
40 views4 pages

Assignment 11-17-15: Michael Petzold November 19, 2015

This document loads libraries and data to perform support vector machine (SVM) classification on handwritten digit recognition. It reads in image and label data, preprocesses it, and splits it into training and test sets. It trains SVMs with different kernels on the data and evaluates the prediction accuracy. The best performing model uses a radial kernel and achieves a prediction accuracy of around 90% on the test set. Parameter tuning is done to find the optimal regularization parameter C value.

Uploaded by

mikey p
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

Assignment 11-17-15

Michael Petzold
November 19, 2015
library(dplyr)
##
## Attaching package: 'dplyr'
##
## The following objects are masked from 'package:stats':
##
##
filter, lag
##
## The following objects are masked from 'package:base':
##
##
intersect, setdiff, setequal, union
library(e1071)
images1.df <- read.csv("images1.csv",sep=' ', header=F)
labels1.df <- read.csv("labels1.csv",sep=' ', header=F)
names(labels1.df) <- "digit"
labels1.df <- mutate(labels1.df,digit=factor(digit))
nrow(labels1.df)
## [1] 10000
images2.df <- read.csv("images2.csv",sep=' ', header=F)
labels2.df <- read.csv("labels2.csv",sep=' ', header=F)
names(labels2.df) <- "digit"
labels2.df <- mutate(labels2.df,digit=factor(digit))
nrow(labels2.df)
## [1] 60000
k <- 300
labels1.df[k,]
## [1] 8
## Levels: 0 1 2 3 4 5 6 7 8 9
img <matrix(as.numeric(images1.df[k,]),ncol=28)
img <- img[,28:1]
image(img,col=c("white","black"))

1.0
0.8
0.6
0.4
0.2
0.0
0.0

0.2

0.4

0.6

data1.df <- data.frame(labels1.df,images1.df)


data2.df <- data.frame(labels2.df,images2.df)
k <- 299
labels1.df[k,]
## [1] 2
## Levels: 0 1 2 3 4 5 6 7 8 9
img <matrix(as.numeric(images1.df[k,]),ncol=28)
img <- img[,28:1]
image(img,col=c("white","black"))

0.8

1.0

1.0
0.8
0.6
0.4
0.2
0.0
0.0

0.2

0.4

0.6

0.8

data.df <- data.frame(labels1.df,images1.df)


## labels are response variable
## predictors are the columns in imaging
## 784 predictors of 0's and 1's. From these predictors, can we predict label?
## It takes WAY too long to train on all of data2.df, so we'll take 5000 rows
rows= sample(60000,5000,rep=F)
train.df <- data2.df[rows,]
test.df <- data1.df
C <- 1
svmfitBigC <- svm(digit~.,data=train.df,kernel="linear",cost=C,scale=F)
##
## cross validate to get the optimal value of C
## use different kernels
## use images2 which is much bigger for train data
svmfit <- svmfitBigC
prediction <- predict(svmfit,newdata=test.df)
mean(prediction==test.df$digit)
## [1] 0.9011
## mean of around .8983
## polynomial kernel
svmfitBigC <- svm(digit~.,data=train.df,kernel="polynomial",cost=C,scale=F)
svmfit1 <- svmfitBigC
3

1.0

prediction <- predict(svmfit1,newdata=test.df)


mean(prediction==test.df$digit)
## [1] 0.1248
## really bad, prediction of only .1211. Was there a mistake? Is it 1-.1211?
## That would be more consistent
## radial kernel
svmfitBigC <- svm(digit~.,data=train.df,kernel="radial",cost=C,scale=F)
svmfit2 <- svmfitBigC
prediction <- predict(svmfit2,newdata=test.df)
mean(prediction==test.df$digit)
## [1] 0.9103
## prediction of .9071, seems to be the best predictor.
## test other values of C
c=seq(0,1,length=11)[2:11]
maximum=0
value.of.c <- 0
## this code takes 10 minutes to
# for (i in c){
#
svmfitBigC <- svm(digit~.,data=train.df,kernel="linear",cost=i,scale=F)
#
svmfit <- svmfitBigC
#
prediction <- predict(svmfit,newdata=test.df)
#
predict.rate=mean(prediction==test.df$digit)
#
if (predict.rate>maximum){
#
maximum <- predict.rate
#
value.of.c <- i
#
}
# }
maximum
## [1] 0
## around .9062
value.of.c
## [1] 0
## around .1

You might also like