-N b [c n -NITI LIyE #TTER

hlon.T nTTd.
G'AD$' 8 OTT$'%mOT$BOOjpq 5%IPYNBq(

o SSI4NMENT 5

o)o ,2Ec mOV nTg p6g2RSDAY(: 3z3t: oocnPM p1ACI0C 6IME(

o)o)o hNST?2aTInNSc

xOUR iUPYTER NOTEBOOj ASSIGNMENT WILL OFTEN HAuE t ELEMENTSc WRITTEN ANSWERS: CODE ANSWERS: AND QUIy
ANSWERS% =OR WRITTEN ANSWERS: YOU MAY INSERT IMAGES OF YOUR HANDWRITTEN WORj IN CODE CELLS: OR WRITE
YOUR ANSWERS IN MARjDOWN AND kA/Ew% =OR QUIy ANSWERS: YOUR '$CO'D%TXT ffLE WILL RECORD YOUR ANSWER
CHOICES IN THE QUIy MODULES FOR SUBMISSION% ~OTH YOUR QUIy ANSWERS AND CODE ANSWERS WILL BE AUTOGRADED ON
4RADESCOPE% /HIS ASSIGNMENT DOES NOT HAuE THE QUIy PORTION%

=OR ALL ELEMENTS: ,9 m9/ l9,-=x /gd bdkkr% 1UT YOUR ANSWERS nNLY IN THE ANSWER CELLS GIuEN: AND
:n NnT :ELETE aELLS% -F YOU FAIL TO FOLLOW THESE INSTRUCTIONS: YOU WILL LOSE POINTS ON YOUR SUBMISSION%

lAjE SURE TO SHOW THE STEPS OF YOUR SOLUTION FOR EuERY QUESTION TO RECEIuE CREDIT: NOT JUST THE ffNAL ANSWER%
xOU MAY SEARCH INFORMATION ONLINE BUT YOU WILL NEED TO WRITE CODE6ffND SOLUTIONS TO ANSWER THE QUESTIONS
YOURSELF% xOU WILL SUBMIT YOUR %IPYNB ffLE AND RECORD%TXT TO GRADESCOPE WHEN YOU ARE ffNISHED%

o)o)3 iATE 1nLIaYc

) REDUCTION FOR THE ffRST DAY AND oz) REDUCTION AFTERWARDS FOR EuERY EXTRA DAY PAST DUE%

o)o)t gnW Tn hNaL2:E xn2? lATH v?ITTEN NSWE?.

xOU COULD USE INLINE k /dw IN MARjDOWN pRECOMMENDED( OR USE MARjDOWNS” INCLUDE IMAGE FUNCTIONALITY TO
SUBMIT YOUR WRITTEN RESPONSES%

i sdw p?EanMMEN:E:( gERE IS A FANTASTIC TUTORIAL FROM bAL/ECH ABOUT USING k /dw IN iUPYTER mOTEBOOj%%
xOU COULD ALSO ffND uARIOUS k /dw TUTORIALS AND CHEAT SHEETS ONLINE%

hNaL2:E hMAES -F YOU ARE STILL GETTING FAMILIAR WITH USING kA/Ew: HANDWRITE THE RESPONSE ON PAPER OR
THE STYLUS% /AjE A PICTURE OR SCREENSHOT OF YOUR ANSWER: AND INCLUDE THAT IMAGE IN THE iUPYTER mOTEBOOj% ~E

o
SURE TO INCLUDE THAT IMAGE IN THE IMGr DIRECTORY% kET”S SAY YOU HAuE YOUR .o RESPONSE SAuED AS IMGr6+o%PNGs
THE MARjDOWN SYNTAX TO INCLUDE THAT IMAGE IS =b+o[pIMGr6+o%PNG(%

o)3 -MPORTANT mOTICE

xOU MUST CHECj BOTH SUBMISSION OUTPUT ON THE GRADESCOPE p rrIGNM$NT 5 h mOT$BOOj AND rrIGNM$NT 5
h lAN2AL 4'ADING( CORRECTLY REECTS YOUR WORj AND RESPONSES% -F YOU NOTICE INCONSISTENCIES BETWEEN YOUR
NOTEBOOj AND THE lANUAL 4RADING PORTION: YOU NEED TO MAjE A 1IAyyA POST: AND WE CAN HELP YOU WITH THAT%

3 .2ESTION oc bONCEPT2AL .2ESTIONS

rELECT THE CORRECT OPTIONpS(% mOTE THAT THERE MIGHT BE MULTIPLE CORRECT OPTIONS%

vRITE YOUR SOLUTION AS A LIST OF STRINGS BY REPLACING THE …•“

dX%c bq q[ IF YOU THINj THE ANSWER IS : AND bq q: q]q[ IF YOU THINj THE ANSWERS ARE AND b(%

3)o o)o p9PTIONAL(

bHOOSE ALL THE uALID ANSWERS TO THE DESCRIPTION ABOUT LINEA? ?E?ESSInN AND LnISTIa ?E?ESSInN FROM THE
OPTIONS BELOWc

% kINEAR REGRESSION IS AN UNSUPERuISED LEARNING PROBLEMs LOGISTIC REGRESSION IS A SUPERuISED LEARNING PROBLEM%

~% kINEAR REGRESSION DEALS WITH THE PREDICTION OF CONTINUOUS uALUESs LOGISTIC REGRESSION DEALS WITH THE PREDICh
TION OF CLASS LABELS%

b% vE CANNOT USE GRADIENT DESCENT TO SOLuE LINEAR REGRESSION% -NSTEAD: WE CAN ONLY USE THE CLOSEDhFORM
SOLUTION TO TACjLE THE LINEAR REGRESSION PROBLEM%

,% kINEAR REGRESSION IS A CONuEX OPTIMIyATION PROBLEM WHEREAS LOGISTIC REGRESSION IS NOT%

onhlTrc z

-N bo[c Qouo 8 bq*q[ n r#k2/-#m

-N b [c G'AD$'%C>$Cjpq+ouoq(

3
3)3 o)3

bHOOSE ALL THE uALID ANSWERS TO THE DESCRIPTION ABOUT ?A:IENT :ESaENT FROM THE OPTIONS BELOWc

% /HE GLOBAL MINIMUM IS GUARANTEED TO BE REACHED BY USING GRADIENT DESCENT%

~% duERY GRADIENT DESCENT ITERATION CAN ALWAYS DECREASE THE uALUE OF LOSS FUNCTION EuEN WHEN THE GRADIENT OF
THE LOSS FUNCTION IS yERO%

b% vHEN THE LEARNING RATE IS uERY LARGE: IT IS POSSIBLE THAT SOME ITERATIONS OF GRADIENT DESCENT MAY NOT
DECREASE THE uALUE OF LOSS FUNCTION%

,% vITH DI0ERENT INITIAL WEIGHTS: IT IS POSSIBLE FOR THE GRADIENT DESCENT ALGORITHM TO OBTAIN TO DI0ERENT LOCAL
MINIMUM%

onhlTrc 3

-N b5[c Qou3 8 bq]q: q,q[ n r#k2/-#m

-N b [c G'AD$'%C>$Cjpq+ou3q(

3)t .2ESTION 3 dRROR lETRICS

/HE GRID BELOW SHOWS 4 − 3 < . POSSIBLE LOCATIONS FOR TARGETS TO APPEAR% vE MAjE GUESSES AT THE LOCATIONS
WHERE TARGETS ARE LOCATED: AND MARj THOSE CELLS IN BLUE% =OR THE REST OF THE LOCATIONS: WE GUESS THAT THOSE
LOCATIONS ARE NONhTARGETS: WHICH ARE MARjED IN WHITE% /HE ACTUAL LOCATIONS FOR THE TARGET ARE MARjED WITH A
CIRCLE IN CELLS: WHILE THE ACTUAL LOCATIONS FOR NONhTARGET ARE MARjED WITH EMPTY CELLS%

3)t)o o) 1LEASE EVAL2ATE THE enLLnWIN MET?IaS en? Yn2? 2ESSES)

f /RUE 1OSITIuE puARIABLE NAME /1(c /HE CASE WHICH IS TARGET AND PREDICTED AS TARGET%
f /RUE mEGATIuE puARIABLE NAME /m(c /HE CASE WHICH IS NONhTARGET AND PREDICTED AS NONhTARGET%
f =ALSE 1OSITIuE puARIABLE NAME !1(c /HE CASE WHICH IS NONhTARGET BUT PREDICTED AS TARGET%
f =ALSE mEGATIuE puARIABLE NAME !m(c /HE CASE WHICH IS TARGET BUT PREDICTED AS NONhTARGET%

vRITE YOUR SOLUTION AS A NUMBER BY REPLACING THE %%% PART AFTER EACH uARIABLE%

gINTc -N THIS QUESTION: ALL THE NUMBERS SHOULD BE INTEGERS% dX%c /18o%

t
onhlTrc 5

-N bs[c /1 8 3 n r#k2/-#m
/m 8 of n r#k2/-#m
!1 8 3 n r#k2/-#m
!m 8 o n r#k2/-#m

-N b [c G'AD$'%C>$Cjpq+3uoq(

3)t)3 3) 1LEASE EVAL2ATE THE enLLnWIN MET?IaS)

xOU MAY USE THE RESULTS FROM THE PREuIOUS QUESTION%

f 1RECISION puARIABLE NAME P'$CIrION(c

NUMBER OF TRUE POSITIuES

1RECISION <
NUMBER OF GUESSES

f qECALL puARIABLE NAME '$CALL(c

NUMBER OF TRUE POSITIuES

qECALL <
NUMBER OF ACTUAL TARGETS

f =huALUE puARIABLE NAME Fu_AL2$(c

 − PRECISION − RECALL
=huALUE <
PRECISION  RECALL

vRITE YOUR SOLUTION AS A NUMBER BY REPLACING THE %%% PART AFTER EACH uARIABLE% 1LEASE ROUND YOUR ANSWERS
TO 3 :EaIMAL PLAaES%

gINTc -N THIS QUESTION: ALL THE NUMBERS SHOULD BE OATS% dX%c P'$CIrION8z%7z%

onhlTrc 7

-N bot[c P'$CIrION 8 z%fz n r#k2/-#m

'$CALL 8 z%7s n r#k2/-#m
Fu_AL2$ 8 z%fs n r#k2/-#m

-N b [c G'AD$'%C>$Cjpq+3u3q(

5
t .2ESTION tc iINEAR qE4RESSION

SSUME WE ARE GIuEN A DATASET 𝐿 < 𝑊 + 𝑥𝑊 ℎ ≥ + + % gERE: 𝑊 ≥ ∑ IS A FEATURE SCALAR pA%j%A% uALUE OF INPUT
uARIABLE( AND 𝑥𝑊 ≥ ∑ IS ITS CORRESPONDING uALUE pA%j%A% uALUE OF DEPENDENT uARIABLE(% -N THIS SECTION: WE AIM
TO ffT DATA POINTS WITH A LINEc

𝑥 < 2   

WHERE 2 :  ≥ ∑ ARE TWO PARAMETERS TO DETERMINE THE LINE% mEXT: WE MEASURE THE QUALITY OF ffTTING BY
EuALUATING A SUMhOFhSQUARES ERROR FUNCTION 𝑓 2 +   c

𝑗
𝑓  2 +    <   2     𝑊 ⎧ 𝑥 𝑊 1
𝑊<

vHEN 𝑓 2 +   IS NEAR yERO: IT MEANS THE PROPOSED LINE CAN ffT THE DATASET AND MODEL AND MODEL AN ACCURATE
RELATION BETWEEN 𝑊 AND 𝑥𝑊 % /HE BEST LINEWITH PARAMETERS 2× + ×  CAN REACH THE MINIMUM uALUE OF THE ERROR
FUNCTION 𝑓 2 +  c

2× + ×  < ARGMIN 𝑓 2 +  

𝑛 0𝑛…

/O OBTAIN THE PARAMETERS OF THE BEST LINE: WE WILL TAjE THE GRADIENT OF FUNCTION 𝑓 2 +   AND SET IT TO yERO%
/HAT ISc

ℒ𝑓 2 +   < .

/HE SOLUTION 2× + ×  OF THE ABOuE EQUATION WILL DETERMINE THE BEST LINE 𝑥 < 2×  ×  THAT ffTS THE DATASET
𝐿% -N REALITY: WE TYPICALLY TACjLE THIS TASj IN A MATRIX FORMc =IRST: WE REPRESENT DATA POINTS AS MATRICES
 < > + 1 + + 𝑗 [𝑦 AND 𝑋 < >𝑥 + 𝑥1 + + 𝑥𝑗 [‖ : WHERE 𝑊 < >+ 𝑊 [‖ IS A FEATURE uECTOR CORRESPONDING TO 𝑊 %
/HE PARAMETERS OF THE LINE ARE ALSO REPRESENTED AS A MATRIX 𝑆 < >2 +  [𝑦 % /HUS: THE SUMhOFhSQUARES ERROR
FUNCTION 𝑓 𝑆  CAN BE DEffNED AS pA%j%A% SQUARED 𝐴1 NORM(c

𝑗
𝑓 𝑆  <  𝑦𝑊 𝑆 ⎧ 𝑥𝑊 1
𝑊<
< 𝑆 ⎧ 𝑋 11
< 𝑆 ⎧ 𝑋 ‖ 𝑆 ⎧ 𝑋 
rIMILARLY: THE PARAMETERS 𝑆 × < >2× + × [‖ OF THE BEST LINE CAN BE OBTAINED BY SOLuING THE EQUATION BELOWc


 𝜂𝑓 𝑆 
ℒ𝑓 𝑆  < <.
𝜂𝑆

t)o .2ESTION t)o 9ir lATRIw eORM

CCORDING TO THE DEffNITION OF 𝑓 𝑆  ABOuE: COMPUTE THE GRADIENT OF 𝑓 𝑆  WITH RESPECT TO 𝑆 % xOUR RESULT
SHOULD BE IN THE FORM OF : 𝑋 : AND 𝑆 %

rHOW YOUR WORj%

onhlTrc 7

sxP, xnt. lru,. g,.,: .,Pi ahle Tghr T,wTp

7
t)3 .2ESTION t)3c 9ir lATRIw rOL2TION

~Y SETTING THE ANSWER OF LAST PART TO z: PROuE THE FOLLOWINGc

𝑆 × < ARGMIN 𝑓 𝑆  <  𝑦 ⎧  𝑦 𝑋

𝑇

mOTEc /HE ABOuE FORMULA DEMONSTRATES A CLOSED FORM SOLUTION OF

𝜂𝑓 𝑆 
ℒ𝑓 𝑆  < <.
𝜂𝑆

rHOW YOUR WORj%

onhlTrc 5

sxP, xnt. lru,. g,.,: .,Pi ahle Tghr T,wTp

t)t pdwTRA bREDIT( .2ESTION t)tc iho dRROR

𝑗
1REuIOUSLY: WE DEffNE A SUMhOFhSQUARES ERROR FUNCTION 𝑓 2 +   < ∈𝑊< 𝑥𝑊 ⎧ 2 ⎧   1 AND REPRESENT IT
1
IN A MATRIX FORM 𝑓 𝑆  < ∗𝑆 ⎧ 𝑋 ∗1 % CTUALLY: WE CAN HAuE MULTIPLE CHOICES OF THE ERROR FUNCTIONc =OR
EXAMPLE: WE CAN DEffNE A SUMhOFhABSOLUTE ERROR FUNCTION 𝑔 2 +  c
𝑗
𝑔 2 +   <  2   𝑊 ⎧ 𝑥𝑊 
𝑊<

AND REPRESENT IT IN A MATRIX FORM 𝑔 𝑆  pA%j%A% 𝐴 NORM(c

;
 𝑗
𝑔 𝑆  <  X𝑦𝑊 𝑆 ⎧ 𝑥𝑊 
𝑊<
< ∗𝑆 ⎧ 𝑋 ∗
CCORDING TO THE INFORMATION ABOuE: COMPUTE THE GRADIENT OF THE ERROR FUNCTION 𝑔 𝑆  WITH RESPECT TO 𝑆 %
xOUR RESULT SHOULD BE IN THE FORM OF X𝑊 : 𝑥𝑊 AND 𝑆 %

gINTc 4IuEN A FUNCTION 𝑒 X ≥ ∑: WE HAuEc

𝜂𝑒 X 𝜂𝑒 X
< SIGN 𝑒 X
𝜂X 𝜂X

WHERE
⎨ + =.
⎩
SIGN  < .+ <.
{
⎩
 ⎧+  ; .

onhlTrc 

sxP, xnt. lru,. g,.,: .,Pi ahle Tghr T,wTp

e
t)5 pxO2 ,9 m96 NEED TO SOLVE TgIS PROaLEM( .2ESTION t)5

-N FACT: THE GRADIENT IN .t%t CAN ALSO BE REPRESENTED ASc

𝜂𝑔 𝑆  ‖ ‖
ℒ𝑔 𝑆  < < ←SIGN 𝑆 ⎧ 𝑋  (
𝜂𝑆
WHERE SIGN , MEANS PERFORMING ELEMENThWISE SIGN 𝑌𝑊𝑖  OuER ALL ELEMENT 𝑌𝑊𝑖 IN A MATRIX ,% /HIS MATRIX FORM
OF GRADIENT CAN BE HELPFUL IN SOME FOLLOWING CODING QUESTIONS%

5 .2ESTION 5c 1ERCEPTRON

-N THIS SECTION: WE WILL APPLY PERCEPTRON LEARNING ALGORITHM TO SOLuE THE BINARY CLASSIffCATION PROBLEMc vE
NEED TO PREDICT A BINARY LABEL 𝑥 ≥ ]⎧+ | FOR A FEATURE uECTOR X < >2 +  [‖ % /HE DECISION RULE OF THE
PERCEPTRON MODEL IS DEffNED ASc

+ IF W‖ X  𝑎 ≠ .+
𝑒 X5 W+ 𝑎 < )
⎧+ OTHERWISE

WHERE W < >2 +  [‖ IS THE WEIGHT uECTOR: AND 𝑎 IS THE BIAS SCALAR% 4IuEN A TRAINING DATASET 𝐿snAgigie <
] X𝑊 + 𝑥𝑊 |+ ℎ < + } + |: WE DEffNE THE TRAINING ERROR 𝑏snAgigie ASc

 𝑗
𝑏snAgigie <  o←𝑥𝑊 ⊤ 𝑒 X𝑊 5 W+ 𝑎
 𝑊<

AND THE TEST ERROR 𝑏sars ON THE TEST SET 𝐿sars CAN BE DEffNED IN THE SAME WAY%

-N THE PERCEPTRON ALGORITHM: WE AIM TO DIRECTLY MINIMIyE THE TRAINING ERROR 𝑏snAgigie IN ORDER TO OBTAIN THE OPh
TIMAL PARAMETERS W× + 𝑎× % -F WE REPRESENT DATA POINTS IN TRAINING SET 𝐿snAgigie AS MATRICES  < >X + X1 + } + X𝑗 [𝑦
AND 𝑋 < >𝑥 + 𝑥1 + } + 𝑥𝑗 [𝑦 : THE PERCEPTRON ALGORITHM IS SHOWN AS BELOWc

-N b3z[c )}nNFhf -NLIN$*ACj$ND%FIG2'$uFO'MAT 8 "'$TINA"

hlon.T N2low r No
hlon.T l ToknTkhapowoknT r okT
F.nl rikd .N hlon.T DATAr$Tr

inA: THE Mn:IzE: h?IS :ATASET

-N b3o[c n -RIS D T SET)

I'Ir 8 DATAr$Tr%LOADuI'Irp( n kO D -RIS D T SET)

n
 w 8 I'Ir%DATA n /HE SH PE OF w IS pofz: 5(: WHICH <E NS
n THERE RE ofz D T POINTS: E CH D T POINT
n H S 5 FE TURES)

n >ERE FOR CONVENIENCE: WE DIVIDE THE t jINDS OF FLOWERS INTO 3 GROUPSc

n x 8 z pOR ! LSE(c rETOS pORIGIN L V LUE z( s _ERSICOLOR pORIGIN L V LUE o(
n x 8 o pOR /RUE(c _IRGINIC pORIGIN L V LUE 3(

n /HUS WE USE pIRIS)T RGET g o)f( TO DIVIDE THE T RGETS INTO 3 GROUPS)
n /HIS LINE OF CODE WILL SSIGNc
n xbI[ 8 /RUE pWHICH IS E+UIV LENT TO o( IF IRIS)T RGETbj[ g o)f p_IRGINIC (
n xbI[ 8 ! LSE pWHICH IS E+UIV LENT TO z( IF IRIS)T RGETbj[ l8 o)f prETOS s _ERSICOLOR(

x 8 pI'Ir%TA'G$T g o%f(%'$r>AP$pho:o(%ArTYP$pFLOAT( n /HE SH PE OF x IS pofz: o(: WHICH <E NS

n THERE RE ofz D T POINTS: E CH D T POINT
n H S o T RGET V LUE)
xbx88z[ 8 ho

wuANDux 8 NP%>rTACjppw: x(( n rT Cj THE< TOGETHER FOR SHUFFLING)

NP%'ANDOM%r$$Dpo( n rET THE R NDO< SEED)
NP%'ANDOM%r>2FFL$pwuANDux( n rHUFFLE THE D T POINTS IN wu NDux RR Y

P'INTpw%r>AP$(
P'INTpx%r>AP$(
P'INTpwuANDuxbz[( n /HE RESULT SHOULD BE LW YSc b f)e 5) o)3 z)3 z) [

pofz: 5(
pofz: o(
b f%e 5% o%3 z%3 ho% [

-N b33[c n ,IVIDE THE D T POINTS INTO TR INING SET ND TEST SET)

wur>2FFL$D 8 wuANDuxbc:c5[
xur>2FFL$D 8 wuANDuxbc:5[

wuT'AIN 8 wur>2FFL$Dbcozz[bc:bt:o[[ n rH PEc pozz:3(

wuT'AIN 8 NP%D$L$T$pwuT'AIN: 53: AXIr8z( n "E<OVE POINT FOR SEP R BILITY)
xuT'AIN 8 xur>2FFL$Dbcozz[ n rH PEc pozz:(
xuT'AIN 8 NP%D$L$T$pxuT'AIN: 53: AXIr8z( n "E<OVE POINT FOR SEP R BILITY)
wuT$rT 8 wur>2FFL$Dbozzc[bc:bt:o[[ n rH PEc pfz:3(
xuT$rT 8 xur>2FFL$Dbozzc[ n rH PEc pfz:(
P'INTpwuT'AIN%r>AP$(
P'INTpxuT'AIN%r>AP$(
P'INTpwuT$rT%r>AP$(
P'INTpxuT$rT%r>AP$(

pnn: 3(
pnn:(

oz
pfz: 3(
pfz:(

uIS2ALIyATInN

-N b3t[c :dF _Irpw: x: v8mnNd: B8mnNd(c

INDIC$ruN$Go 8 px 88 ho(%NONy$'Op(bz[
INDIC$ruPOro 8 px 88 o(%NONy$'Op(bz[
PLT%rCATT$'pwbc:z[bINDIC$ruN$Go[: wbc:o[bINDIC$ruN$Go[:
C8"BL2$": LAB$L8"CLArr ho"(
PLT%rCATT$'pwbc:z[bINDIC$ruPOro[: wbc:o[bINDIC$ruPOro[:
C8"'$D": LAB$L8"CLArr o"(
PLT%L$G$NDp(
PLT%XLAB$Lp"dXuzd"(
PLT%YLAB$Lp"dXuod"(

hF v hr NnT mnNdc
n WzXz.WoXo.B8z 8g Xo8hWzXzsWohBsWo
Wz 8 vbz[
Wo 8 vbo[
T$MP 8 hWo^NP%A''AYpbwbc:o[%MINp(: wbc:o[%MAXp([(6WzhB6Wz
XzuMIN 8 MAXpT$MP%MINp(: wbc:z[%MINp((
XzuMAX 8 MINpT$MP%MAXp(: wbc:o[%MAXp((
Xz 8 NP%LINrPAC$pXzuMIN:XzuMAX:ozz(
Xo 8 hWz^Xz6WohB6Wo
PLT%PLOTpXz:Xo:COLO'8"BLACj"(

PLT%r>OWp(

-N b35[c n _ISU LIyE TR INING SET)

_IrpwuT'AIN: xuT'AIN(

oo
-N b3f[c n _ISU LIyE TEST SET)
_IrpwuT$rT: xuT$rT(

o3
5)o .2ESTION 5)o

-N THIS PROBLEM: WE WOULD LIjE TO TRAIN A PERCEPTRON MODEL FOR THE CLASSIffCATION TASj ON A MODIffED -RIS
DATASET% 1LEASE COMPLETE THE FOLLOWING CODE pYOU SHOULD ONLY INSERT YOUR CODE IN THE %%% PART( TO IMPLEMENT
THE ALGORITHM ABOuE%

mOTE THAT IN THE CODE: WE USE wuT'AIN AND xuT'AIN TO REPRESENT THE FEATURE uECTOR  AND LABELS 𝑋 IN
TRAINING SET 𝐿snAgigie % ~ESIDES: WE USE v AND B TO REPRESENT THE WEIGHT uECTOR W AND BIAS SCALAR 𝑎%

gINTc =OR THE IMPLEMENTATION OF SOME FUNCTIONS: YOU MAY REFER TO gvt .5%

onhlTrc e

-N b37[c n iUDGE FUNCTIONc op =8 B()

:dF i2DG$pA: B(c
qqq

ot
 iUDGE FUNCTIONc op =8 B()
"ETURN o IF =8 B: OTHERWISE RETURN z)
qqq
n *d4-m r#k2/-#m
hF A =8 Bc
.dT2.N o
dkrdc
.dT2.N z
n dm, r#k2/-#m

:dF FuP$'C$PT'ONpX: v: B(c

qqq
1ERCEPTRON CL SSIFIERc FpX: v: B(
/HIS FUNCTION SHOULD RETURN ho OR o)

X SHOULD BE 3hDI<ENSION L VECTOR:

v SHOULD BE 3hDI<ENSION L VECTOR:
B SHOULD BE SC L R)
qqq
n *d4-m r#k2/-#m
hF v%/%DOTpX( . B g8 zc
.dT2.N o
dkrdc
.dT2.N ho
n dm, r#k2/-#m

n ] LCUL TE ERROR GIVEN FE TURE VECTORS w ND L BELS x)

:dF CALCu$''O'pw: x: v: B(c
$ 8 z n r#k2/-#m
N 8 L$Npw( n r#k2/-#m
Fn. pXI: YI( hN yIPpw: x(c
n >INTc 2SE JUDGEp( ND FuPERCEPTRONp(
n *d4-m r#k2/-#m
$ .8 i2DG$pYI: FuP$'C$PT'ONpXI: v: B((
n dm, r#k2/-#m

n >INTc RE<E<BER WE W NT THE VER GE ERROR)

$ 8 o%z ^ $ 6 N n r#k2/-#m
.dT2.N $

-N b3s[c n rO<E SETTINGS)

$''O'r 8 b[ n dRROR HISTORY)
LAM 8 o n k <BD WHICH CONTROLS THE STEP SIyE)

n -NITI LIy TION)

v 8 NP%y$'Orp3( n vEIGHT)
B 8 z%z n *I S)

n 1ERCEPTRON LE RNING LGORITH<)

ughkd CALCu$''O'pwuT'AIN: xuT'AIN: v: B( g zc
Fn. XI:YI hN yIPpwuT'AIN: xuT'AIN(c n -TER TE OVER LL D T POINTS)
n ]O<PUTE THE <ODEL PREDICTION)
YIuP'$D 8 FuP$'C$PT'ONpXI: v: B( n r#k2/-#m

o5
 n ]O<P RE PREDICTION ND L BEL)
IruCO''$CT 8 YIuP'$D 88 YI n r#k2/-#m
hF IruCO''$CTc
}nNThN2d n h -F CORRECT: CONTINUE)
dkrdc
n *d4-m r#k2/-#m
v 8 v.LAM^pYIhYIuP'$D(^XI n h -F NOT: UPD TE WEIGHT ND BI S)
B 8 B.LAM^pYIhYIuP'$D(
n dm, r#k2/-#m

n /R Cj TR INING ERRORS)
$''O'r%APP$NDpCALCu$''O'pwuT'AIN: xuT'AIN: v: B((

-N b [c G'AD$'%C>$Cjpq+5uP$'C$PT'ONuCOD$q(

5)3 .2ESTION 5)3

1LEASE COMPLETE THE FOLLOWING CODES TO uISUALIyE THE DECISION BOUNDARY OF THE PERCEPTRON MODEL% xOU MAY
USE THE _Ir FUNCTION DEffNED ABOuE%

LSO: PLEASE PLOT THE TRAINING ERROR CURuE WITH RESPECT TO THE NUMBER OF ITERATIONS%

xOU SHOULD ONLY INSERT YOUR CODE IN THE %%% PART%

onhlTrc 5

-N bte[c n rHOW DECISION BOUND RY: TR INING ERROR ND TEST ERROR)

P'INTp",$CIrION BO2NDA'Yc bcptF{Xz.bcptF{Xo.bcptF{8z"%FO'MATpvbz[:vbo[:B((
_IrpwuT'AIN: xuT'AIN: v: B( n r#k2/-#m
P'INTp"/'AINING $''O'c b{"%FO'MATpCALCu$''O'pwuT'AIN: xuT'AIN: v: B(((
_IrpwuT$rT: xuT$rT: v: B( n r#k2/-#m
P'INTp"/$rT $''O'c b{"%FO'MATpCALCu$''O'pwuT$rT: xuT$rT: v: B(((

,$CIrION BO2NDA'Yc sz%3zzXz.h35%ezzXo.h57%zzz8z

o
/'AINING $''O'c z%z

o7
/$rT $''O'c z%o

-N btn[c n 1LOT TR INING ERROR CURVE)

n *d4-m r#k2/-#m
PLT%TITL$p"/'AINING $''O' C2'_$"(
PLT%PLOTp$''O'r(
PLT%XLAB$Lp"IT$'ATION"(
PLT%YLAB$Lp"$''O'"(
PLT%r>OWp(
n dm, r#k2/-#m

o;
 .2ESTION c 1ARAaOLA dSTIMATION

vE ARE GIuEN A DATASET 𝐿 < ] 𝑊 + 𝑥𝑊 + ℎ < + } + |% gERE: 𝑊 IS A FEATURE SCALAR% -N THIS SECTION: WE AIM TO ffT
DATA POINTS WITH A PARABOLAc
𝑥 <  2       1 1
WHERE 2 +  + 1 ≥ ∑ ARE THREE PARAMETERS TO DETERMINE THE PARABOLA% /HEN: WE REPRESENT THE DATA POINTS
AS MATRICES  < >X + X1 + } + X𝑗 [‖ AND 𝑋 < >𝑥 + 𝑥1 + } + 𝑥𝑗 [‖ : WHERE X𝑊 < >+ 𝑊 + 1𝑊 [‖ IS A FEATURE uECTOR CORREh
SPONDING TO THE DATA 𝑊 % /HE PARAMETERS OF THE PARABOLA ARE ALSO REPRESENTED AS A MATRIX 𝑆 < >2 +  + 1 [‖ %
mEXT: WE DEffNE SOME LOSS FUNCTION ℝ 𝑆  AND ATTEMPT TO OBTAIN THE BEST PARAMETERS 𝑆 × THAT MINIMIyES
ℝ 𝑆 %

-N b5z[c n -<PORTING -<PORT NT P Cj GES pNOTHING TO DD TO THIS CELL(

hlon.T N2low r No
hlon.T l ToknTkhapowoknT r okT

oe
-N b5o[c n kO DING THE D T pNOTHING TO DD TO THIS CELL(

wuANDux 8 NP%LOADp"%6PA'ABOLAu$rTIMATION%NPY"(
OLDuw 8 wuANDuxbc: z[ n rH PEc ptzz:(
x 8 wuANDuxbc: o[ n rH PEc ptzz:(
OLDuw%r>AP$

92Tb5o[c ptzz:(

)o uIS2ALIyIN4 TgE ,ATA

-N b53[c n _ISU LIy TION OF THE ORIGIN L D T POINTS pNOTHING TO DD TO THIS CELL()

:dF _IrpWz: Wo: W3(c

D'AWuPLAN$ 8 pWz hr NnT mnNd( N: pWo hr NnT mnNd( N: pW3 hr NnT mnNd(
hF D'AWuPLAN$c
wuLIN$ 8 NP%LINrPAC$pz:oz:tzz(
xuLIN$ 8 Wz . Wo ^ wuLIN$ . W3 ^ pwuLIN$^^3(
PLT%PLOTpwuLIN$: xuLIN$: COLO'8"O'ANG$"(

PLT%rCATT$'pOLDuw: x: COLO'8"G'AY"(
PLT%XLAB$Lp"X"(
PLT%YLAB$Lp"Y"(
PLT%r>OWp(

_IrpmnNd: mnNd: mnNd(

on
)3 .2ESTION )oc 1ARAaOLA dSTIMATION vITg rQ2ARED 𝐴1 mORM

bONSIDER SQUARED 𝐴1 NORM AS THE LOSS FUNCTION ℝ 𝑆 c

𝑗
ℝ 𝑆  <  X𝑦𝑊 𝑆 ⎧ 𝑥𝑊 1
𝑊<
1
< ∗𝑆 ⎧ 𝑋 ∗1
< 𝑆 ⎧ 𝑋 𝑦 𝑆 ⎧ 𝑋 

bOMPLETE THE FOLLOWING CODE TO USE THE aLnSE: en?M SnL2TInN TO COMPUTE 𝑆 × < ARG MIN𝑇 ℝ 𝑆  AND PLOT
THE SCATTER GRAPH OF DATA AND ESTIMATED PARABOLA%

vRITE YOUR CODE IN THE %%% PART%

gINTc xOU MAY REFER TO .t%3 FOR THE ANALYTIC SOLUTION% LSO: NP%>rTACj MAY BE USEFUL FOR CONSTRUCTING THE

3z
DESIGN MATRIX %

onhlTrc t

-N b5t[c :dF L3uANALYTICu$rTIMATIONpOLDuw: x(c

n ]ONSTRUCT DESIGN < TRIX OF SH PE ptzz:t(
w 8 NP%MAT'IXpNP%>rTACjppNP%ON$rppL$NpOLDuw(:o((: OLDuw%'$r>AP$pho:o(: pOLDuw^^3(%'$r>AP$ph

n >INTc -N THE FOR< OF w ND x ND SHOULD H VE SH PE OF pt:o(

v 8 w%/%DOTpw(%-%DOTpw%/(%DOTpx( n r#k2/-#m

Wz: Wo: W3 8 NP%A''AYpv(%'$r>AP$pho( n r#k2/-#m

.dT2.N w: NP%A''AYpbWz: Wo: W3[(

w: v 8 L3uANALYTICu$rTIMATIONpOLDuw: x(
Wz: Wo: W3 8 v
P'INTp"Y 8 bcp3F{ . bcp3F{^X . bcp3F{^Xa3"%FO'MATpWz: Wo: W3((
_IrpWz: Wo: W3(

Y 8 fo%zs . ho7%z7^X . 3%t7^Xa3

3o
)t .2ESTION )3 1ARAaOLA dSTIMATION vITg 𝐴 mORM

bONSIDER 𝐴 NORM AS THE LOSS FUNCTION ℝ 𝑆 c

𝑗
ℝ 𝑆  <  X𝑦𝑊 𝑆 ⎧ 𝑥𝑊 
𝑊<
< ∗𝑆 ⎧ 𝑋 ∗

-N THIS PROBLEM: WE WOULD LIjE TO USE THE ?A:IENT :ESaENT TO CALCULATE THE PARAMETERS 𝑆 FOR THE PARABOLA%

-F WE HAuE A LOSS FUNCTION ℝ 𝑆 : THEN A TYPICAL GRADIENT DESCENT ALGORITHM CONTAINS THE FOLLOWING STEPSc

rTEP o% -NITIALIyE THE PARAMETERS v%

FOR I 8 o TO IT$'ATIONrc

 𝑇 
f rTEP 3% bOMPUTE THE GRADIENT ℒℝ 𝑆  < 𝑇 %

f rTEP t% 2PDATE THE PARAMETERS 𝑆 𝑤 ℝ 𝑆  < 𝑆 ⎧ 𝛼 𝑇𝑇  WHERE 𝛼 IS THE LEARNING RATE%

1LEASE COMPLETE THE FOLLOWING CODE TO CALCULATE THE GRADIENT FOR THE LOSS FUNCTION: AND PERFORM THE GRADIENT
DESCENT ALGORITHM%

vRITE YOUR CODE IN THE %%% PART%

gINT oc xOU MAY REFER TO .t%5 FOR THE GRADIENT OF 𝐴 NORM%

gINT 3c xOU MAY USE YOUR PREuIOUS DESIGN MATRIX OF %

onhlTrc 5

-N b5f[c n 4R DIENT OF kpv( WITH RESPECT TO v

npYOU NEED TO DD CODE TO THIS CELL S INDIC TED BELOW()

:dF G'ADukuvu_$'opw: x: v(c

G'AD 8 NP%rIGNpw%DOTpv( h x(%/%DOTpw(%/ n r#k2/-#m

33
 rrd.T G'AD%r>AP$ 88 pt:o(
.dT2.N G'AD

-N b57[c n dSTI< TING v: WHICH DEFINES THE HYPERPL NE USING GR DIENT DESCENT pYOU NEED TO DD CODE TO TH
n Y 8 Wz . Wo^X . W3^Xa3
:dF LouG'ADuD$rC$NTpw: x(c
rrd.T w%r>AP$ 88 ptzz:t(: FqxO2' D$rIGN MAT'IX Ir r>AP$D W'ONG: GOT bw%r>AP${q

n rO<E SETTINGS)
x 8 x%'$r>AP$pho: o(
IT$'ATIONr 8 tzzzzz
L$A'NINGu'AT$ 8 z%zzzzzo

n 4R DIENT DESCENT LGORITH<)

n rTEP o) -NITI LIyE THE P R <ETERS v USING NP)yEROS
v 8 NP%MAT'IXpNP%y$'Orppt:o((( n r#k2/-#m

Fn. I hN 'ANG$pIT$'ATIONr(c
n rTEP 3) ] LCUL TE THE GR DIENT OF kpv( W)R)T) v)
G'AD 8 G'ADukuvu_$'opw: x: v( n r#k2/-#m
n rTEP t) 2PD TE P R <ETERS v)
v h8 L$A'NINGu'AT$ ^ G'AD n r#k2/-#m

n rTORE THE P R <ETERS OF THE P R BOL )

Wz: Wo: W3 8 NP%A''AYpv(%'$r>AP$pho(

.dT2.N NP%A''AYpbWz:Wo:W3[(

n _ISU LIy TION)

Wz: Wo: W3 8 LouG'ADuD$rC$NTpw: x(
P'INTp"Y 8 bcp3F{ . bcp3F{^X . bcp3F{^Xa3"%FO'MATpWz: Wo: W3((
_IrpWz: Wo: W3(

Y 8 o%75 . o%of^X . o%zf^Xa3

3t
)5 .2ESTION )t 1ARAaOLA dSTIMATION vITg rQ2ARED 𝐴1 mORM AND 𝐴 mORM

-N THIS PROBLEM: WE WOULD LIjE TO USE THE GRADIENT DESCENT TO CALCULATE THE PARAMETERS 𝑆 FOR THE PARABOLA%
/HE LOSS FUNCTION ℝ 𝑆  NOW CONTAINS TWO PARTSc SQUARED 𝐴1 NORM AND A 𝐴 NORM% COECIENT ∇ IS USED
TO CONTROL THE RATIO OF THESE TWO NORMSc

𝑗
1
ℝ 𝑆  <  ∇←X𝑦𝑊 𝑆 ⎧ 𝑥𝑊    ⎧ ∇X𝑦𝑊 𝑆 ⎧ 𝑥𝑊 (
𝑊<
1
< ∇ ∗𝑆 ⎧ 𝑋 ∗1   ⎧ ∇ ∗𝑆 ⎧ 𝑋 ∗

bOMPLETE THE FOLLOWING CODE TO USE THE ?A:IENT :ESaENT TO ffND 𝑆 × WHEN ∇ < .+ ∇ < ..+ ∇ < ..4+ ∇ <
.: AND ∇ < : ESPECTIuELY%

vRITE YOUR CODE IN THE %%% PART%

35
gINTc xOU MAY REFER TO .t%o FOR THE GRADIENT OF 𝐴1 NORM%

mnTEc -T MAY TAjE 3at MINS TO RUN THE ALGORITHM%

onhlTrc 7

-N b5n[c n 4R DIENT OF kpv( WITH RESPECT TO v pYOU NEED TO DD CODE TO THIS CELL S INDIC TED BELOW()
:dF G'ADukuvu_$'3pw: x: v: ALP>A(c
G'AD 8 ALP>A ^p3^w%/%DOTpw(%DOTpv( h 3^w%/%DOTpx(( . pohALP>A(^pNP%rIGNpw%DOTpv( h x(%/%DOT
.dT2.N G'AD

-N bfz[c n !UNCTION TO USE GR DIENT DESCENT TO ESTI< TE P R BOL GIVEN LIST OF LPH S
n pYOU NEED TO DD CODE TO THIS CELL S INDIC TED BELOW()
n >INTc !OR E CH LPH : YOU NEED TO USE GR DIENT DESCENT: HENCE: YOU NEED TO WRITE LOOP INSID

:dF LouL3uG'ADuD$rC$NTpOLDuw: w: x: ALP>AuLIrT(c

n rO<E SETTINGS
x 8 x%'$r>AP$pho: o(
IT$'ATIONr 8 tzzzzz
L$A'NINGu'AT$ 8 z%zzzzzo

Wu>IrTO'Y 8 b[
n kOOP OVER LPH pS()
Fn. ALP>A hN ALP>AuLIrTc n r#k2/-#m

n 4R DIENT DESCENT LGORITH<)

n rTEP o) -NITI LIyE THE P R <ETERS v)
v 8 NP%MAT'IXpNP%y$'Orppt:o((( n r#k2/-#m
Fn. I hN 'ANG$pIT$'ATIONr(c
n rTEP 3) ] LCUL TE THE GR DIENT OF kpv( W)R)T) v)
G'AD 8 G'ADukuvu_$'3pw: x: v: ALP>A( n r#k2/-#m
n rTEP t) 2PD TE P R <ETERS v)
v 8 v h L$A'NINGu'AT$ ^ G'AD n r#k2/-#m

n 4ET THE P R <ETERS OF THE P R BOL )

Wz: Wo: W3 8 NP%A''AYpv(%'$r>AP$pho(
Wu>IrTO'Y%APP$NDppWz:Wo:W3((

.dT2.N NP%A''AYpWu>IrTO'Y(

-N bfo[c n dSTI< TE THE P R BOL S GIVEN THE LIST OF LPH pS()

ALP>Ar 8 bz: z%zt: z%zf: z%o: o[
Wu>IrTO'Y 8 LouL3uG'ADuD$rC$NTpOLDuw: w: x: ALP>AuLIrT8ALP>Ar(

n PRINT ND PLOT THE RESULT

PLT%rCATT$'pOLDuw: x: COLO'8"G'AY"(
Fn. ALP>A: Wr hN yIPpALP>Ar: Wu>IrTO'Y(c
Wz: Wo: W3 8 Wr

3
 n PLOT
wuLIN$ 8 NP%LINrPAC$pz:oz:tzz(
xuLIN$ 8 Wz . Wo ^ wuLIN$ . W3 ^ pwuLIN$^^3(
PLT%PLOTpwuLIN$: xuLIN$: LAB$L8"ALP>A8b{"%FO'MATpALP>A((

n PRINT
P'INTp"v>$N ALP>A 8 b{:"%FO'MATpALP>A((
P'INTp"Y 8 bcp3F{ . bcp3F{^X . bcp3F{^Xa3"%FO'MATpWz: Wo: W3((
PLT%L$G$NDp(
PLT%XLAB$Lp"X"(
PLT%YLAB$Lp"Y"(
PLT%r>OWp(

v>$N ALP>A 8 z:
Y 8 o%75 . o%of^X . o%zf^Xa3
v>$N ALP>A 8 z%zt:
Y 8 3z%e3 . hf%3s^X . o%ft^Xa3
v>$N ALP>A 8 z%zf:
Y 8 tz%os . he%5f^X . o%ss^Xa3
v>$N ALP>A 8 z%o:
Y 8 53%to . ho3%se^X . 3%oo^Xa3
v>$N ALP>A 8 o:
Y 8 fo%zs . ho7%z7^X . 3%t7^Xa3

37
) .2ESTION )5 bOMPARISON

bOMPARE THE PARABOLAS IN .%t% /RY TO EXPLAIN THE TREND FROM ∇ < . pI%E% 𝐴 NORM(: ∇ < ..: ∇ < ..4:
∇ < .: TILL ∇ <  pI%E% SQUARED 𝐴1 NORM(%

xOU MAY TYPE YOUR ANSWERS DIRECTLY IN THE lARjDOWN CELL BELOW%

gINTc xOU MAY NEED TO CONSIDER THE OUTLIERS IN THE DATA POINTS FOR YOUR REASONING%

onhlTrc t

sxP, xnt. lru,. g,.,: .,Pi ahle Tghr T,wTp

o% 𝐴 CURuE MODELS THE uALID DATA POINTS WELL% 𝐴1 AND 𝐴  𝐴1 CURuES LIE BETWEEN OUTLIERS AND uALID DATA
POINTS%
3% 𝐴1 LOSS CAUSES TOO MUCH PENALTY FROM OUTLIERS: WHICH MAjES OUTLIERS “PULL“ THE CURuE FAR AWAY FROM
uALID POINTS: WHILE 𝐴 LOSS” PENALTY FROM OUTLIERS IS RELATIuELY LESS TO jEEP THE CURuE LYING AMONG THE
uALID DATA POINTS%
t% -DEALLY 𝐴  𝐴1 CURuES SHOULD LIE BETWEEN 𝐴 CURuE AND 𝐴1 CURuE pAND INDEED IT IS(: BUT DUE TO THE
𝐴1 LOSS == 𝐴 LOSS: THE 𝐴1 PART IS DOMINATING IN THE 𝐴  𝐴1 LOSS: WHICH MAjES THE SHAPE OF 𝐴  𝐴1
CURuES MORE SIMILAR TO 𝐴1 CURuE pALMOST OuERLAPPED( THAN 𝐴 CURuE%

s dND OF 5

s)o r2aMISSION

lAjE SURE YOU HAuE RUN ALL CELLS IN YOUR NOTEBOOj IN ORDER BEFORE RUNNING THE CELL BELOW: SO THAT ALL
IMAGES6GRAPHS APPEAR IN THE OUTPUT% /HE CELL BELOW WILL GENERATE A yIP ffLE FOR YOU TO SUBMIT%

1LEASE MAjE SURE TO SEE THE OUTPUT OF THE GRADESCOPE AUTOGRADER% xOU ARE RESPONSIBLE FOR WAITING AND
ENSURING THAT THE AUTOGRADER IS EXECUTING NORMALLY FOR YOUR SUBMISSION% 1LEASE CREATE A 1IAyyA POST IF YOU
SEE ERRORS IN AUTOGRADER EXECUTION%

-N b [c G'AD$'%$XPO'TpPDF8e krd: FO'C$urA_$8s.2d: '2NuT$rTr8s.2d: FIL$r8b"IMGr": "'$Q2I'$M$NTr%TXT": "PA

3;