1

AI System Semiconductor Design

Lecture5 (In-person): Logistic Regression
3/19/2025
Lecturer: Taewook Kang
Acknowledgments
Lecture material adapted from
Prof. Woowhan Jung, DSLAB, Hanyang Univ.
PyTorchZeroToAll, Prof. Sung Kim, HKUST
Dr. Hyungjoo Seo, Apple, CA, USA

SKKU Kang Research Group / SEE3007 Spring 2025 1 1

 Class Schedule - Python Part
▪ Week2
▪ What is ML?
▪ Classification Vs. Regression
▪ Linear regression
▪ Week3
▪ Gradient descent for linear regression
▪ Logistic regression
▪ Week4
▪ Shallow Neural Networks
▪ Shallow Neural Networks – gradient descent
▪ Deep Neural Networks
▪ Stochastic Gradient Descent
▪ MNIST implementation

SKKU Kang Research Group / SEE3007 Spring 2025 2

 REVIEW – LOGISTIC REGRESSION

SKKU Kang Research Group / SEE3007 Spring 2025 3

 Binary Classification
𝒙 Model 𝑦ො

Output Label
Model =
0 Not a Cat
Architecture
1 Cat
+ Parameters
123
94
…
202
𝒙=
123
94
…
142
SKKU Kang Research Group / SEE3007 Spring 2025 4
 Learning Process
Many vector/matrix
operations

Input Model Output Label

Architecture 𝑃 𝐶𝑎𝑡 𝑃 𝐶𝑎𝑡

+ Parameters = 0.9 =1

Compute loss
Update parameters
to minimize the loss

SKKU Kang Research Group / SEE3007 Spring 2025 5

 Logistic Regression - Model

Model
Input Model Output Label

Architecture 𝑃 𝐶𝑎𝑡 𝑃 𝐶𝑎𝑡

+ Parameters = 0.9 =1

Compute loss
Update parameters
to minimize the loss

SKKU Kang Research Group / SEE3007 Spring 2025 6

 Logistic Regression
▪ A simple model for binary classification
▪ Maybe one of the simplest neural networks
▪ A training example 𝒙, 𝑦
▪ Input: 𝒙 ∈ ℝ𝑛
▪ Output: 𝑦 ∈ 0,1
▪ m training examples (or Mtrain)
▪ 𝒙 𝟏 ,𝑦 1 , 𝒙 2 ,𝑦 2 ,… 𝒙 𝑚 ,𝑦 𝑚

SKKU Kang Research Group / SEE3007 Spring 2025 7

 Review: Linear Regression
▪ Input: 𝒙 ∈ ℝ𝑛
▪ Goal: 𝑦ො ≈ 𝑦 Model = Architecture +
Parameters

𝑦ො = 𝒘⊤ 𝒙 + 𝑏

Parameters: 𝒘 ∈ ℝ𝑛 , 𝑏 ∈ ℝ

SKKU Kang Research Group / SEE3007 Spring 2025 8

 Logistic Regression
▪ Input: 𝒙 ∈ ℝ𝑛
▪ Goal: 𝑦ො = 𝑃 𝑦 = 1 𝒙 Model = Architecture +
Parameters

𝑦ො = 𝜎 𝒘⊤ 𝒙 + 𝑏

Parameters: 𝒘 ∈ ℝ𝑛 , 𝑏 ∈ ℝ

1
Sigmoid 𝜎 𝑥 =
1+𝑒 −𝑥
𝜎 −∞ = 0
𝜎 +∞ = 1

SKKU Kang Research Group / SEE3007 Spring 2025 9

10
Logistic Regression – Cost Function & Loss

Input Model Output Label

Architecture 𝑃 𝐶𝑎𝑡 𝑃 𝐶𝑎𝑡

+ Parameters = 0.9 =1

Compute loss
Update parameters
to minimize the loss Cost function & Loss
SKKU Kang Research Group / SEE3007 Spring 2025 10
 Logistic Regression: Cost Function
▪ Given 𝒙 𝟏 , 𝑦 1 , … 𝒙 𝑚 , 𝑦 𝑚 , want 𝑦ො (𝑖) = 𝑃(𝑦 𝑖 = 1) Roughly speaking , 𝑦ො (𝑖) ≈ 𝑦 𝑖

▪ Loss function: Binary Cross Entropy

𝐿 𝑦,
ො 𝑦 = −𝑦 log 𝑦ො − 1 − 𝑦 log 1 − 𝑦ො
If 𝑦 = 1: 𝐿 𝑦,
ො 𝑦 = − log 𝑦ො
If 𝑦 = 0: 𝐿 𝑦,
ො 𝑦 = − log 1 − 𝑦ො
− log 𝑦ො
▪ Cost function:
𝑚
1
𝐽 𝒘, 𝑏 = ෍ 𝐿 𝑦ො (𝑖) , 𝑦 (𝑖)
𝑚
𝑖=1
Logistic regression model

𝑦ො = 𝜎 𝒘⊤ 𝒙 + 𝑏
SKKU Kang Research Group / SEE3007 Spring 2025 11
12
Logistic Regression – Optimization

Input Model Output Label

Architecture 𝑃 𝐶𝑎𝑡 𝑃 𝐶𝑎𝑡

+ Parameters = 0.9 =1

Compute loss
Update parameters
to minimize the loss
Optimization
SKKU Kang Research Group / SEE3007 Spring 2025 12
 Optimization
▪ Logistic regression model
1
▪ 𝑦ො = 𝜎 𝒘⊺𝒙 + 𝑏 , where 𝜎 𝑧 = 1+𝑒−𝑧
▪ Cost function
𝑚 𝑚
1 (𝑖) (𝑖)
1
𝐽 𝒘, 𝑏 = ෍ 𝐿 𝑦ො , 𝑦 = − ෍ 𝑦 (𝑖) log 𝑦ො (𝑖) + 1 − 𝑦 (𝑖) log 1 − 𝑦ො (𝑖)
𝑚 𝑚
𝑖=1 𝑖=1

▪ Our goal
▪ Find parameters 𝒘 ∈ ℝ𝑛 , 𝑏 ∈ ℝ that minimize 𝐽 𝒘, 𝑏
▪ Gradient Descent!

SKKU Kang Research Group / SEE3007 Spring 2025 13

 Review - Gradient Descent
▪ Algorithm to minimize a cost function 𝐽 𝜃 θ = {𝒘, 𝑏}
▪ 𝐽 𝜃 : cost function
▪ 𝜃: model parameters
▪ 𝜂: Learning rate

Repeatedly update

𝜃 ∗ = 𝜃- 𝜂 ⋅ 𝛻𝜃 𝐽(𝜃)
𝜃∗
By Hakky St
SKKU Kang Research Group / SEE3007 Spring 2025 14
 Computing the Parameters with Gradient Decent
𝑦ො = 𝜎 𝒘⊤ 𝒙 + 𝑏
∗
𝑚 𝜃 = 𝜃- 𝜂 ⋅ 𝛻𝜃 𝐽(𝜃)
1
𝐽 𝒘, 𝑏 = ෍ 𝐿 𝑦ො (𝑖) , 𝑦 (𝑖)
𝑚
𝑖=1

𝜕𝐽 𝜃
𝜕𝑤1
𝑤1 𝜕𝐽 𝜃
𝑤2 𝜕𝑤2
𝜃= … 𝛻𝜃 𝐽 𝜃 = …
𝑤𝑛 𝜕𝐽 𝜃
𝑏 𝜕𝑤𝑛
𝜕𝐽 𝜃
𝜕𝑏

SKKU Kang Research Group / SEE3007 Spring 2025 15

 GRADIENT DESCENT
: LOGISTIC REGRESSION

SKKU Kang Research Group / SEE3007 Spring 2025 16

 Background: Derivatives

𝑑 𝑑 1
1. log e 𝑥 = 𝑙𝑛 𝑥 =
𝑑𝑥 𝑑𝑥 𝑥

𝑑
2. 𝜎 𝑥 = 𝜎 𝑥 (1 − 𝜎 𝑥 )
𝑑𝑥

1 𝑑 −1
𝜎 𝑧 =
1 + 𝑒 −𝑧
𝜎 𝑥 = −𝑒 −𝑧 = 𝜎 𝑥 (1 − 𝜎 𝑥 )
𝑑𝑥 1+𝑒 −𝑧 2

SKKU Kang Research Group / SEE3007 Spring 2025 17

 Logistic Regression Recap
1
▪ 𝑦ො = 𝜎 𝒘 𝒙 + 𝑏 where 𝜎 𝑧 = 1+𝑒 −𝑧
⊺

▪ 𝐿 𝑦,
ො 𝑦 = −𝑦 log 𝑦ො − 1 − 𝑦 log 1 − 𝑦ො
For the simplicity
𝑥1 𝑤1
𝒙= 𝑥 ,𝒘= 𝑤
2 2
𝑥1
𝑥2
𝑤1 𝑧 = 𝑤1 𝑥1 + 𝑤2 𝑥2 + 𝑏 𝑦ො = 𝑎 = 𝜎 𝑧 𝐿 𝑎, 𝑦
𝑤2
𝑏
𝑦
Target to learn

SKKU Kang Research Group / SEE3007 Spring 2025 18

 Logistic Regression Derivative
𝐿 𝑎, 𝑦 = −𝑦 log 𝑎 − 1 − 𝑦 log 1 − 𝑎

𝑦
𝑥1
𝑥2
𝑧 = 𝑤1 𝑥1 + 𝑤2 𝑥2 + 𝑏 𝑦ො = 𝑎 = 𝜎 𝑧 𝐿 𝑎, 𝑦
𝑤1
𝑑𝑧 𝑑𝑎 𝑑𝐿
𝑤2 𝑑𝑤𝑘 𝑑𝑧 𝑑𝑎

𝑏 𝑑𝐿 𝑑𝐿 𝑑𝑎 𝑑𝑧 𝑦 1−𝑦
= = − + 𝜎 𝑧 (1 − 𝜎 𝑧 )𝑥𝑘
𝑑𝑤𝑘 𝑑𝑎 𝑑𝑧 𝑑𝑤𝑘 𝑎 1−𝑎

SKKU Kang Research Group / SEE3007 Spring 2025 19

 Logistic Regression Derivative
𝐿 𝑎, 𝑦 = −𝑦 log 𝑎 − 1 − 𝑦 log 1 − 𝑎

𝑦
𝑥1
𝑥2
𝑧 = 𝑤1 𝑥1 + 𝑤2 𝑥2 + 𝑏 𝑦ො = 𝑎 = 𝜎 𝑧 𝐿 𝑎, 𝑦
𝑤1
𝑤2
𝑏 𝑑𝐿 𝑑𝐿 𝑑𝑎 𝑑𝑧 𝑦 1−𝑦
= = − + 𝜎 𝑧 (1 − 𝜎 𝑧 )𝑥𝑘
𝑑𝑤𝑘 𝑑𝑎 𝑑𝑧 𝑑𝑤𝑘 𝑎 1−𝑎
𝑦 1−𝑦
= − + 𝑎(1 − 𝑎)𝑥𝑘 = −𝑦 1 − 𝑎 𝑥𝑘 + 1 − 𝑦 𝑎𝑥𝑘
𝑎 1−𝑎

= 𝑎 − 𝑦 𝑥𝑘
SKKU Kang Research Group / SEE3007 Spring 2025 20
 Logistic Regression Derivative – Calculate on your own
𝑑𝐿
=
𝑑𝑤1

𝑑𝐿
=
𝑑𝑤2

𝑑𝐿
=
𝑑𝑏

SKKU Kang Research Group / SEE3007 Spring 2025 21

 Logistic Regression Derivative
𝐿 𝑎, 𝑦 = −𝑦 log 𝑎 − 1 − 𝑦 log 1 − 𝑎

𝑦
𝑥1
𝑥2
𝑧 = 𝑤1 𝑥1 + 𝑤2 𝑥2 + 𝑏 𝑦ො = 𝑎 = 𝜎 𝑧 𝐿 𝑎, 𝑦
𝑤1
𝑤2
𝑏 𝑑𝐿 𝑑𝐿 𝑑𝑎 𝑑𝑧 𝑑𝐿
= = 𝑎 − 𝑦 𝑥1 𝑤1∗ ≔ 𝑤1 − 𝜂
𝑑𝑤1 𝑑𝑎 𝑑𝑧 𝑑𝑤1 𝑑𝑤1
𝑑𝐿 𝑑𝐿 𝑑𝑎 𝑑𝑧 𝑑𝐿
= = 𝑎 − 𝑦 𝑥2 𝑤2∗ ≔ 𝑤2 − 𝜂
𝑑𝑤2 𝑑𝑎 𝑑𝑧 𝑑𝑤2 𝑑𝑤2

𝑑𝐿 𝑑𝐿 𝑑𝑎 𝑑𝑧 𝑑𝐿
= =𝑎−𝑦 𝑏∗ ≔𝑏−𝜂
𝑑𝑏 𝑑𝑎 𝑑𝑧 𝑑𝑏 𝑑𝑏
SKKU Kang Research Group / SEE3007 Spring 2025 22
 Gradient descent on m examples
𝑚
1
𝐽 𝒘, 𝑏 = ෍ 𝐿 𝑦ො (𝑖) , 𝑦 (𝑖)
𝑚
𝑖=1

𝑚 𝑚
𝑑 1 𝑑 (𝑖) (𝑖)
1
𝐽 𝒘, 𝑏 = ෍ 𝐿 𝑦ො , 𝑦 = ෍ 𝑦ො (𝑖) − 𝑦 𝑥𝑘
𝑑𝑤𝑘 𝑚 𝑑𝑤𝑘 𝑚
𝑖=1 𝑖=1

𝑚 𝑚
𝑑 1 𝑑 (𝑖) (𝑖)
1
𝐽 𝒘, 𝑏 = ෍ 𝐿 𝑦ො , 𝑦 = ෍ 𝑦ො (𝑖) − 𝑦
𝑑𝑏 𝑚 𝑑𝑏 𝑚
𝑖=1 𝑖=1

SKKU Kang Research Group / SEE3007 Spring 2025 23

 Logistic Regression on #m Examples
▪ Initialize 𝒘, 𝑏
▪ lr = 0.1
▪ For e=1 to 𝑛𝑒𝑝𝑜𝑐ℎ :
▪ J = 0; d_w1 = 0; d_w2 = 0; d_b=0
▪ For i=1 to m:
▪ 𝑧 = 𝑤1𝑥1(𝑖) + 𝑤2𝑥2(𝑖) + 𝑏 𝑚 𝑚
𝑑 1 𝑑 1
▪𝑎 =𝜎 𝑧 𝑑𝑤2
𝐽 𝒘, 𝑏 = ෍
𝑚 𝑑𝑤2
(𝑖)
𝐿 𝑎 ,𝑦 (𝑖) = ෍ 𝑎(𝑖) − 𝑦 (𝑖) 𝑥2
𝑚
𝑖=1 𝑖=1
▪ d_w1+= 𝑎 − 𝑦 𝑥1(𝑖)
𝑚 𝑚
𝑑 1 𝑑 1
▪ d_w2+= 𝑎 − 𝑦 (𝑖)
𝑥2 𝐽 𝒘, 𝑏 = ෍ (𝑖)
𝐿 𝑎 ,𝑦 (𝑖) = ෍ 𝑎(𝑖) − 𝑦 (𝑖) 𝑥2
𝑑𝑤2 𝑚 𝑑𝑤2 𝑚
▪ d_b+= 𝑎 − 𝑦 𝑖=1 𝑖=1

▪ 𝑤1 −= 𝑙𝑟 ∗ 𝑑_𝑤1/m 𝑑 1 𝑑
𝑚
1
𝑚
(𝑖) (𝑖) = ෍ 𝑎(𝑖) − 𝑦 (𝑖)
▪ 𝑤2 −= 𝑙𝑟 ∗ 𝑑_𝑤2/m 𝑑𝑏
𝐽 𝒘, 𝑏 = ෍ 𝐿 𝑎 , 𝑦
𝑚 𝑑𝑏 𝑚
𝑖=1 𝑖=1
▪ 𝑏 −= 𝑙𝑟 ∗ 𝑑_𝑏/m

SKKU Kang Research Group / SEE3007 Spring 2025 24

 Terminology: Epoch, batch size, iteration
Batch size (=3) Whole dataset (e.g., m=12)

1 Epoch

▪ If the training ends up using a whole dataset, it corresponds to 1 epoch training

▪ Due to the memory limit, we split #m data to smaller size (batch size)
▪ Running 1 iteration over batch_size data
▪ For 1 epoch training, #m/batch_size of iteration required
▪ E.g., to do 1 epoch training, 12/3 = 4 Iteration loop should be done
▪ After every one iteration training with #batch_size data, the parameter of model is
updated

SKKU Kang Research Group / SEE3007 Spring 2025 https://m.blog.naver.com/qbxlvnf11/22144929703 25

 Linear regression vs Logistic regression
𝑦 Linear
𝑥1 Regression
𝑥2 𝑦ො = 𝑎 = 𝑤1 𝑥1 + 𝑤2 𝑥2 + 𝑏 𝐿 𝑎, 𝑦
𝑤1 2
𝐿 𝑦,
ො 𝑦 = 𝑦ො − 𝑦
𝑤2
𝜕𝐿 𝜕𝐿 𝜕𝑎 𝜕𝐿 𝜕𝐿 𝜕𝑎
𝑏 = = 2 𝑎 − 𝑦 𝑥𝑘 = =2 𝑎−𝑦
𝜕𝑤𝑘 𝜕𝑎 𝜕𝑤𝑘 𝜕𝑏 𝜕𝑎 𝜕𝑏 For the simplicity
𝑥1 𝑤1
𝒙= 𝑥 ,𝒘= 𝑤
2 2
𝑥1
𝑥2 𝑧 = 𝑤1 𝑥1 + 𝑤2 𝑥2 + 𝑏 𝑦ො = 𝑎 = 𝜎 𝑧 𝐿 𝑎, 𝑦
𝑤1
𝑤2 𝐿 𝑦,
ො 𝑦 = −𝑦 log 𝑦ො − 1 − 𝑦 log 1 − 𝑦ො
𝑦
𝑏
𝜕𝐿 𝜕𝐿 𝜕𝑎 𝜕𝑧 𝜕𝐿 𝜕𝐿 𝜕𝑎 𝜕𝑧 Logistic
= = 𝑎 − 𝑦 𝑥𝑘 = = 𝑎−𝑦
𝜕𝑤𝑘 𝜕𝑎 𝜕𝑧 𝜕𝑤𝑘 𝜕𝑏 𝜕𝑎 𝜕𝑧 𝜕𝑏 Regression
SKKU Kang Research Group / SEE3007 Spring 2025 26
 ASSIGNMENT 3 REVIEW

SKKU Kang Research Group / SEE3007 Spring 2025 27

28
Lecture3 Assignment
▪ Implement linear regression with the given dataset below (x_data, y_data). Copy these 5 lines in top
of your code.

m_train=1000
Model definition
w=2 𝑦ො = 𝑤 ∗ 𝑥 + 𝑏
b=3 𝑚_𝑡𝑟𝑎𝑖𝑛
x_data=10*np.array(range(m_train))/m_train 1 2
y_data=x_data*w+np.random.randn(m_train)+b 𝑙𝑜𝑠𝑠 = 𝑀𝑆𝐸 = ෍ 𝑦ො𝑛 − 𝑦𝑛
𝑚_𝑡𝑟𝑎𝑖𝑛
𝑛=1

▪ Report guidelines (doesn’t have to be long. make it easy to read and short)
▪ Plot training data
▪ Describe the data shape in a few sentences. E.g., how does it look like?
▪ Guess your fitting results without coding and explain your reasoning for the guess
▪ Attach your full code
▪ Show the training result → w, b, cost (J)
▪ How many times did you repeat training(number of epochs) to get a good result?
▪ Submit to iCampus
▪ Due: 3/19, 8:59AM

SKKU Kang Research Group / SEE3007 Spring 2025 28

 Lecture3 Assignment
Answer
import numpy as np
import matplotlib.pyplot as plt
▪ Training data is in linear
m_tra in=100 0
w=2
shape with added noise
b=3
x_d ata =10*np.array(ran ge(m_tr ain))/m_train
y_d ata =x_d ata *w+b+np.ran dom.r andn(m_tra in)

def forward_cal(w, b, x) :
▪ The fitting model
return x*w+b
def loss_cal(w, b, x, y, y_pred) :
return (y_pred-y)**2
supposed to be
▪ w=2, b=3
def w_g radient(w, b, x, y, y_pred) :
return 2*x*(y_pred-y)
def b_grad ient(w, b, x, y, y_pred) :
return 2*(y_pr ed-y)

w=0
b=0
lr=0.02
Nepoch=400
▪ You may observe the
# Training loop
for epo ch in ran ge(Nepoch):
# initialize
training diverges if you
d_w=0
d_b=0
loss=0
give too large learning
for x_val, y_val in zip(x_data, y_d ata ):
y_p red = forward_ca l(w, b, x_val)
d_w += w_gr adient(w, b, x_val, y_val, y_pred )
rate.
d_b += b _gr adient(w, b, x_val, y_val, y_pred )
loss += loss_cal(w, b, x_val, y_val, y_pred )
loss = lo ss/m_train
w = w - lr * d_w/m_train
b = b - lr * d_b/m_train
if epo ch%100==0:
prin t("e poch=% d, pre viou s_loss=%f, w'=% f, b'=%f" %( epoch, loss, w, b))

plt.figu re()
plt.plot(x_data, y_data, '*', labe l='Training data')
plt.plot(x_data, w*x_da ta+b, labe l='Trained model')
plt.xlab el('Input x')
plt.ylab el('Output y')
plt.legend( )
plt.sho w()

SKKU Kang Research Group / SEE3007 Spring 2025 29

 Large Learning Rate can be Unstable
▪ With a too-large learning rate, the parameter can
move back and forth
▪ You can see w trace is oscillating without bound
▪ We can qualitatively explain by the below graph

SKKU Kang Research Group / SEE3007 Spring 2025 30

 PYTHON PRACTICE

SKKU Kang Research Group / SEE3007 Spring 2025 31

 Move on to Jupyter Notebook
▪ Let’s use Jupyter Notebook for the class for easier use
▪ Also, it gives us better interactive environment
▪ PyCharm has powerful debugging ability, but in our class scope, we may not need
that complexity.
▪ Install Jupyter

▪ Launch Jupyter

SKKU Kang Research Group / SEE3007 Spring 2025 32

 Jupyter Notebook – Create a New Code

Select “Notebook”

Select “Python 3”

SKKU Kang Research Group / SEE3007 Spring 2025 33

 Jupyter Notebook Tip - Documentation

“Shift+Enter”
at each
Markdown line

▪ Change the line type from as “Markdown”

▪ Shift+Enter at each line makes it a text
▪ Put “#” in your comment
▪ It will help your documenting code
▪ If you put “##”, it is a sub-hierarchy of # line
SKKU Kang Research Group / SEE3007 Spring 2025 34
 Logistic regression: logical AND

x1 x2 y

0 0 0

0 1 0

1 0 0

1 1 1

𝑦 = 𝑥1 𝐴𝑁𝐷 𝑥2

SKKU Kang Research Group / SEE3007 Spring 2025 35

 Data Preparation

▪ You may wonder why we only have 4 training samples.

▪ The number of samples won’t matter for logistic regression. We can still train our model
with only 4 samples.
▪ But we will explore real-looking data in next week, too.

SKKU Kang Research Group / SEE3007 Spring 2025 36

 Data Plot
▪ Boolean value of training data is represented with
color
▪ If the label coding part is difficult, you can also use the
below version without the label.

SKKU Kang Research Group / SEE3007 Spring 2025 37

 Model

SKKU Kang Research Group / SEE3007 Spring 2025 38

 If you haven’t heard of Python “class”
▪ “class” will make your coding a lot easier → Object Oriented Programming
▪ This is a must!
▪ Every programmer uses “class”
▪ Korean tutorial
▪ https://wikidocs.net/16071
▪ English tutorial
▪ https://www.hackerearth.com/practice/python/object-oriented-programming/classes-and-objects-
i/tutorial/#:~:text=A%20class%20is%20a%20code,the%20instance%20of%20the%20class.

SKKU Kang Research Group / SEE3007 Spring 2025 39

 Training

model = logistic_regression_model() #This will initialize “model”

Nepoch=1000
Cost_list=[]
for epoch in range(Nepoch):
cost = ~~
cost_list.append(cost)
SKKU Kang Research Group / SEE3007 Spring 2025 40
 Testing
▪ Result analysis
▪ model.predict((0, 0)) → 7.1e-6
▪ It means when you give (0, 0), the result is 7.1e-6
▪ Hence, our trained model tells us the output is 0.
▪ model.predict((1, 1)) → 0.98
▪ Our model predicts the output as 0.98, which means 1
▪ The model predicts output well.

SKKU Kang Research Group / SEE3007 Spring 2025 41

 LOGISTIC REGRESSION
ASSIGNMENT

SKKU Kang Research Group / SEE3007 Spring 2025 42

 Logistic Regression: Boolean Operators
▪ Repeat the practice for OR and XOR operators, too.
▪ Requirements
▪ AND, OR, XOR
▪ Use a dataset for each operator →
▪ The model may not work for some operators
▪ We will cover the details in the lecture 6
▪ Use numpy arrays
▪ Initialization with lists: x, y
▪ Random initialization: w, b
▪ Use numpy operator
▪ Inner product
▪ Addition

SKKU Kang Research Group / SEE3007 Spring 2025 43

 Lecture 5 Assignment
▪ Logistic regression on Boolean data cost
▪ Report required contents
▪ Add source code for model & training
▪ For each operator (AND, OR, XOR):
▪ Cost plot for epoch epoch
▪ Use different learning rates (at least 3 learning rates)
▪ Show predicted results for whole input combinations→ e.g.,

▪ Explain whether the logistic regression model works well for AND, OR, XOR data
▪ For one operator, the logistic regression won’t work. Which one?
▪ Due: 2024/3/27 (Thu) 11:59 PM (after in-person lecture07)
▪ So, you have two more classes (including today) to ask questions to finish.
▪ Submit to iCampus

SKKU Kang Research Group / SEE3007 Spring 2025 44

 NumPy usages

Import numpy

Initialization with lists

Inner product

Addition

Scalar multiplication

Radom initialization with

𝑁(0,1)

SKKU Kang Research Group / SEE3007 Spring 2025 45

 Preview

SKKU Kang Research Group / SEE3007 Spring 2025 46

 Limitations of logistic regression
▪ Logistic regression does not work well with the XOR dataset
▪ Think about why. How do we classify the data? (Hint: we are using a straight line so far)

▪ We need a more complicated model!

▪ Neural Network

SKKU Kang Research Group / SEE3007 Spring 2025 47