Creative Programming

Spring 2025
CUL1122 Lecture #14
 Statistical Problems:

Introduction to Machine Learning

Today

❖Machine Learning
▪ Traditional Programming vs. Machine Learning: A Comparison
▪ The Basic Paradigm of Machine Learning
❖Understanding Distance Measures
❖Feature Representation and Engineering
❖Exercise: Classifying Reptiles

3
Machine Learning

❖A computer program that ‘automatically learns’ something.

❖Early definition of machine learning:
▪ “A field of study that gives computers the
ability to learn without being explicitly
programmed.” – Arthur Samuel (1959)
❖Arthur Samuel, a computer pioneer,
wrote the first self-learning program,
which played checkers and learned from experience.
❖In 1956, his checker program was developed for play on the IBM 701
computer, and it was demonstrated to the public on television.
4
Traditional Programming vs. Machine Learning

❖In traditional programming, a programmer provides instructions to the

computer.
❖A program consists of a series of commands that tell the computer
what to do and in what order.

5
Traditional Programming vs. Machine Learning

❖Machine learning is an automated process enabling computers to solve

problems through data analysis, not preset programs.
❖In machine learning, we provide a sample set of input-output pairs,
which allows the system to learn a method for mapping inputs to correct
outputs, effectively creating a program.

6
Basic Paradigm of Machine Learning

❖Observe a set of samples known as training data.

❖Infer something about the process that generated the data.
❖Use this inference to make predictions on previously unseen test data.

7
Basic Paradigm of Machine Learning

❖Variations on the Paradigm:

▪ 1) Supervised Learning: Given a set of feature-label pairs, the goal is to find a
rule that predicts the label associated with a previously unseen input.
▪ 2) Unsupervised Learning: Given a set of feature vectors without labels, the
objective is to group them into “natural clusters.”
Supervised (w/ labels) Unsupervised (w/o labels)

8
Basic Paradigm

❖Examples of Two Variations in Machine Learning Techniques:

9
Supervised Learning

❖1) Classification: Predict a discrete value (label) associated with a

feature vector.
❖2) Regression: Predict a continuous value (real number) associated with
a feature vector.

10
How Should We Classify the Data?

❖We aim to determine the “similarity” of examples, with the goal of

predicting the label associated with a previously unseen input.
❖Similarity refers to the quality or
state of being similar, characterized
by likeness or resemblance, such as
a similarity of features.
❖While it is challenging to define,
similarity functions as a distance
measure in machine learning.

11
Defining Distance Measures
Definition: Let O1 and O2 be two objects from the universe of possible objects.
The distance (dissimilarity) between O1 and O2 is a real number denoted by D(O1, O2)

gene1
gene2

0.23 3 342.7

12
Defining Distance Measures

❖Euclidian distance ❖Minkowski distance

13
Attribute-Based Labeling through Distance Measure

❖For example, in the following scenario, similarity is determined based

on ear shape and nose size.

14
Feature Representation

❖Features never fully describe a situation.

▪ For example, ear shape and nose size alone cannot fully describe dogs or cats.
❖Feature engineering involves representing examples using feature
vectors, which facilitates generalization.
❖For instance, suppose you want to use 100 existing samples to predict
which students will receive an A in this course.
▪ Some features are undoubtedly helpful, such as GPA and prior programming
experience (though not perfect predictors).
▪ However, others might lead to overfitting, such as birth month or eye color.

15
An Example Process of Feature Representation

❖Initial model with 5 features

Features Label
Name Egg-laying Scales Poisonous Cold- # legs Reptile
blooded

Cobra True True True True 0 Yes

Rattlesnake True True True True 0 Yes
Boa False True False True 0 Yes The Boa does not
constrictor
fit this model.
Chicken True True False False 2 No
Alligator True True False True 4 Yes
Dart frog True False True False 4 No
Salmon True True False True 0 No
Python True True False True 0 Yes
16
An Example Process of Feature Representation

❖Refined model with 3 features: scales, cold-blooded, 0 leg

Features Label
Name Egg-laying Scales Poisonous Cold- # legs Reptile
blooded

Cobra True True True True 0 Yes

Rattlesnake True True True True 0 Yes
Boa False True False True 0 Yes
constrictor

Chicken True True False False 2 No

Alligator True True False True 4 Yes The Alligator does
Dart frog True False True False 4 No not fit this model.
Salmon True True False True 0 No
Python True True False True 0 Yes
17
An Example Process of Feature Representation

❖Refined model with 3 features: scales, cold-blooded, 0 or 4 legs

Features Label
Name Egg-laying Scales Poisonous Cold- # legs Reptile
blooded

Cobra True True True True 0 Yes

Rattlesnake True True True True 0 Yes
Boa False True False True 0 Yes
constrictor

Chicken True True False False 2 No

Alligator True True False True 4 Yes
Dart frog True False True False 4 No
Salmon True True False True 0 No No (easy) way to
Python True True False True 0 Yes
classify salmon
and python.
18
An Example Process of Feature Representation

❖Current model: scales, cold-blooded; not perfect but no false negatives

Features Label
Name Egg-laying Scales Poisonous Cold- # legs Reptile
blooded

Cobra True True True True 0 Yes - Anything classified

as “not reptile” is
Rattlesnake True True True True 0 Yes
correctly labeled.
Boa False True False True 0 Yes
constrictor - Some animals may
be incorrectly
Chicken True True False False 2 No labeled as reptiles.
Alligator True True False True 4 Yes
Dart frog True False True False 4 No
Salmon True True False True 0 No
Python True True False True 0 Yes
19
Feature Engineering

❖We need to measure the distance between features.

❖This involves deciding which features to include and identifying those
that may add noise to the classifier.
❖Additionally, we must define how to measure distances between
training examples, which extends to classifiers and new instances.
❖Furthermore, we need to determine how to weigh the relative
importance of different dimensions of the feature vector, as this affects
the definition of distance.

20
Measuring Distance between Animals

❖We can consider our animal samples as consisting of four binary

features and one integer feature.
Name Egg-laying Scales Poisonous Cold- # legs Reptile
blooded

Rattlesnake True True True True 0 Yes

Boa False True False True 0 Yes
constrictor

Dart frog True False True False 4 No

Rattlesnake = [1,1,1,1,0]
Boa constrictor = [0,1,0,1,0]
Dart frog = [1,0,1,0,4]

21
Euclidean Distance between Animals

❖One way to distinguish between reptiles and non-reptiles is by

measuring the distance between pairs of samples and clustering nearby
samples into a common class for unlabeled data.
❖For example, using Euclidean distance, a rattlesnake and a boa
constrictor are much closer to each other than either is to a dart frog.
Rattlesnake = [1,1,1,1,0] Rattlesnake Boa constrictor Dart frog
Boa constrictor = [0,1,0,1,0] Rattlesnake - 1.414 4.243
Dart frog = [1,0,1,0,4]
Boa 1.414 - 4.472
constrictor
Dart frog 4.243 4.472 -

22
Add an Alligator

alligator = Animal(‘alligator’, [1,1,0,1,4])

animals.append(alligator)
compareAnimals(animals, 3)

23
Add an Alligator

❖The alligator is closer to the dart frog than to the snakes. Why?
▪ The alligator differs from the dart frog in three features, whereas it differs from
the boa constrictor in only two features.
▪ However, the scale for the “legs” feature ranges from 0 to 4, while the scales
for the other features range from 0 to 1.
▪ As a result, the “legs” dimension is disproportionately large.
Rattlesnake Boa constrictor Dart frog Alligator
Rattlesnake - 1.414 4.243 4.123
Boa constrictor 1.414 - 4.472 4.123
Dart frog 4.243 4.472 - 1.732
Alligator 4.243 4.123 1.732 -
24
Using Binary Features

❖Now, the alligator is closer to snakes than it is to the dart frog.

❖This highlights the importance of feature engineering!
Rattlesnake = [1,1,1,1,0] Rattlesnake = [1,1,1,1,0]
Boa constrictor = [0,1,0,1,0] Boa constrictor = [0,1,0,1,0]
Dart frog = [1,0,1,0,4] Dart frog = [1,0,1,0,1]
A11igator = [1,1,0,1,4] A11igator = [1,1,0,1,1]

Rattlesnake Boa constrictor Dart frog Alligator

Rattlesnake - 1.414 1.732 1.414
Boa constrictor 1.414 - 2.236 1.414
Dart frog 1.732 2.236 - 1.732
Alligator 1.414 1.414 1.732 -

25
Exercise: Classifying Reptiles
Exercise #1: Reptile Classification

❖Develop a reptile classification model.

▪ Create a script that categorizes animals as reptiles or non-reptiles based on five
features: egg-laying, scales, poisonous, cold-blooded, and number of legs.
▪ Use Euclidean distance to calculate the similarity between animals and display
the results in the table below.

27
Exercise #1: 1) Define the Animal Class

❖Define the Animal class, which should include a feature vector and a
method for measuring the distance between features.
class Animal(object):
def __init__(self, name, features):
# Assume name a string; features a list of numbers
self.name = name
self.features = numpy.array(features)

def distance(self, other):

# Return the Euclidean distance between feature vectors of self and other
return math.dist(self.getFeatures(), other.getFeatures())
…

28
Exercise #1: 2) Calculate Similarity between Animals

❖Define a function that computes the similarity between animals.

def compareAnimals(animals, precision):
…
# Get distances between pairs of animals
for a1 in animals: # For each row
row = []
for a2 in animals: # For each column
if a1 == a2:
row.append('--')
else:
distance = a1.distance(a2)
row.append(str(round(distance, precision)))
tableVals.append(row)
….
29
Exercise #1: 3) Add Samples

❖Provide animal samples to calculate their similarities.

rattlesnake = Animal('rattlesnake', [1, 1, 1, 1, 0])
boa = Animal(‘boa_constrictor', [0, 1, 0, 1, 0])
dartFrog = Animal('dart frog', [1, 0, 1, 0, 4])
animals = [rattlesnake, boa, dartFrog]
compareAnimals(animals, 3)

alligator = Animal('alligator', [1, 1, 0, 1, 4])

animals.append(alligator)
compareAnimals(animals, 3)

30
Exercise #1: 4) Improve Features

❖Change “Number of Legs” to a boolean indicating leg presence, and

recalculate the animal similarities.
rattlesnake = Animal('rattlesnake', [1, 1, 1, 1, 0])
boa = Animal(‘boa_constrictor', [0, 1, 0, 1, 0])
dartFrog = Animal('dart frog', [1, 0, 1, 0, 1])
alligator = Animal('alligator', [1, 1, 0, 1, 1])
animals = [rattlesnake, boa, dartFrog, alligator]
compareAnimals(animals, 3)

31
수고하셨습니다!

32