Applied Machine Learning

Introduction to the Course

Richard Johansson
[email protected]
 Welcome to the course!

· Machine learning is increasingly popular among students

· our courses take increasing volumes
· many thesis projects develop or apply ML models
· …and in industry, public sector
· many companies come to us looking for students
· joint research projects
· Why the fuss and why now?
Success stories: image recognition
Success stories: machine translation
Data

[source]
[source]
Applications…

[source]
 Topics covered in the course

· The usual “zoo”: a selection of machine learning models

· what’s the idea behind them?
· how are they implemented? (at least on a high level)
· what are the use cases?
· how can we apply them practically?
· But hopefully also the “real-world context”:
· extended “messy” practical assignments requiring that you
think of what you’re doing
· invited talks from industry and/or the healthcare sector
· annotation of data, evaluation
· ethical and legal issues, interpretability
 Overview

Practical issues about the course

Fundamental concepts in machine learning

Machine learning libraries in Python

 Course webpage

· The official course webpage is the Canvas page

https://chalmers.instructure.com/courses/33104
 People involved in the course

· Richard: examiner, responsible for the course

· Anton, Jack, Newton, Selma, Philipp, Laleh, Styrbjörn: helping
you with the assignments
 Structure of teaching

· Lecture discussions Tuesdays and Fridays 13–15

· we will use a flipped classroom format with pre-recorded
lectures you are expected to have watched before the session
· summary and discussion of the content of the recorded
lectures
· interactive coding
· solving a few exercises when we have time
· feel free to ask questions before the session!
· Assistance sessions Thursdays 13–17
· our TAs help you work on your assignments
· please let me know if it’s too crowded
· in a computer lab room (with possibly additional remote
sessions)
 Assignments

· Five compulsory assignments:

PA 1 intro to the ML workflow, decision trees
PA 2 random forests
PA 3 text classification
PA 4 neural network software
PA 5 medical image classification
· We will use the Python programming language
· Please refer to the course PM for details about grading
· Assignments are done in groups
 Programming assignment 1

· Warmup lab exercise: quick tour of the scikit-learn library

· Introduction to decision trees
· For a high grade: implement decision tree regression
· Assistance sessions this Thursday
· Submission deadline: January 27
 Literature

· We won’t follow a book closely, but we’ll give pointers to

reading material in this book:
· Machine Learning: A course for engineers and scientists by
Lindholm et al: http://smlbook.org/
· And additional papers to read for some topics
· Some notes to complement the lectures
· Example code will be posted on the course page
 Additional material along the way

· Exercise sheets, old exams

· Online quizzes
 Exam, mid-March

· This is a take-home exam: a written assignment

· Will be available during the whole exam period
· Two-part structure:
1. a first compulsory part about basic concepts: you need to
answer most of these questions correctly to pass
2. a second optional part that requires more insight: answer these
questions for a higher grade
 Student representatives

· If you’re interested in being a student representative, please

send me an email!
· The workload is light and there will be a small reward…
 Overview

Practical issues about the course

Fundamental concepts in machine learning

Machine learning libraries in Python

 Predictive models

· Given some object, make a prediction

· is this patient diabetic?
· what animal does this image show?
· what is the market value of this apartment?
· what are the phonemes contained in this speech signal?
 Predictive models

· Given some object, make a prediction

· is this patient diabetic?
· what animal does this image show?
· what is the market value of this apartment?
· what are the phonemes contained in this speech signal?
· The goal of machine learning is to build the predictive models
by observing data
 Predictive models

· Given some object, make a prediction

· is this patient diabetic?
· what animal does this image show?
· what is the market value of this apartment?
· what are the phonemes contained in this speech signal?
· The goal of machine learning is to build the predictive models
by observing data
· Contrast: expert-defined or data-driven

[source]
 Why machine learning?

Why would we want to “learn” the function from data instead of

just implementing it?
· Usually because we don’t really know how to write down the
function by hand
· speech recognition
· image classification
· machine translation
·…
· Might not be necessary for limited tasks where we know
· What is more expensive in your case? knowledge or data?
 Don’t forget your domain expertise!

ML makes some tasks automatic, but we still need our brains:

· defining the tasks, terminology, evaluation metrics
· annotating (hand-labeling) training and testing data
· designing features
· error analysis
 Example: is the patient diabetic?

In order to predict, we make some measurements of

properties we believe will be useful: these are called the
features
 Example: is the patient diabetic?

· In order to predict, we make some measurements of

properties we believe will be useful: these are called the
features
 More terminology: what is the output?

· Classification: learning to output a category label

· spam/non-spam; positive/negative; …
· Regression: learning to guess a number
· value of a share; number of stars in a review; …
 How is the training signal provided?

· In supervised learning, the training set consists of

input–output pairs
· our goal is to learn to produce the outputs
 Types of supervision: alternatives

· Unsupervised learning: we are given “unorganized” data

· our goal is to discover some structure
15 15

10 10

5 5

0 0

5 5

10 10
7.5 5.0 2.5 0.0 2.5 5.0 7.5 10.0 7.5 5.0 2.5 0.0 2.5 5.0 7.5 10.0

· Reinforcement learning: our problem is formalized as a game

· an agent carries out actions and receives rewards
Example: Fisher’s iris data

versicolor
2.4 virginica

2.2

2.0

petal_width
1.8

1.6

1.4

1.2

1.0
3.0 3.5 4.0 4.5 5.0 5.5 6.0 6.5 7.0
petal_length
 Approach 1: linear separator

if 0.85 · petal_length + 2.42 · petal_width ≥ 8.34:

return virginica
else
return versicolor

2.4

2.2

2.0
petal_width

1.8

1.6

1.4

1.2

1.0
3.0 3.5 4.0 4.5 5.0 5.5 6.0 6.5
petal_length
Approach 2: if/then/else tree

2.4

2.2

2.0
petal_width

1.8

1.6

1.4

1.2

1.0
3.0 3.5 4.0 4.5 5.0 5.5 6.0 6.5
petal_length
Basic supervised machine learning workflow
 Basic ML methodology: evaluation

· Select an evaluation procedure (a “metric”) such as

· classification accuracy: proportion correct classifications?
· mean squared error often used in regression
· or some domain-specific metric
· Compare to one or more baselines
· trivial solution
· rule-based solution
· existing solution
· Apply your model to a held-out test set and evaluate
· the test set must be different from the training set
· also: don’t optimize on the test set; use a development set or
cross-validation!
Managing your data
Managing your data
Managing your data
Managing your data for evaluation and cross-validation

[source]
 Overview

Practical issues about the course

Fundamental concepts in machine learning

Machine learning libraries in Python

 Use cases for machine learning

· Standard use cases: standard

solutions are available

· Special cases: we may need to tailor

our own solutions
The Python machine learning ecosystem (selection)
 Machine learning software: a small sample

· General-purpose software, large collections of algorithms:

· scikit-learn: http://scikit-learn.org
▶ Python library – will be used in this course
· Weka: http://www.cs.waikato.ac.nz/ml/weka
▶ Java library with nice user interface
· Special-purpose software, small collections of algorithms:
· Keras, PyTorch, TensorFlow, JAX for neural networks
· LibSVM/LibLinear for support vector machines
· XGboost, lightgbm for tree ensembles
·…
· large-scale learning in distributed architectures:
· Spark MLLib
· H2O
 Scikit-learn toy example

See also
https://scikit-learn.org/stable/getting_started.html
 Up next

· Thursday: lab sessions for programming assignment 1

· Topic of Friday’s discussion:
· decision trees
· ensembles and random forests
· generalization, under/overfitting
· Please prepare for assignment 1 by reading my code and the
extra reading on decision trees