Course Logistics and Introduction

CSN-526
Machine Learning
 2
Course Logistics
▪ Course Name: Machine Learning (CSN-526)
▪ An introductory course – supposed to be your first intro to the subject

▪ Three lectures every week on Microsoft Team

▪ All material will be posted on Microsoft Team for the course

▪ Course-related announcements will be sent on the class mailing list (and also on
Microsoft Teams)

▪ Timings:
Lectures: Monday (02:00 - 02:55), Tuesday (02:00 - 02:55), Thursday (02:00 - 02:55)
Tutorials: Wednesday (04:05-05:00), Wednesday (05:05 - 06:00)
 3
Grading

▪ Mid-term Exam: 25%

▪ End-term Exam: 50%

▪ Course Project: 25%

 4
Course Project: 25%
• Project to be done in groups of five (5) students each
• Start forming your group from today onwards
• Will ask you to submit group details after 1-2 weeks
• Groups can only contain registered students (no auditors)
• You may propose your own project idea
• but please discuss the scope/feasibility with TAs
• Many types of projects possible (theoretical/applied/mixed); even building a
cool/useful app/portal for something using ML as its backend would be a nice
project. Explore around for ideas.
• Important: Don’t (re)use a project from another course you’ve done before (or
doing currently)
• Need to submit a formal project proposal
• Should contain a problem description, tentative plan of action, etc (I’ll provide more guidelines
later)
 5
Course Project: 25%
• Multiple Project presentations to assess the progress of the project
• Project evaluation will be done based on the novelty and new ideas of
the work
• Final Submission will include code + report
• Code should be in Python – start learning Python today
• Report must be in LaTeX – start learning LaTeX today
• Python Resources: several available –choose your favourite
• www.geeksforgeeks.org/python-programming-language/
• LaTeX resources: several available –choose your favourite
• www.overleaf.com/learn/latex/Tutorials
 6
Textbook and References
▪ Many excellent texts but none “required”. Some include:

▪ Different books might vary in terms of

▪ Set of topics covered
▪ Flavor (e.g., classical statistics, deep learning, probabilistic/Bayesian, theory)
▪ Terminology and notation (beware of this especially)
▪ We will provide you the reading material from the relevant sources
▪ Avoid using too many sources until you have developed a reasonable
understanding of a concept
 7
Textbook and References
• Hal Daumé III, A Course in Machine Learning (CIML), 2017 (freely available online)
• Kevin Murphy, Machine Learning: A Probabilistic Perspective (MLAPP), MIT Press,
2012
• Christopher Bishop, Pattern Recognition and Machine Learning (PRML), Springer,
2007.
• David G. Stork, Peter E. Hart, and Richard O. Duda. Pattern Classification (PC), Wiley-
Blackwell, 2000
• Ian Goodfellow and Yoshua Bengio and Aaron Courville. Deep Learning (DL), MIT
Pess, 2016 (individual chapters freely available online)
• Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical
Learning (ESL), Springer, 2009 (freely available online)
• Shai Shalev-Shwartz and Shai Ben-David. Understanding Machine Learning: From
Theory to Algorithms (UML), Cambridge University Press, 2014
• Mehryar Mohri, Afshin Rostamizadeh and Ameet Talwalkar. Foundations of Machine
Learning (FOML), MIT Press, 2012
 8
Textbook and References
▪ Some of the course materials have been taken from the following links:

▪ https://www.cse.iitk.ac.in/users/piyush/courses/ml_autumn18/index.html

▪ https://github.com/purushottamkar/ml19-20w

▪ Thanks to Prof. Piyush for supporting in preparing the course material

 9
Course Goals..
▪ Introduction to the foundations of machine learning models and algos

▪ Focus on developing the ability to

▪ Understand the underlying principles behind ML models and algos
▪ Understand how to implement and evaluate them
▪ Understand/develop intuition on choosing the right ML model/algo for your problem

▪ (Hopefully) inspire you to work on and learn more about ML

▪ Not an introduction to popular software frameworks and libraries, such as

scikit-learn, PyTorch, Tensorflow, etc
▪ Can explore once you have some understanding of various ML techniques
Introduction to Machine Learning
 11
Machine Learning (ML)
▪ Designing algorithms that ingest data and learn a model of the data
▪ The learned model can be used to
▪ Detect patterns/structures/themes/trends etc. in the data
▪ Make predictions about future data and make decisions

▪ Modern ML algorithms are heavily “data-driven”

▪ No need to pre-define and hard-code all the rules (infeasible/impossible anyway)
▪ The rules are not “static”; can adapt as the ML algo ingests more and more data
 12
ML: From What It Does to How It Does It?
▪ ML enables intelligent systems to be data-driven rather than rule-driven
▪ How: By supplying training data and building statistical models of data
▪ Pictorial illustration of an ML model for binary classification:

A Linear Classifier (the statistical model)

 13
ML: From What It Does to How It Does It?
▪ ML enables intelligent systems to be data-driven rather than rule-driven
▪ How: By supplying training data and building statistical models of data
▪ Pictorial illustration of an ML model for binary classification:

A Probabilistic Classifier (the statistical model)

P(“cat”|image) P(“dog”|image)
 14
Overfitting = Bad ML
▪ Doing perfectly on training data is not good enough

▪ A good ML model must generalize well on unseen (test data)

▪ Simpler models should be preferred over more complex ones!

 15
ML Applications Abound..

Picture courtesy: gizmodo.com,rcdronearena.com,www.wiseyak.com,www.charlesdong.com

 16
Key Enablers for Modern ML
▪ Availability of large amounts of data to train ML models

▪ Increased computing power (e.g., GPUs)

 17
ML: Some Success Stories
 18
ML: Some Success Stories
 19
ML: Some Success Stories

Picture courtesy: https://news.microsoft.com/

 20
ML: Some Success Stories
▪ Automatic Program Correction

Example from “Compilation error repair: for the student programs, from the student programs”, Ahmed et al (2018)
 21
Good ML Systems Should be Fair and Unbiased
▪ Good ML should not just be about getting high accuracies

▪ Should also ensure that the ML models are fair and unbiased
Criminals?

Not Criminals?

An image captioning Don’t want a self-driving car Don’t want a predictive policing
system should not always that is more likely to hit black system that predicts criminality
assume a specific gender in people than white people using facial features
examples like the above

▪ A lot of recent focus on Fairness and Transparency of ML systems

Picture courtesy: Bhargava and Forsyth (2019), https://www.thestranger.com/, Xiaolin Wu and Xi Zhang, “Automated Inference on Criminality Using Face Images”
 22
Machine Learning Tools
▪ Scikit-Learn
▪ Pytorch
▪ Google Colab
▪ Jupyter Notebook
▪ Tensorflow
▪ Google Cloud AutoML
▪ Numpy
▪ MatPlotLib
▪ Open CV
▪ Microsoft Azure Machine Learning
▪ Amazon Machine Learning
 23
Next Class
▪ Various Flavors of ML problems

▪ Data and features

▪ Basic mathematical operations on data and features