IE 7374 Machine Learning in Engineering

Course This course covers practical algorithms and the theory for machine learning from a
Overview variety of perspectives. Topics include supervised learning(generative/discriminative
learning, parametric/non-parametric learning, deep neural networks, support vector
machines), unsupervised learning(clustering, dimensionality reduction, kernel
methods). The course will also discuss recent applications of machine learning, such
as computer vision, data mining, natural language processing, speech recognition and
robotics. Students will learn the implementation of selected machine learning
algorithms via python and Tensorflow.

Prerequisites Introduction to Probability and Statistics, Linear Algebra. Comfortable with python
programming language.

Objectives To learn theory and application of machine learning techniques.

To learn how evaluate the performance of machine learning models.
To implement machine learning models using python programming and Tensorflow.
To design and validate a machine learning problem.

Textbooks 1. [CB] Christopher Bishop, Pattern recognition and machine learning. [Required]
2. [KM] Kevin P. Murphy, Machine Learning: A Probabilistic Perspective.
[Required]

Syllabus Supervised Learning

• Linear regression, overfitting, regularization, sparsity
• Bias-variance tradeoff
• Logistic regression
• Naive Bayes
• Perceptron
• Neural networks and deep learning: DNNs, RNNs, CNNs
• SVM and kernels
Unsupervised Learning
• Clustering: k-means, mixture models, expectation maximization, k-
medoids, spectral clustering
• Dimensionality reduction: PCA, Kernel PCA
• Matrix and tensor factorization
• Topic modeling
• Model selection

Homeworks Homework 1 - 5

Project Project - Posting Date Feb 15th Submission Date April 15th
Schedule

Spring 2021 Page 1 of 5

Course HomeWorks - 50%
Evaluation Exams - 25%
Final Project - 20%

Software Python and Tensorflow

Install Python and Jupyter notebook for Mac/Windows: Follow the instructions in
the link below to install Python and Jupyter notebook for Mac/Windows. All the steps
remain same as shown in the video except the Python and Jupyter notebook version.

Mac version: https://www.youtube.com/watch?v=daVgEXjv6DE

Windows version: https://www.youtube.com/watch?v=5mDYijMfSzs

Study Plan Lecture 1: Introduction to ML

Lecture 2: Probability Review. (CB chapter 1 and 2) and (KM chapter 1 and 2)
Lecture 3: Linear Algebra Review
Lecture 4: Introduction to Regression, Convex Functions and Optimality,
Lecture 5: Linear Regression: Closed-form Solution, Gradient Descent and SGD,
Basis Function Expansion (CB chapter 3) and (KM chapter 7)
Lecture 6: Robust Regression, Overfitting, Regularization,
Lecture 7: Hyper-parameter Tuning, Cross Validation (CB chapter 3) and (KM
chapter 7)
Lecture 8: Python class
Lecture 9: Point Estimation, Maximum Likelihood Estimation, MAP Estimation
Bayesian Learning, Generative Modeling for Classification (CB chapter 2 and 3)
and (KM Chapter 3).
Lecture 10: Generative Modeling for Classification: Naive Bayes, Gaussian
Discriminant Analysis. (CB chapter 4.2) and (KM Chapter 2 and 3).
Lecture 11: Discriminative Modeling for Classification: Logistic Regression,
SoftMax Regression. (CB chapter 4. 3) and (KM Chapter 8).
Lecture 12: Python class
Lecture 13: Midterm exam
Lecture 14: Convex Optimization, Lagrangian Function, KKT Conditions. Support
Vector Machines: Vanilla SVM, Dual SVM. (CB chapter 7) and (KM Chapter 14).
Lecture 15: Support Vector Machines (SVM), Max-Margin Classification. Kernels,
Kernel SVM, Soft-Margin SVM, SMO Algorithm, Multi-Class SVM. (CB chapter 7)
and (KM Chapter 14).
Lecture 16: Support Vector Machines (SVM) (continuation)
Lecture 17: Support Vector Machines (SVM) (continuation)
Lecture 18: Python class
Lecture 19: Dimensionality reduction – PCA, SVD
Lecture 20: Centroid Clustering via KMeans, Subspace Clustering via Subspaces.
Similarity Graphs, Graph Laplacian and its Properties, Spectral Clustering. (KM
Chapter 25)
Lecture 21: Centroid Clustering via KMeans, Subspace Clustering via Subspaces.
Similarity Graphs, Graph Laplacian and its Properties, Spectral Clustering. (KM
Chapter 25) (continuation)
Lecture 22: Neural Networks, Architectures, Activations, Outputs, Forward
Propagation. Feed Forward NNs, Forward and Backward Propagation, Training via
Backpropagation Algorithm (CB chapter 5) and (KM Chapter ).
Lecture 23: Neural Networks, Architectures, Activations, Outputs, Forward
Propagation. Feed Forward NNs, Forward and Backward Propagation, Training via
Backpropagation Algorithm (CB chapter 5) and (KM Chapter ). (continuation)

Spring 2021 Page 2 of 5

 Lecture 24: Convolutional Neural Networks, Autoencoders Architectures, Training,
Examples.
Lecture 25: Centroid Clustering via KMeans, Subspace Clustering via Subspaces.
Similarity Graphs, Graph Laplacian and its Properties, Spectral Clustering. (KM
Chapter 25)
Lecture 26: Python class
Lecture 27: Python class
Lecture 28: Project presentation
Lecture 29: Final Exam

Grading

Evaluation 1. Unless otherwise specified, all homeworks and assignments are due for
submission at 11:59 pm on the day they are due. Late submission of homework
Policy
will receive penalty. For two-day delay, grade will be cut by 10%. Beyond two
days past the deadline, the submission will not be accepted.
2. Discussion between students for homeworks and projects is not permissible.
Each submission must be an outcome of individual effort.

Academic A commitment to the principles of academic integrity is essential to the mission of

Northeastern University. The promotion of independent and original scholarship
Integrity ensures that students derive the most from their educational experience and their
pursuit of knowledge. Academic dishonesty violates the most fundamental values of
an intellectual community and undermines the achievements of the entire
University.

As members of the academic community, students must become familiar with their
rights and responsibilities. In each course, they are responsible for knowing the
requirements and restrictions regarding research and writing, examinations of
whatever kind, collaborative work, the use of study aids, the appropriateness of
assistance, and other issues. Students are responsible for learning the conventions
of documentation and acknowledgment of sources in their fields. Northeastern

Spring 2021 Page 3 of 5

 University expects students to complete all examinations, tests, papers, creative
projects, and assignments of any kind according to the highest ethical standards,
as set forth either explicitly or implicitly in this Code or by the direction of
instructors. Go to http://www.northeastern.edu/osccr/academic-integrity-policy/ to
access the full academic integrity policy.
Recording of Classes will be recorded to enable all students to review material covered in
Classes synchronous classes. Please contact me if you have any concerns.

Student Northeastern University and the Disability Resource Center (DRC) are committed to
Accommodations providing disability services that enable students who qualify under Section 504 of
the Rehabilitation Act and the Americans with Disabilities Act Amendments Act
(ADAAA) to participate fully in the activities of the university. To receive
accommodations through the DRC, students must provide appropriate
documentation that demonstrates a current substantially limiting disability.

For more information, visit http://www.northeastern.edu/drc/getting-started-with-the-

drc/.
Diversity and Northeastern University is committed to equal opportunity, affirmative action,
Inclusion diversity and social justice while building a climate of inclusion on and beyond
campus. In the classroom, members of the University community work to cultivate
an inclusive environment that denounces discrimination through innovation,
collaboration and an awareness of global perspectives on social justice. It is my
intention that students from all backgrounds and perspectives will be well served by
this course, and that the diversity that students bring to this class will be viewed as
an asset. I welcome individuals of all ages, backgrounds, beliefs, ethnicities,
genders, gender identities, gender expressions, national origins, religious
affiliations, sexual orientations, socioeconomic background, family education level,
ability – and other visible and nonvisible differences. All members of this class are
expected to contribute to a respectful, welcoming and inclusive environment for
every other member of the class. Your suggestions are encouraged and
appreciated.

Please visit http://www.northeastern.edu/oidi/ for complete information on Diversity

and Inclusion.
TITLE IX Title IX of the Education Amendments of 1972 protects individuals from sex or
gender-based discrimination, including discrimination based on gender-identity, in
educational programs and activities that receive federal financial assistance.

Northeastern’ s Title IX Policy prohibits Prohibited Offenses, which are defined as

sexual harassment, sexual assault, relationship or domestic violence, and stalking.
The Title IX Policy applies to the entire community, including male, female,
transgender students, faculty and staff.

In case of an emergency, please call 911.

Please visit www.northeastern.edu/titleix for a complete list of reporting options and

resources both on- and off-campus
Contact Instructor Email: [email protected]
TA Contact: Michelle Rebello ([email protected])
TA office hours: Wednesdays 3-5 pm (Via zoom) and Fridays 11am-1 pm (in
person)

Spring 2021 Page 4 of 5

 TA office location: TBD

1. All homework and projects will be delivered by the instructor via Canvas LMS
2. Students need to submit homework and projects via Canvas
3. Submission formats will be outlined for each homework and project.
4. Students can access Canvas via https://my.northeastern.edu/ website
5. There are good resources in https://my.northeastern.edu/ to help you get used
to Canvas tool
6. Lectures in form of videos will be uploaded twice a week
7. All videos, lecture materials, reading materials, and codes will be uploaded
twice a week
8. For personal (private) communication it is best to ask the questions via
Microsoft Teams. Type the name “Ramin Mohammadi” in the search bar of
Teams. Then click on the profile and send your questions.
9. Students can also email the instructor [email protected]
10. Students can setup meeting with TAs according to their availability
11. Students attending the online class from abroad need to notify in advance if
some tools are restricted in their home country. This will enable the instructor to
create alternate mechanisms to deliver the class content

Spring 2021 Page 5 of 5