Computer Science EN.601.

482/682
Machine Learning: Deep Learning
Spring, 2023 (4 credits, E)

Instructor
Mathias Unberath, [email protected], https://arcade.cs.jhu.edu/
Office hours: Thursdays 3.00 – 4.00 pm
Office hour location: Malone Hall 327

Teaching Assistants
Yiqing Shen, [email protected]
Office hours: Tuesdays 3.00 – 4.00pm
Office hour location: Malone Hall 216

Jingfeng Wu, [email protected]

Office hours: Friday 5.30 – 6.30pm
Office hour location: Malone Hall 107 (on April 14th, Malone 338)

Meetings
Meeting location: Remsen Hall 1

Mondays from 4.30pm to 5.45pm

Wednesdays from 4.30pm to 5.45pm
Fridays from 4.30pm to 5.20pm

Zoom connection: Zoom meeting link

Panopto recordings: Link to Panopto

Attendance in person is strongly recommended to achieve the best learning outcomes.

The zoom virtual presence option is provided exclusively for increased accessibility, and
in extraordinary circumstances (such as acute illness). Lecture recordings are not an
adequate substitution for in person attendance. Recordings are made available to
facilitate recollection and working through the materials after class.
Specifically, if you are feeling sick, or have reason to believe you may be sick, please do
not come to class.

Mask Policy
Deep Learning is a large class and ventilation of and air quality in auditoriums is limited;
prosperous conditions for the spread of airborne diseases such as COVID. While JHU’s
current masking policy does not allow class-specific mask mandates, I strongly
encourage everyone in this class to please wear masks to minimize exposure of students
and instructors. JHU provides these masks for free (see here for locations:
https://covidinfo.jhu.edu/diagnostic-testing/testing-locations-and-schedules/#distribution)
. Your serious consideration of this request is highly appreciated.

Textbook
This course does not explicitly follow a specific textbook, though many good textbooks
on the topic are available. A fairly exhaustive list of additional reading material
(including textbooks, blog articles, tutorials, and scientific articles) will be made
available in the “Resource” section of Piazza.

Online Resources
Please log in to Piazza for all materials related to this course.

Course Information
Synopsis: Deep learning (DL) has emerged as a powerful tool for solving data-intensive
learning problems such as supervised learning for classification or regression,
dimensionality reduction, and control. As such, it has a broad range of applications
including language processing, computer vision, medical imaging, and perception-based
robotics. The goal of this course is to introduce the basic concepts of DL. The course will
include a brief introduction to the basic theoretical and methodological underpinnings of
machine learning, commonly used architectures for DL, current challenges including
ethics and fairness, and specialized applications with a particular focus on computer
vision. Students will be expected to solve several DL problems on standardized data sets
and will be given the opportunity to pursue team projects on topics of their own interest.

Prerequisites: (AS.110.201 or AS.110.212 or EN.553.291) and (EN.553.310 EN.553.311

or EN.553.420 or EN.560.348) and (EN.601.475 or equivalent); Calc III and numerical
optimization recommended. Recommended co-req: EN.601.382.
Communication Plan: In Spring 2023, this course will be held in person. Virtual presence
through zoom is possible in case of need (accommodations, illness, higher power). No
prior approval is required to attend virtually. Sessions will be recorded to facilitate
recollection and in preparation for assignments and exams. Watching the recordings are
not an acceptable substitute for in person attendance.

Where do I find course material? The course’s home is on Piazza and you should sign up
as soon as possible using this link: piazza.com/jhu/spring2023/cs482682. On Piazza you
will find lecture slides, assignments, and other relevant course material and resources.
You are encouraged to post any questions and discussions on Piazza and contribute to
answering questions your peers have posted. Please note that, while you can remain
anonymous to peers, posting anonymously to instructors is disabled.

What about homework assignments? We will offer recitations as well as homework

orientation and Q/A sessions during the Friday slot. Attending these sessions is highly
recommended. Questions about homework assignments should be posted to Piazza
(again, ideally as public post so that everyone can benefit from our answers) and can also
be discussed in greater detail during office hours. Homework assignments are submitted
through Gradescope with course code DJGEWX.

Any other communication platforms? Yes – We will use Google Forms for quizzes.
Links to Quizzes will be shared during synchronous sessions.
Course Goals
Specific Outcomes for this course are that
● Students will learn fundamental concepts of machine learning
● Students will learn the theoretical underpinnings of deep learning
● Students will learn contemporary architectures, applications, and challenges of
deep learning
● Students will learn to design, implement, and validate deep learning-based
solutions to machine learning problems
This course will address the following CSAB ABET Criterion 3 Student Outcomes
Graduates of the program will have an ability to:
1. Design, implement, and evaluate a computing-based solution to meet a given set
of computing requirements in the context of the program’s discipline.
2. Recognize professional responsibilities and make informed judgments in
computing practice based on legal and ethical principles.
3. Function effectively as a member or leader of a team engaged in activities
appropriate to the program’s discipline.
4. Apply computer science theory and software development fundamentals to
produce computing-based solutions.
Course Topics
Homework assignments are released / are due Wednesdays (latest submission 11.59pm).

Week 1 (Jan 23): Introduction and Basics

L-1) Welcome, Course Logistics and Topic Overview
L-2) Basics Part I: Image Features, Regression, and Classification
Recitation: Homework 1 review
Wednesday, Jan 25: Homework 1 released

Week 2 (Jan 30): Basics

L-3) Basics Part II: Regularization and Optimization
L-4) Computational Graphs and Backpropagation Part I
Voluntary review: Intro to Python video
Recitation: Weeks 1/2 and homework 2
Wednesday, Feb 1: Homework 1 due, Homework 2 released

Week 3 (Feb 06): Convolutional Neural Networks

L-5) History of and Introduction to Neural Networks
L-6) Convolutional Neural Networks
Recitation: Week 3 and homework 2
Week 4 (Feb 13): Training Neural Networks
L-7) Training Part I: Activation, Initialization, Preproc., Dropout, Batch norm
L-8) Training Part II: Updates & Momentum, Augmentation, Transfer Learning
Recitation: Week 4 and homework 3
Wednesday, Feb 15: Homework 2 due, Homework 3 released

Week 5 (Feb 20): Architectures

L-9) Inverse Classroom: It’s not working! Help!
L-10) Network Architectures: AlexNet, VGG, ResNet, U-Net, …
Recitation: Week 5 and homework 4
Wednesday, Feb 23: Homework 3 due, Homework 4 released

Week 6 (Feb 27): Architectures continued – Form project groups

L-11) Inverse Classroom: What does this network do? Part I
L-12) RNNs and LSTM
Recitation: Week 6 and homework 4

Week 7 (Mar 06): Unsupervised Learning and Generative Models

L-13) Unsupervised and Self-supervised Learning
L-14) Autoencoders, Variational Autoencoders, and Disentanglement
Recitation: Week 7 and homework 5
Wednesday, Mar 08: Homework 4 due, Homework 5 released

Week 8 (Mar 13): Unsupervised Learning and Generative Models

L-15) Generative Adversarial Networks
L-16) Inverse Classroom: Labeling? Ain’t nobody got time for that.
Recitation: Week 8 and homework 6
Wednesday, Mar 16: Homework 5 due, Homework 6 released

Week 9 (Mar 20): Spring Break

The week of Monday Mar 20 is designated as spring break

Week 10 (Mar 27): Advanced Topics – Start project work

L-17) Generalization, domain gaps, and explainable AI
L-18) Domain gaps and black boxes
Recitation: Week 10 and homework 6

Week 11 (Apr 03): Advanced Topics

L-19) Character and word embedding
L-20) Transformers for Language and Vision
Recitation: Week 11 and homework 6
Wednesday, Apr 06: Project proposals due, Homework 6 due, Bonus assignment
released
Week 12 (Apr 10): Advanced Topics
L-21) Neural Radiance Fields
L-22) Diffusion Models
Recitation: Midterm Q/A

Week 13 (Apr 17): Midterm and Healthcare Applications

L-23) Midterm Exam
L-24) DL4Health
Recitation: Week 13

Week 14 (Apr 24): Wrap up

L-25) Human-centered AI, ethics, etc.
L-26) Wrap Up
Wednesday, Apr 26: Bonus assignment due

Wednesday, TBD: Final project reports due

TBD (final exam slot): Synchronous project pitch and breakout rooms

Course Expectations & Grading

We will release short weekly quizzes for you to test your comprehension and recollection
of the course material. These quizzes are not graded and solely serve the purpose for you
to evaluate your comprehension. The link to participate in quizzes will be shared during
the respective session.
The course has 6+1 assignments: 3 are written and the remaining 3+1 focus on
programming. The bulk of the homework assignment workload (1 to 6) is condensed into
the first three quarters of the semester to free up time for the final project. Starting in
Week 11, most time will be spent on the final project that will be completed in groups of
four. Individual grades will be computed as a weighted combination of these factors:
1) Homework Assignments: 50%
2) Midterm Exam: 25%
3) Final Project: 25%
There is an opportunity for bonus points. Bonus points are earned by completing
additional assignments that will be released as bonus assignment 7.

For late assignment submissions, you have a total of 5 late days that you can use at your
discretion. However, no smaller quantity than “one day” can be used but you can use
multiple days for the same homework. You do not need to request late days but will have
to keep track of your remaining quota yourself. Once you run out of late days but submit
late anyway, the submission will not award any points.

Assignments & Readings

See above schedule for assigned lectures and materials in every week.
Ethics
The strength of the university depends on academic and personal integrity. In this course,
you must be honest and truthful, abiding by the Computer Science Academic Integrity
Policy:
Cheating is wrong. Cheating hurts our community by undermining academic
integrity, creating mistrust, and fostering unfair competition. The university will
punish cheaters with failure on an assignment, failure in a course, permanent
transcript notation, suspension, and/or expulsion. Offenses may be reported to
medical, law or other professional or graduate schools when a cheater applies.
Violations can include cheating on exams, plagiarism, reuse of assignments
without permission, improper use of the Internet and electronic devices,
unauthorized collaboration, alteration of graded assignments, forgery and
falsification, lying, facilitating academic dishonesty, and unfair competition.
Ignorance of these rules is not an excuse.
Academic honesty is required in all work you submit to be graded. Except where
the instructor specifies group work, you must solve all homework and
programming assignments without the help of others. For example, you must not
look at anyone else’s solutions (including program code) to your homework
problems. However, you may discuss assignment specifications (not solutions)
with others to be sure you understand what is required by the assignment.
To clarify, any work submitted to this course must be the student’s own, original
work. We understand that “another individual” in the Computer Science
Academic Integrity Policy and Homewood Undergraduate Ethics Policy includes
artificial intelligence (AI) assistants such as ChatGPT and GitHub Copilot.
Therefore, use of such AI assistants is considered a form of unauthorized
collaboration and is not allowed in this course. To ensure the integrity of the
academic work submitted, we will not accept any assignments that have been
written with the assistance of AI.
If your instructor permits using fragments of source code from outside sources,
such as your textbook or on-line resources, you must properly cite the source. Not
citing it constitutes plagiarism. Similarly, your group projects must list everyone
who participated.
Falsifying program output or results is prohibited.
Your instructor is free to override parts of this policy for particular assignments.
To protect yourself: (1) Ask the instructor if you are not sure what is permissible.
(2) Seek help from the instructor, TA or CAs, as you are always encouraged to do,
rather than from other students. (3) Cite any questionable sources of help you may
have received.
On every exam, you will sign the following pledge: "I agree to complete this
exam without unauthorized assistance from any person, materials or device.
[Signed and dated]". Your course instructors will let you know where to find
copies of old exams, if they are available.
Report any violations you witness to the instructor.

You can find more information about university misconduct policies at these sites:
● For undergraduates:
http://e-catalog.jhu.edu/undergrad-students/student-life-policies/
● For graduate students:
http://e-catalog.jhu.edu/grad-students/graduate-specific-policies/

Personal Wellbeing
● If you are sick please notify me by email so that we can make appropriate
accommodations should this affect your ability to attend class, complete
assignments, or participate in assessments. The Student Health and Wellness
Center is open and operational for primary care needs. If you would like to
speak with a medical provider, please call 410-516-8270, and staff will
determine an appropriate course of action based on your geographic
location, presenting symptoms, and insurance needs. Telemedicine visits are
available only to people currently in Maryland. See also
https://studentaffairs.jhu.edu/student-life/student-outreach-support/absen
ces-from-class/illness-note-policy/
● The Johns Hopkins COVID-19 Call Center (JHCCC), which can be reached at
833-546-7546 seven days a week from 7 a.m. to 7 p.m., supports all JHU
students, faculty, and staff experiencing COVID-19
symptoms. Primarily intended for those currently within driving distance of
Baltimore, the JHCCC will evaluate your symptoms, order testing if needed,
and conduct contact investigation for those affiliates who test positive. More
information on the JHCCC and testing is on the coronavirus
information website.  
● All students with disabilities who require accommodations for this course
should contact me at their earliest convenience to discuss their specific
needs. If you have a documented disability, you must be registered with the
JHU Office for Student Disability Services (385 Garland Hall; 410-516-4720;
http://web.jhu.edu/disabilities/) to receive accommodations.
● Students who are struggling with anxiety, stress, depression or other mental
health related concerns, please consider connecting with resources through
the JHU Counseling Center. The Counseling Center will be providing services
remotely to protect the health of students, staff, and communities. Please
reach out to get connected and learn about service options based on where
you are living this fall at 410-516-8278 and online at
http://studentaffairs.jhu.edu/counselingcenter/.
● Student Outreach & Support will be fully operational (virtually) to help
support students. Students can self-refer or refer a friend who may need
extra support or help getting connected to resources. To connect with SOS,
please email [email protected], call 410-516-7857, or students can
 schedule to meet with a Case Manager by visiting the Student Outreach &
Support website and follow “Schedule an Appointment”. 

Classroom Climate
As your instructor, I am committed to creating a classroom environment that values the
diversity of experiences and perspectives that all students bring. Everyone here has the
right to be treated with dignity and respect. I believe fostering an inclusive climate is
important because research and my experience show that students who interact with peers
who are different from themselves learn new things and experience tangible educational
outcomes. Please join me in creating a welcoming and vibrant classroom climate. Note
that you should expect to be challenged intellectually by me, the TAs, and your peers, and
at times this may feel uncomfortable. Indeed, it can be helpful to be pushed sometimes in
order to learn and grow. But at no time in this learning process should someone be
singled out or treated unequally on the basis of any seen or unseen part of their identity.

If you ever have concerns in this course about harassment, discrimination, or any unequal
treatment, or if you seek accommodations or resources, I invite you to share directly with
me or the TAs. I promise that we will take your communication seriously and to seek
mutually acceptable resolutions and accommodations. Reporting will never impact your
course grade. You may also share concerns with the Department Head (Randal Burns,
[email protected]), the Director of Undergraduate Studies (Joanne Selinski,
[email protected]), the Assistant Dean for Diversity and Inclusion (Darlene
Saporu, [email protected]), or the Office of Institutional Equity ([email protected]). In
handling reports, people will protect your privacy as much as possible, but faculty and
staff are required to officially report information for some cases (e.g. sexual harassment).

Family Accommodations Policy

You are welcome to bring a family member to class on occasional days when your
responsibilities require it (for example, if emergency childcare is unavailable, or for
health needs of a relative).  In fact, you may see my children in class on days when no
other accommodations are possible.  Please be sensitive to the classroom environment,
and if your family member becomes uncomfortably disruptive, you may leave the
classroom and return as needed. 

University Policy on Incompletes

Teaching staff recognizes that some students may find themselves in unexpected
situations where study is difficult if not impossible. Students who are confronted with
extraordinary circumstances that interfere with their ability perform their academic work
may request an incomplete grade from the instructor. While approval of such a request is
not automatic, we will make every effort to accommodate students dealing with illness in
the family and other significant hardships. 

Deadlines for Adding, Dropping and Withdrawing from Courses

Students may add a course up to February 3, 2023. They may drop courses up to March
5, 2023 provided they remain registered for a minimum of 12 credits. Between March 5
and April 14, 2021, a student may withdraw from a course with a W on their academic
record. A record of the course will remain on the academic record with a W appearing in
the grade column to indicate that the student registered and then withdrew from the
course.
For more information on these and other academic policies, see
https://e-catalogue.jhu.edu/engineering/full-time-residential-programs/undergraduate-poli
cies/academic-policies/grading-policies/

The Office of Academic Support at JHU

All programs are free to students, please see below for specifics:
● PILOT Learning – Peer-Led Team Learning
o Students are organized into small study teams who meet weekly to
collaborate on faculty-developed problems-sets.  Students work together
as a team to solve problems.
o A trained student leader acts as captain and facilitates the weekly meetings
using various strategies to foster a collaborative learning environment.
o Registration opens on August 31st at 9pm EST; registration will remain
open throughout the semester if space allows.
o Contact: Ariane Kelly - [email protected]
o Instagram: @jhupilot
● Learning Den Tutoring Program - Small Group Tutoring 
o Small group, tailored tutoring of 4 students or less which is headed by one
tutor. Visit the website (above) to access zoom links for drop-in sessions
o Tutors can assist with but are not limited to:
▪ Review and strengthening of subject-specific material knowledge
▪ Assist with homework-like problems
▪ Course-specific study skills and exam preparation
▪ Contact:  Kaitlin Quigley – [email protected]
▪ Instagram: @jhulearningden
● The Study Consulting Program
o Students work one-on-one with a study consultant to set academic goals
and develop customized strategies for success. Areas addressed include
but are not limited to:
▪ Time management
▪ Note taking and test preparation
▪ Mastering large amounts of information
o Contact: Dr. Sharleen Argamaso – [email protected]
o Instagram: @jhustudyconsulting
● The Writing Center
o Undergraduate and graduate students in KSAS/Whiting School of
Engineering can schedule 50-min sessions with a Writing Center tutor to
look over a draft of written work (up to 10 pages) or a personal statement
for graduate study
o Contact: Robert Tinkle – [email protected]
o Web Address: https://krieger.jhu.edu/writingcenter/