Assignment 3 Introduction to Machine Learning (WBAI056-2024.

2025)

Introduction in Machine Learning - WBAI056-2024.2025

Assignment

Total points: 80
Starting date: 3 December 2024
Submission deadline: 23:59, 21 January 2025

Lecturers Dr. Matias Valdenegro, Dr. Andreea Sburlea.

• The tasks are targeted at groups of three students. Please make sure that the load is well divided:
every student should contribute.

• Please take advantage of the practical sessions to ask your questions about the tasks.

• Provide a (short but comprehensive) explanation of what you are doing for each task.

• A reviewer should be able to understand plots independently; be sure to label axes, a legend for
colors, use an easily readable font size, etc.

• Refer to all plots, tables, code blocks, etc. in your report.

• For the report: you can use a jupyter notebook or write a PDF in a word processor of your
preference. Please include code as .py or .ipynb files as attachments in Brightspace.

• In your report, please include a statement from all group members, indicating what was their
contribution to this assignment.

Homework Assignment
Maximum obtainable points: 80

Learning Objective: The aim of this assignment is for students to learn usage of machine learning
frameworks and to code a small project where they train machine learning models for a given task.

Step 1. Each group selects a topic for their projects and writes a short proposal (max 1 page) about
what they would like to do. Consider the following information:

• General idea/motivation, Dataset, task, and model(s) you will use. Set a simple goal.

• Short description of what you will do (train models, use X evaluation metrics, etc)

Some general project ideas are available below:

• Train a model on a dataset from scratch and provide a correct evaluation.

• Fine-tune a model on a dataset and provide a correct evaluation.

• Implement part of a research paper that you find interesting.

• Perform an evaluation task from the following: estimate training set size vs performance curves
(sample complexity), train a model and evaluate performance on multiple test sets, train and
evaluate a model using noisy labels (you add noise to labels).

• Reproduce a machine learning concept on a dataset: bias-variance trade-off, double descent, or

1
Assignment 3 Introduction to Machine Learning (WBAI056-2024.2025)

You can also propose ideas of your own that are outside what is said above, but close enough, you can
ask via email.

Your project must be a coding one and you should train machine learning models using python DL
framework of your choice (tensorflow, keras, pytorch, etc).

Consider that the project should take around 25-30 hours of your time (10 hrs per week for around
2-3 weeks). Submit the proposal as a PDF file in Brightspace, deadline is latest December 11. You
will get feedback and accept/reject within a couple of days. If you have doubts about your project
being accepted before submitting, you can ask the coordinator via email.

Rejection might come because the project is too difficult or might take you much more than the allo-
cated time, and then changes can be made to adapt requirements.

Step 2. Once your project topic is accepted, start working on your project on your time and during
the practicals. Attend practicals to ask questions to the TA or via email to the coordinator.

Step 3. By the project’s end (current deadline is January 21, 2025, flexible deadline until January
31), you should submit:
1. A written report (3-10 pages), indicating what you did in your project, results, analysis, to
what degree you fulfilled your goal, and include source code as attachments. Any source code
you reused from past projects or from the internet can be used but must be properly cited and
referenced. It should be clear what is new for this project and what is reused from the internet.
Make sure that your report is readable and understandable, and that it contains all necessary de-
tails for someone else to reproduce your results. Any use of material should be clearly cited and
quoted/referenced, including any paper(s) that you might use as basis.

Evaluation Rubric
What we will evaluate and in what weight is described below:
30% Project quality and consistency with the learning outcome. Did the group train a model on a
task of significant difficulty (not a toy dataset like MNIST)? Did the group succeed to a certain
degree with respect to their own goals? Failure is not unexpected and generally acceptable, if
students tried enough (model variations, different datasets, preprocessing, etc) this also counts
as success. Failure to achieve the task without explanation or further reasons will decrease the
grade significantly. The project should be approx 40 hours of work. Significantly less than this
standard will decrease the grade.
40% Model training. Are the models correctly trained? Is there a validation/test set? Is overfitting
correctly detected/absent? Note that a model overfitting is not by itself a bad outcome, but this
should be detected properly by the group.
30% Report structure and content, citations, figure/table referencing, proper reuse of previous work.
Overall grade for the format, structure, and content of the report, does it follow like a scientific
paper? Are the steps performed by students well described? Is the project reproducible to a
certain degree? (We cannot expect full reproducibility, but only to a good amount). There is
no need to describe the general methods that were used if they are covered in the lectures, only
what is specific to the project.
The report should be understandable and clearly written/structured, we will not be checking grammar
or typos, but overall description of the project. If parts are not understandable, points can be deducted
or marked as incorrect.