Numpy Tutorial

K. Breininger, V. Christlein, Z. Yang, L. Rist, M. Nau, S. Jaganathan, C. Liu, N. Maul, L. Folle, M. Zinnen,
K. Packhäuser
Pattern Recognition Lab, Friedrich-Alexander University of Erlangen-Nürnberg
October 20, 2023
 Who are we? - Lab Members

Andreas Zijin Alexander Chang Leonhard

Maier Yang Barnhill Liu Rist

Merlin Noah Mathias Srikrishna Kai

Nau Maul Zinnen Jaganathan Packhäuser

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 1

 Who are we? - Student Members

Lisa Majid Chengze Teena Leyi

Schmidt Sharghi Ye Tom Dieck Tang

Supraja Karlo Gabriel Jingyi Anna-Sophie Philip

Ramesh Fonseca Yao Stephan Wagner
Yakovenko

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 2

Organisation
 Contact us ...

• in StudOn
• via the tutors mailing list: [email protected]
• in MS Teams
Important: Don’t hesitate to ask questions/give comments!

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 3

 Online teaching

• Exercise will be completely online

• Follow the Guide in StudOn
• You can also find all information under
Organisational_concepts in StudOn
homepage
• We will use MS Teams
• Team activation in IDM required!
• "General" channel for general questions
and comments
• "Private" channel for each exercise day
• Direct support during exercise hours can
be requested in resp. channel

Important: Feedback and suggestions are most welcome!

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 4

 Semester plan

• Five exercises:
0. Python + Numpy Recap and Data Generation
1. Fully Connected Networks
2. CNNs and Optimization
3. Regularization and Recurrent Neural Networks
4. Image Classification with PyTorch
• Platform: MS Teams

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 5

 Semester plan

• Five exercises:
0. Python + Numpy Recap and Data Generation
1. Fully Connected Networks
2. CNNs and Optimization
3. Regularization and Recurrent Neural Networks
4. Image Classification with PyTorch
• Platform: MS Teams
• Materials available in StudOn
• Each exercise takes 2-4 weeks → start early, submit early
• Bonus points for exam up to 10% (6 points). Unittests coverage determines
your bonus points.
• Written exam (mock exam available in StudOn)

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 5

 Submission

• Group submission possible - pairs of two

→ “Finding Group Partners" channel in MS Teams
• Personal submission only (max group of 2 people)
• Required to get the bonus points
• We prioritize the requests, have a look into our online course guide at StudOn
→ Organisational_concepts/03 – MS Teams
• Mind the deadlines !!!
→ please use the provided script (dispatch.py ) to prepare your upload
• Upload your code to StudOn
• Explain your code (next week after uploading) (screen sharing)

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 6

 No Plagiarism!

• Plagiarism is strictly forbidden

• We will check that with plagiarism software!

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 7

Exercise Setup
 First part:

Build a neural network from scratch using test based development

• Implementation task is defined by description and unit tests
• No skeletons
• Every function and structure is built as a layer
→ As a class in its own file
→ Mandatory functions __init__(), forward(), backward()
• Unit tests help to expose bugs and errors
→ Tested and debugged with python3

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 8

 Second part:

Build some common neural networks with PyTorch

• Some functionality provided
• No exhaustive unit tests

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 9

 Schedule

Date Event
23.10.2023 Handout all exercises
10.11.2023 Deadline Ex.0
24.11.2023 Deadline Ex.1
15.12.2023 Deadline Ex.2
19.01.2024 Deadline Ex.3
09.02.2024 Deadline Ex.4

A detailed time table could be found in StudOn with lecture suggestions

→ Organisational_concepts/05 – Time table
• These deadlines are for code uploading. You have until the end of next week
of each deadline to present your code

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 10

 Bonus Points System

• Exercises contribute to max. 10% of bonus for the final exam

• Unittest coverage corresponds to bonus points
→ Each exercise consists of several TestCases
→ TestCases subdivide the bonus points
→ Each TestCase consist of several unittests
→ All unittests of one TestCase must pass to get the respective bonus
• The percentages correlate to effort and difficulty → but also the small
bonuses add up
• The unittest files can compute the points for you → have a look in the
description
• Be aware, some TestCases depend on others
→ It’s impossible to test a neural network without having its layers first
• You only get the bonus if you submit in time! Mind the deadlines!

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 11

 Bonus Points - Ex0

• Ex0 (1% in exam = 0.6 point from 60 points)

• (10%) TestCircle
• (10%) TestSpectrum
• (10%) TestChecker
• (70%) TestGen

• Percentage next to Ex0 represents the bonus points in the exam (here 1%)
• E.g., TestCircle contributes with 10% to the 1% (resulting in 0.1% for the
exam)
• TestGen seems important with 70% and hence causes the most effort
• TestGen contains also more unittests than the others.

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 12

 Bonus Points Distribution

• Ex1 (1.5% in exam = 0.9 point from 60 points)

• (45%) TestFullyConnected1
• ( 5%) TestReLU
• (10%) TestSoftMax
• (10%) TestCrossEntropy
• ( 5%) TestOptimizers1
• (25%) TestNeuralNetwork1 [1]
• Ex2 (3% in exam = 1.8 point from 60 points)
• (45%) TestConv [2,3,4]
• (15%) TestPooling [3]
• ( 2%) TestFlatten
• ( 5%) TestInitialization
• ( 8%) TestOptimizers2 [0]
• ( 2%) TestFullyConnected2 [0]
• (23%) TestNeuralNetwork2 [0, 1]

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 13

 Bonus Points Distribution

• Ex0 (1% in exam = 0.6 point from 60 points)

• Ex1 (1.5% in exam = 0.9 point from 60 points)
• Ex2 (3% in exam = 1.8 point from 60 points)
• Ex3 (3% in exam = 1.8 point from 60 points)
• (2.5%) TestSigmoid
• (2.5%) TestTanH
• ( 5%) TestConstraints [2, 5]
• ( 5%) TestDropout
• (25%) TestBatchNorm [2, 3, 4, 6]
• (40%) TestRNN [2, 4, 6]
• (20%) TestNeuralNetwork3 [0, 1]
• Ex4 (1.5% in exam = 0.9 point from 60 points)
• All Tests + Leaderboard f1 mean 0.6

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 14

 Bonus Points Table

Example of bonus table for ex0. Tutors will assign your bonus points base on this table. See Description.pdf how
to run this table.

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 15

 Bonus Points - Dependencies

• [0] requires its predecessor (e.g. TestOptimizers1 requires TestOptimizers1)

• [1] requires all tests of current exercise
• [2] requires TestOptimizers1
• [3] requires TestFlatten
• [4] requires TestInitialization
• [5] requires TestOptimizers2
• [6] requires TestFullyConnected1

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 16

 Bonus Points - Recommendation

Our recommmendation:
Do them all! Why?
1. You do not get confused by dependencies (it’s easier to keep an overview if
you do them all)

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 17

 Bonus Points - Recommendation

Our recommmendation:
Do them all! Why?
1. You do not get confused by dependencies (it’s easier to keep an overview if
you do them all)
2. It is easier to get used to the framework if you do everything

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 17

 Bonus Points - Recommendation

Our recommmendation:
Do them all! Why?
1. You do not get confused by dependencies (it’s easier to keep an overview if
you do them all)
2. It is easier to get used to the framework if you do everything
3. You get the maximum of bonus points - 10% (tentatively two grades).

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 17

 Bonus Points - Recommendation

Our recommmendation:
Do them all! Why?
1. You do not get confused by dependencies (it’s easier to keep an overview if
you do them all)
2. It is easier to get used to the framework if you do everything
3. You get the maximum of bonus points - 10% (tentatively two grades).
4. We will cover content for the exam - thus highly relevant!

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 17

 Bonus Points - Recommendation

Our recommmendation:
Do them all! Why?
1. You do not get confused by dependencies (it’s easier to keep an overview if
you do them all)
2. It is easier to get used to the framework if you do everything
3. You get the maximum of bonus points - 10% (tentatively two grades).
4. We will cover content for the exam - thus highly relevant!
5. Perform way better in the exam

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 17

 Bonus Points - Recommendation (cont.)

Figure: performance of 267 students from SS22. Students who completed all exercises perform much better than
students who haven’t completed the exercises

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 18

Python Overview
 About Python...

• Programming language with good readability

• Interpreted scripting language
→ Relies on the call of libraries written in lower-level programming languages
→ Basic programming semantics exist but are very inefficient
• Huge amount of libraries for all sorts of applications

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 19

 About Numpy...

• Essential python package

• Central object: Numpy array
→ Acts like a matrix/vector
→ Enables all sorts of mathematical operations
→ Optimised for speed
• A cheat sheet with handy functions for this exercise can be found in the
StudOn group

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 20

 About Scipy...

• Python package closely linked to numpy

• Provides additional functionality
→ Signal processing
→ Statistical operations

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 21

Recommendations
 Package Manager (not needed in CIPs)

We recommend Anaconda (Windows)

• Open source
• One click installation
• Also installs python
• Easy handling of virtual environments

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 22

 IDE
We recommend PyCharm
• Open source
• Easy package handling
• Debugging possibilities
• Free licenses for professional version for students

One alternative: Visual Studio Code with Live Share

Plugin (allows remote pair programming)

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 23

 Version Control

We recommend using GitLab!

• Please use the university’s gitlab server: https://gitlab.cs.fau.de/
• Perfect for co-working
• Compare your code with old versions
• Please use private projects! You can add your study partner as additional
developer.

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 24

Today’s Exercise
 1st Task

Use basic numpy functions to create

• A binary checkerboard pattern
• A binary circle
• An RGB color spectrum

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 25

 2nd Task

Often it is not possible or desired to train your neural network on the whole data set
at once. → Divide your data into smaller portions.
Use numpy to implement an image generator class which also enables data
augmentation.
• The generator yields so called batches (subsets of the training data) in an
iterative manner.
• Batch in this context means a set of images, which are returned at once (by
calling "next").
• These batches of images must be returned together with their corresponding
labels.
• It returns batches until no training samples are left. One pass through the
whole data set is also known as one epoch.

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 26

 Figure: Example image generator output.

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 27

 Get Started

• Open the IDE of your choice

• If you want to use PyCharm in the CIP:
type module load pycharm-community into the console and open it by
typing pycharm
• Follow the instructions of the exercise sheet
• Implement the tasks

Yang, Rist | DL Exercise 0: Numpy Tutorial October 20, 2023 28

Thanks for listening.
Any questions?