Scribd
Upload
47 views18 pages

00 Intro

This document outlines a course on neural networks and deep learning. It will be taught by Jacopo Pegoraro and Daniele Mari over 24 hours of theory lessons and 16 hours of lab sessions. The course will cover fundamental concepts in neural networks as well as advanced architectures like convolutional neural networks and generative adversarial networks. Students will complete a group project and oral presentation as their exam. Prerequisites include calculus, probability, and linear algebra.

Uploaded by

Ghaleb Elayan
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
47 views18 pages

00 Intro

This document outlines a course on neural networks and deep learning. It will be taught by Jacopo Pegoraro and Daniele Mari over 24 hours of theory lessons and 16 hours of lab sessions. The course will cover fundamental concepts in neural networks as well as advanced architectures like convolutional neural networks and generative adversarial networks. Students will complete a group project and oral presentation as their exam. Prerequisites include calculus, probability, and linear algebra.

Uploaded by

Ghaleb Elayan
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 18

Neural Networks and Deep Learning

Jacopo Pegoraro Daniele Mari


[email protected] [email protected]

2022-2023
Instructors
● Jacopo Pegoraro ([email protected])
● 16 theory lessons
● Applications
● 32 hours

● Daniele Mari ([email protected])


● 8 lab sessions
● 16 hours
Contact us
● For any question regarding theory lessons or the exam
● Email [email protected]
● Start the object of the email with «[NNDL]»

● For any questions regarding lab sessions


● Email [email protected]
● Cc [email protected]
● Start the object of the email with «[NNDL]»
(Artificial) Neural Networks (ANN)
● Network of artificial neurons
● Input signals propagate to outputs
● Layered structure
● Tunable connectivity

● ANN vs Human brain


● Extremely simplified topology
● Discrete vs continuous inputs
Course outline (1)
1. Neural Networks basics
● Learning problems and optimization methods
● Regression and classification
● Overfitting and regularization
● Feedforward Neural Networks (FFNN)
● Training neural networks: backpropagation
● Convolutional Neural Networks (CNN)
● Overfitting in NNs and advanced optimization
Course outline (2)
2. Advanced NN architectures
● Batch normalization
● Residual networks
● Auto-encoders and variational auto-encoders
● Generative Adversarial Networks
● Continual Learning
Lab sessions (1)
● Python programming language

● library for scientific computing

● library to build NN models

● Guided lab coding sessions


● Google Colab
● Nice and clean text/code integration
● Easy access to GPUs
Lab sessions (2)
1. Intro to Python & NumPy + Least Squares
2. Intro to PyTorch
3. Regression and classification with a NN (PyTorch)
4. Neural Network implementation from scratch (NumPy)
5. Convolutional NN and Residual Networks
6. Auto-encoders and Variational Auto-encoders
7. Generative Adversarial Networks
8. Continual Learning
Prerequisites
● For the theory lessons
● Calculus - derivatives, functions, integrals, …
● Probability - distributions, expectation, variance, …
● Linear Algebra - matrix/vector operations, matrix properties, …

● For the labs


● Everything will be covered from scratch
● Basics of programming can help
Relation to other courses
● Machine Learning (ICT)
● Highly recommended Deep
Learning
● Learning theory
● Non NN-based algorithms
Neural Networks
● Human Data Analytics (ICT)
● Advanced ML&DL course Machine Learning
● Focus on sequential data
Course material
● Lecture slides
● https://stem.elearning.unipd.it/course/view.php?id=3680
● Enrollment key: nndl22
● Cover theory and (some) applications

● Lab material
● Code (notebooks)
● Links to documentation and tutorials

● Lecture recordings
Self study material
● Books
[1] J. Watt, R. Borhani, A. K. Katsaggelos, Machine learning refined: foundations,
algorithms and applications, Cambridge University Press, 2016.
[2] S. S. Shwartz, S. Ben-David, Understanding machine learning: from theory to
algorithms, Cambridge University Press, 2014.
[3] Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press,
2016. https://www.deeplearningbook.org
[4] Christopher M. Bishop, and Nasser M. Nasrabadi. Pattern recognition and machine
learning. New York: Springer, 2006.

● Research papers provided after each lecture


Exam modalities
● STEP 1: GROUP PROJECT (MAX 3 people)
● Implement and test a neural network on a public dataset
● Projects provided by the instructors (or proposed by you, after approval)
● STEP 2: PROJECT OUTPUT
● Report: max 10 pages + contribution description for each member
● Complete source code
● To be delivered 1 week before the oral exam in step 3
● STEP 3: ORAL PRESENTATION
● 15 min. per group, each member gets to present something
● 10 min. questions on the project and/or on the course topics
Grading
● PROJECT – 50% ● Originality
● Preprocessing
● Learning model
● Comparison with other methods

● WRITTEN REPORT - 25% ● Clarity


● Completeness
● Results

● ORAL – 25% ● Clarity of the presentation


● Questions
Research
● Neuromorphic computing
● Deep learning applied to Radio Frequency signals
Neuromorphic computing
Synaptic
weights
Information is
encoded in
continuous
time pulses

Leaky integrate-and-fire model

Sensory Neural networks


inputs that mimic the
brain in hardware
DL for joint communication and sensing
● Indoor human detection and sensing through a testbed implementing the new
IEEE 802.11ay Wi-Fi standard at 60 GHz

processed Wi-Fi data


HAR classifier Identification classifier
• Walking • Person 0
•

Softmax

Softmax
Running

Dense

Dense
• Person 1
Residual Block • Sitting Residual Block • Person 2
• Waving hands
X4 X4 • …
• Standing
Walking spectrogram
micro-Doppler input
If you are curious…
● M. Canil*, J. Pegoraro*, and M. Rossi, “MilliTRACE-IR: Contact Tracing and Temperature
Screening via mmWave and Infrared Sensing”, IEEE Journal of Selected Topics in Signal
Processing, vol. 16, no. 2, pp. 208-223, May 2022.

● J. Pegoraro and M. Rossi, “Human Tracking with mmWave Radars: a Deep Learning
Approach with Uncertainty Estimation”, IEEE International Workshop on Signal Processing
Advances in Wireless Communications (SPAWC), Oulu, Finland, July 2022.

● J. Pegoraro, J. O. Lacruz, F. Meneghello, E. Bashirov, M. Rossi, and J. Widmer, “RAPID:


Retrofitting IEEE 802.11ay Access Points for Indoor Human Detection and Sensing”, under
submission, 2022.

● Furber Steve and Temple Steve, Neural systems engineering Journal of the Royal Society
Interface, 4193–206, 2007

You might also like