Scribd Logo
Upload

Welcome to Scribd!

  • 5 views

    Tutorial 1 - Regularization Techniques Part 1 - Neuromatch Academy - Deep Learning

    Uploaded by

    Ravi kumar
    The document discusses regularization techniques for deep learning models. It introduces regularization as a method for reducing overfitting in neural networks by preventing complex co-adaptations on the training data. The document covers how overfitting occurs in large neural networks, and how techniques like regularization, early stopping, and adding noise to the training process can help improve generalization to new examples.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    Tutorial 1 - Regularization Techniques Part 1 - Neuromatch Academy - Deep Learning

    Uploaded by

    Ravi kumar
    5 views3 pages
    The document discusses regularization techniques for deep learning models. It introduces regularization as a method for reducing overfitting in neural networks by preventing complex co-adaptations on the training data. The document covers how overfitting occurs in large neural networks, and how techniques like regularization, early stopping, and adding noise to the training process can help improve generalization to new examples.

    Original Title

    Tutorial 1- Regularization techniques part 1 — Neuromatch Academy- Deep Learning

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    The document discusses regularization techniques for deep learning models. It introduces regularization as a method for reducing overfitting in neural networks by preventing complex co-adaptations on the training data. The document covers how overfitting occurs in large neural networks, and how techniques like regularization, early stopping, and adding noise to the training process can help improve generalization to new examples.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    5 views3 pages

    Tutorial 1 - Regularization Techniques Part 1 - Neuromatch Academy - Deep Learning

    Uploaded by

    Ravi kumar
    The document discusses regularization techniques for deep learning models. It introduces regularization as a method for reducing overfitting in neural networks by preventing complex co-adaptations on the training data. The document covers how overfitting occurs in large neural networks, and how techniques like regularization, early stopping, and adding noise to the training process can help improve generalization to new examples.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 3

    Skip to main content

    Tutorial 1: Regularization techniques part 1


    Print to PDF
    Contents
    Tutorial 1: Regularization techniques part 1
    Tutorial Objectives
    Setup
    Section 0: Defining useful functions
    Section 1: Regularization is Shrinkage
    Section 2: Overfitting
    Section 3: Memorization
    Section 4: Early Stopping
    Summary
    Bonus: Train with randomized labels

    Open in Colab Open in Kaggle

    Week 2, Day 1: Regularization

    By Neuromatch Academy

    Content creators: Ravi Teja Konkimalla, Mohitrajhu Lingan Kumaraian, Kevin Machado Gamboa, Kelson Shilling-Scrivo,
    Lyle Ungar

    Content reviewers: Piyush Chauhan, Siwei Bai, Kelson Shilling-Scrivo

    Content editors: Roberto Guidotti, Spiros Chavlis

    Production editors: Saeed Salehi, Gagana B, Spiros Chavlis

    1. Big Artificial Neural Networks (ANNs) are efficient universal approximators due to their adaptive basis functions
    2. ANNs memorize some but generalize well
    3. Regularization as shrinkage of overparameterized models: early stopping

    If you want to download the slides: https://osf.io/download/mf79a/

    https://deeplearning.neuromatch.io/tutorials/W2D1_Regularization/student/W2D1_Tutorial1.html 04/06/24, 3 01 PM
    Page 1 of 34
    :
    Note that some of the code for today can take up to an hour to run. We have therefore “hidden” the code and shown
    the resulting outputs.

    Install dependencies
    WARNING: There may be errors and/or warnings reported during the installation. However, they should be ignored.

    Show code cell source

    Install and import feedback gadget


    Show code cell source

    # Imports
    import time
    import copy
    import torch
    import pathlib

    import numpy as np
    import torch.nn as nn
    import torch.optim as optim
    import torch.nn.functional as F
    import matplotlib.pyplot as plt
    import matplotlib.animation as animation

    from tqdm.auto import tqdm


    from IPython.display import HTML
    from torchvision import transforms
    from torchvision.datasets import ImageFolder

    Figure Settings
    Show code cell source

    https://deeplearning.neuromatch.io/tutorials/W2D1_Regularization/student/W2D1_Tutorial1.html 04/06/24, 3 01 PM
    Page 2 of 34
    :
    Loading Animal Faces data
    Show code cell source

    Start downloading and unzipping `AnimalFaces` dataset...

    Download completed.

    Loading Animal Faces Randomized data


    Show code cell source

    Start downloading and unzipping `Randomized AnimalFaces` dataset...

    Download completed.

    Plotting functions
    Show code cell source

    Set random seed


    Executing set_seed(seed=seed) you are setting the seed

    Show code cell source

    Set device (GPU or CPU). Execute set_device()


    Show code cell source

    SEED = 2021
    set_seed(seed=SEED)
    DEVICE = set_device()

    Random seed 2021 has been set.


    WARNING: For this notebook to perform best, if possible, in the menu under `Runtime` -> `Change
    runtime type.` select `GPU`

    Let’s start the tutorial by defining some functions which we will use frequently today, such as: AnimalNet , train ,
    test and main .

    https://deeplearning.neuromatch.io/tutorials/W2D1_Regularization/student/W2D1_Tutorial1.html 04/06/24, 3 01 PM
    Page 3 of 34
    :

    You might also like