Scribd Logo
Upload

Welcome to Scribd!

  • 23 views

    DeepLearning Tutorial3

    Uploaded by

    Bharath kumar
    This document discusses using deep neural networks (DNNs) as models of the visual system and comparing their representations to those in the brain. It describes training a DNN on an orientation discrimination task and using representational similarity analysis to compare the network's internal representations to neural responses recorded from brain area V1. The analysis shows that the pooling layer representations best match V1, while the fully connected layer representations diverge. This demonstrates that DNNs can be used as normative models of the visual system and RSA is an effective way to evaluate how similar the models' representations are to those in the brain.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    DeepLearning Tutorial3

    Uploaded by

    Bharath kumar
    23 views16 pages
    This document discusses using deep neural networks (DNNs) as models of the visual system and comparing their representations to those in the brain. It describes training a DNN on an orientation discrimination task and using representational similarity analysis to compare the network's internal representations to neural responses recorded from brain area V1. The analysis shows that the pooling layer representations best match V1, while the fully connected layer representations diverge. This demonstrates that DNNs can be used as normative models of the visual system and RSA is an effective way to evaluate how similar the models' representations are to those in the brain.

    Original Description:

    DeepLearning_Tutorial3

    Original Title

    DeepLearning_Tutorial3

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    This document discusses using deep neural networks (DNNs) as models of the visual system and comparing their representations to those in the brain. It describes training a DNN on an orientation discrimination task and using representational similarity analysis to compare the network's internal representations to neural responses recorded from brain area V1. The analysis shows that the pooling layer representations best match V1, while the fully connected layer representations diverge. This demonstrates that DNNs can be used as normative models of the visual system and RSA is an effective way to evaluate how similar the models' representations are to those in the brain.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    23 views16 pages

    DeepLearning Tutorial3

    Uploaded by

    Bharath kumar
    This document discusses using deep neural networks (DNNs) as models of the visual system and comparing their representations to those in the brain. It describes training a DNN on an orientation discrimination task and using representational similarity analysis to compare the network's internal representations to neural responses recorded from brain area V1. The analysis shows that the pooling layer representations best match V1, while the fully connected layer representations diverge. This demonstrates that DNNs can be used as normative models of the visual system and RSA is an effective way to evaluate how similar the models' representations are to those in the brain.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 16

    Deep learning in neuroscience,

    Tutorial 3
    Yalda Mohsenzadeh, PhD
    About me

    Assistant Professor of Computer Science


    Brain and Mind Institute
    Western University (Canada)

    Cognitive Neuroscience & AI Lab:

    ○ Machine Vision and AI

    ○ Human Perception and Memory

    ○ Cognitive Computational Neuroscience


    Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 2
    DNNs as models of the visual System

    (Khaligh-Razavi et al. 2014, Yamins et al. 2014, Guclu et al. 2015, Cichy et al. 2016; Mohsenzadeh et al. 2020)

    Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 3


    Using an orientation discrimination task
    reference
    orientation
    stimulus stimulus

    Train a deep network to perform this task,


    are its representations of the stimuli the
    same as the brain’s representations?

    choose left choose right

    Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 4


    A neural network for orientation discrimination

    Jorge A. Menendez
    Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 5
    Convolutional neural network (in 2D)

    input
    filter
    output

    Pooling
    (K=2) Pooling ⇒ stability w.r.t.
    small translations

    Vincent Dumoulin, Francesco Visin - A guide to convolution arithmetic for deep learning; Jorge A. Menendez

    Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 6


    Pytorch code for network

    Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 7


    Comparing
    representations
    between brains
    and DNNs

    Yalda Mohsenzadeh ⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 8


    Example unit activations from notebook

    Brain V1 Responses Model Pool Layer Responses Model FC Layer Responses


    Neural Response

    Stimulus Orientation Stimulus Orientation Stimulus Orientation


    Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 9
    Representational Dissimilarity Matrix (RDM)
    neural
    activity
    stimulus pattern

    Compute
    dissimilarity
    (1 - corr_coef)

    Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 10


    RDMs from orientation discrimination task

    Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 11


    Representational Similarity Analysis (RSA)
    Activation Patterns

    Correlation of Model Layer RDM


    versus

    with Brain V1 RDM


    Correlation
    Neural Network Layer

    versus

    Pool Layer FC Layer


    Region of Interest Neural Network Model Layer
    Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 12
    Python code (RSA)

    Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 13


    Representational Similarity Analysis (RSA)

    Brain RDM Image pixels RDM

    High

    Dissimilarity
    Neural Net RDM Behavior RDM
    Low

    Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 14


    What have we learned?

    ● How to use deep learning to build a normative encoding model of the visual
    system

    ● How to use RSA to evaluate how the model's representations match to those
    in the brain

    Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 15


    T-SNE Visualization of Population Responses

    Brain V1 Responses Model Pool Layer Responses Model FC Layer Responses

    Stimulus Orientation

    Stimulus Orientation

    Stimulus Orientation
    Dimension 2

    Dimension 1 Dimension 1 Dimension 1


    Yalda Mohsenzadeh⦁ Deep Learning Week 2 ⦁ Day 1 ⦁ Tutorial 3 16

    You might also like