Deep Neural

Networks

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Deep Learning in 2D image (Recap)
Fully-Connected Fully-Connected
Neural Network Neural Network
Conv2 ReLU activation Softmax activation
Conv1 Convolution
Convolution 16 filters, 3x3
8 filters, 3x3 Valid padding
Valid padding Max-Pooling Max-Pooling
2x2, Stride=2 Stride=1 2x2, Stride=2
Stride=1

Cat

8 Channels 8 Channels 16 Channels 16 Channels

INPUT
28x28x3
Input Image Classification Model 10 Units
64 Units
Output Class

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Deep Learning in 1D Signal
Convolutional layers Flatten layer Fully-Connected Layer

Cat

Input Signal Output Class

Classification Model

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Convolution in Time Series Signal

Inverted Kernel Convolution Operation

1 2 .5 1 x 0 =0 0

Time series Signal

Padding = Same
0 1 0 0 0 0 -1 0 0 0 0

Time series Signal Representation

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Convolution in Time Series Signal

Inverted Kernel Convolution Operation

1 2 .5 2 x 1 =2 0 2

Time series Signal

Padding = Same
0 1 0 0 0 0 -1 0 0 0 0

Time series Signal Representation

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Convolution in Time Series Signal

Inverted Kernel Convolution Operation

1 2 .5 .5 x 0 = 0 0 2 0
+
Time series Signal
2 Padding = Same
0 1 0 0 0 0 -1 0 0 0 0

Time series Signal Representation

Amity Centre for Artificial Intelligence, Amity University, Noida, India

Convolution in Time Series Signal
Time series Signal
0 0 1 0 0 0 0 -1 0 0 0

Inverted Kernel
Padding = Same
1 2 .5

=
Time series Signal
0 .5 2 1 0 0 -.5 -2 -1 0 0

Amity Centre for Artificial Intelligence, Amity University, Noida, India

Convolution in Time Series Signal
Kernel Kernel size=3

Time series Signal

X X

Padding = Same

Convolution
Inverted Kernel

Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

Convolution in Time Series Signal
Kernel size=3

Time series Signal

X X X

Padding = Same

Convolution

Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

Convolution in Time Series Signal
Kernel size=3

Time series Signal

X X X

Padding = Same

Convolution

Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

Convolution in Time Series Signal
Kernel size=3

Time series Signal

X X X

Padding = Same

Convolution

Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

Convolution in Time Series Signal
Kernel size=3

Time series Signal

X X X

Padding = Same

Convolution

Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

Convolution in Time Series Signal
Kernel size=3

Time series Signal

X X X

Padding = Same

Convolution

Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

Convolution in Time Series Signal
Kernel size=3

Time series Signal

X X X

Padding = Same

Convolution

Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

Convolution in Time Series Signal
Kernel size=3

Time series Signal

X X X

Padding = Same

Convolution

Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

Convolution in Time Series Signal
Kernel size=3

Time series Signal

X X X

Padding = Same

Convolution

Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

Convolution in Time Series Signal
Kernel size=3

Time series Signal

X X X

Padding = Same

Convolution

Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

Convolution in Time Series Signal
Kernel size=3

Time series Signal

X X

Padding = Same

Convolution

Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Activation Function

0 .5 2 1 0 0 -.5 -2 -1 0 0
Time series Signal

ReLu Activation

0 .5 2 1 0 0 -.5 -2 -1 0 0 Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Activation Function

0 .5 2 1 0 0 -.5 -2 -1 0 0
Time series Signal

ReLu Activation

0 .5 2 1 0 0 -.5 -2 -1 0 0 Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Activation Function

0 .5 2 1 0 0 -.5 -2 -1 0 0
Time series Signal

ReLu Activation

0 .5 2 1 0 0 -.5 -2 -1 0 0 Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Activation Function

0 .5 2 1 0 0 -.5 -2 -1 0 0
Time series Signal

ReLu Activation

0 .5 2 1 0 0 -.5 -2 -1 0 0 Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Activation Function

0 .5 2 1 0 0 -.5 -2 -1 0 0
Time series Signal

ReLu Activation

0 .5 2 1 0 0 0 0 -1 0 0 Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Activation Function

0 .5 2 1 0 0 -.5 -2 -1 0 0
Time series Signal

ReLu Activation

0 .5 2 1 0 0 0 0 0 0 0 Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Activation Function

0 .5 2 1 0 0 -.5 -2 -1 0 0
Time series Signal

ReLu Activation

0 .5 2 1 0 0 0 0 0 0 0 Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Activation Function

Time series Signal

ReLu Activation

Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Activation Function

Time series Signal

ReLu Activation

Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Activation Function

Time series Signal

ReLu Activation

Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Activation Function

Time series Signal

ReLu Activation

Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Activation Function

Time series Signal

ReLu Activation

Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Activation Function

Time series Signal

ReLu Activation

Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Activation Function

Time series Signal

ReLu Activation

Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Activation Function

Time series Signal

ReLu Activation

Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Activation Function

Time series Signal

ReLu Activation

Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Activation Function

Time series Signal

ReLu Activation

Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Activation Function

Time series Signal

ReLu Activation

Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Max Pooling
Time series Signal
0 .5 2 1 0 0 0 0 0 0 0

max

Time series Signal

.5

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Max Pooling
Time series Signal
0 .5 2 1 0 0 0 0 0 0 0

max

Time series Signal

.5 2

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Max Pooling
Time series Signal
0 .5 2 1 0 0 0 0 0 0 0

max

Time series Signal

.5 2 0

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Max Pooling
Time series Signal
0 .5 2 1 0 0 0 0 0 0 0

max

Time series Signal

.5 2 0 0

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Max Pooling
Time series Signal
0 .5 2 1 0 0 0 0 0 0 0

max

Time series Signal

.5 2 0 0 0

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Max Pooling
Max Pool size=2

Time series Signal

X X

Pooling
max

Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Max Pooling
Max Pool size=2

Time series Signal

X X

Pooling
max

Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Max Pooling
Max Pool size=2

Time series Signal

X X

Pooling
max

Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Max Pooling
Max Pool size=2

Time series Signal

X X

Pooling
max

Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Max Pooling
Max Pool size=2

Time series Signal

X X

Pooling
max

Result

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Convolution with 2 kernels
0 0
0 0.5
1 2
0 1 1
0 2 0 Padding = Same
0 0.5 0 Activation = ReLU
0 Inverted Kernel 1 0
-1 0
0 0
0 0
0 0
Time series Signal
Feature Maps

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Convolution with 2 kernels
0 0
0
0 0.5
2
1 2
1.5
0 1 1
0 0
0 2 0 Padding = Same
1.5 0
0 0.5 0 Activation = ReLU
2 0
0 0
Inverted Kernel 2 0
-1 0
0
0 0
0
0 0
0
0 0
0
Time series Signal
Feature Maps

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Flatten Layer
0
0 0
0
0 0.5
2
1 2
1.5
0 1 1
0 0
0 2 0 Padding = Same
1.5 0
0 0.5 0 Activation = ReLU
2 0
0 0
0
-1 0
0
0 0
0
0 0
0
0 0
0
Time series Signal
Feature Maps Flatten Layer

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Flatten Layer
0
0 0
0 0.5
0 0.5
2
1 2
1.5
0 1 1
0 0
0 2 0 Padding = Same
1.5 0
0 0.5 0 Activation = ReLU
2 0
0 0
0
-1 0
0
0 0
0
0 0
0
0 0
0
Time series Signal
Feature Maps Flatten Layer

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Flatten Layer
0
0 0
0 0.5
0 0.5
2 2
1 2
1.5
0 1 1
0 0
0 2 0 Padding = Same
1.5 0
0 0.5 0 Activation = ReLU
2 0
0 0
0
-1 0
0
0 0
0
0 0
0
0 0
0
Time series Signal
Feature Maps Flatten Layer

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Flatten Layer
0
0 0
0 0.5
0 0.5
2 2
1 2
1.5 .
0 1 1 .
0 0 .
0 2 0 Padding = Same
1.5 0 .
0 0.5 0 Activation = ReLU
2 0 .
0 0 .
0 .
-1 0
0 .
0 0
0
0 0
0
0 0
0
Time series Signal
Feature Maps Flatten Layer

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Flatten Layer
0
0 0
0 0.5
0 0.5
2 2
1 2
1.5 .
0 1 1 .
0 0 .
0 2 0 Padding = Same
1.5 0 .
0 0.5 0 Activation = ReLU
2 0 .
0 0 .
0 .
-1 0
0 .
0 0
0 0
0 0
0
0 0
0
Time series Signal
Feature Maps Flatten Layer

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Flatten Layer
0
0 0
0 0.5
0 0.5
2 2
1 2
1.5 .
0 1 1 .
0 0 .
0 2 0 Padding = Same
1.5 0 .
0 0.5 0 Activation = ReLU
2 0 .
0 0 .
0 .
-1 0
0 .
0 0
0 0
0 0
0 0
0 0
0
Time series Signal
Feature Maps Flatten Layer

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Flatten Layer
0
0 0
0 0.5
0 0.5
2 2
1 2
1.5 .
0 1 1 .
0 0 .
0 2 0
1.5 0 .
0 0.5 0
2 0 .
0 0 .
0 .
-1 0
0 .
0 0
0 0
0 0
0 0
0 0
0 0
Time series Signal
Feature Maps Flatten Layer

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Fully Connected Layer
0
0 0
0 0.5
0 0.5
2 2
1 2
1.5 .
0 1 1 . 7
0 0 .
0 2 0
1.5 0 .
0 0.5 0
2 0 . -2
0 0 .
0 .
-1 0
0 .
0 0 0.4
0 0
0 0
0 0
0 0
0 0
Time series Signal
Feature Maps Flatten Layer FC Layer

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Activation Function
0
0 0
0 0.5
0 0.5
2 2
1 2
1.5 .
0 1 1 . 7
0 0 .
0 2 0
1.5 0 .
0 0.5 0
2 0 . -2
0 0 .
0 . Sigmoid
-1 0
0 .
0 0 0.4
0 0
0 0
0 0
0 0
0 0
Time series Signal
Feature Maps Flatten Layer FC Layer

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Activation Function
0
0 0
0 0.5
0 0.5
2 2
1 2
1.5 .
0 1 1 7 0.99
.
0 0 .
0 2 0
1.5 0 .
0 0.5 0
2 0 . -2
0 0 .
0 . Sigmoid
-1 0
0 .
0 0 0.4
0 0
0 0
0 0
0 0
0 0
Time series Signal
Feature Maps Flatten Layer FC Layer

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Activation Function
0
0 0
0 0.5
0 0.5
2 2
1 2
1.5 .
0 1 1 7 0.99
.
0 0 .
0 2 0
1.5 0 .
0 0.5 0
2 0 . -2 0.1
0 0 .
0 . Sigmoid
-1 0
0 .
0 0 0.4
0 0
0 0
0 0
0 0
0 0
Time series Signal
Feature Maps Flatten Layer FC Layer

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Activation Function
0
0 0
0 0.5
0 0.5
2 2
1 2
1.5 .
0 1 1 7 0.99
.
0 0 .
0 2 0
1.5 0 .
0 0.5 0
2 0 . -2 0.1
0 0 .
0 . Sigmoid
-1 0
0 . 0.59
0 0 0.4
0 0
0 0
0 0
0 0
0 0
Time series Signal
Feature Maps Flatten Layer FC Layer

Amity Centre for Artificial Intelligence, Amity University, Noida, India

 Classification
0
0 0
0 0.5
0 0.5
2 2
1 2
1.5 . Class 1
0 1 1 7 0.99
.
0 0 .
0 2 0
1.5 0 .
0 0.5 0
2 0 . -2 0.1
0 0 .
0 . Sigmoid
-1 0
0 . 0.59
0 0 0.4
0 0
0 0
0 0
0 0
0 0 Probability value
Time series Signal
Feature Maps Flatten Layer FC Layer for 3 classes

Amity Centre for Artificial Intelligence, Amity University, Noida, India

Time Series Signal Classifications using CNN

Input CNN Architecture for 1D signal Output

Amity Centre for Artificial Intelligence, Amity University, Noida, India