Scribd
Upload
15 views16 pages

Deep Learning Types

Uploaded by

luthfihakim2004
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
15 views16 pages

Deep Learning Types

Uploaded by

luthfihakim2004
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 16

Types of Deep Learning

Dr. Asif Awaludin


Why Deep Learning : Decision boundary

Decision boundary of logistic regression logistic regression cannot learn decision


boundaries for nonlinear data

Machine Learning algorithms cannot solve classification problem that involve a complex relationship
Why Deep Learning : Feature Engineering
Feature engineering is a two-step process:

➢ Feature extraction : extract all the required


features for our problem statement

➢ Feature selection : select the important


features that improve the performance

Consider an image classification problem.


Extracting features manually from an image
needs strong knowledge of the subject as well
as the domain.
It is an extremely time-consuming process.

Thanks to Deep Learning, we can automate


the process of Feature Engineering!
Basic Neural Network : Percepteron
Types of Neural Networks in Deep Learning

• Artificial Neural Networks (ANN)


• Recurrent Neural Networks (RNN)
• Convolution Neural Networks (CNN)
Artificial Neural Networks (ANN)
ANN is a Feed-Forward NN consisting a group of multiple percepterons at each layer.
ANN is capable of learning any nonlinear function.
Hence, popularly known as Universal Function Approximators.

Contoh Artificial Neural Networks :

from keras.layers import Dense


from keras.models import Sequential
from keras import optimizers

model = Sequential()
model.add(Dense(2, input_shape = (2,), activation='sigmoid'))
model.add(Dense(10, activation= 'sigmoid'))
model.add(Dense(10, activation= 'sigmoid'))
model.add(Dense(1))

model.fit(X_train, Y_train, batch_size= 40, epochs=100, verbose= 0)


train_pred = model.predict(X_train)
test_pred = model.predict(X_test)
Recurrent Neural Network (RNN)

As you can see here, RNN


has a recurrent connection
on the hidden state.

This looping constraint


ensures that sequential
information is captured in
the input data.
Advantages of RNN (1)
• RNN captures the sequential information present in the input data
i.e. dependency between the words in the text while making
predictions
Advantages of RNN (2)
• RNNs share the parameters across different time steps. This is
popularly known as Parameter Sharing. This results in fewer
parameters to train and decreases the computational cost
Hidden state:

h(t) represents a hidden state at


time t and acts as “memory”
of the network.

h(t) is calculated based on the


current input and the previous
time step’s hidden state:
h(t) = f(U x(t) + W h(t−1)).
Long Short-Term Memory (LSTM)
• LSTM networks are a type of recurrent neural network (RNN) designed to capture
long-term dependencies in sequential data.
• LSTM networks have memory cells and gates that allow them to retain or forget
information over time selectively. This makes LSTMs effective in speech recognition,
natural language processing, time series analysis, and translation.
• The challenge with LSTM networks lies in selecting the appropriate architecture and
parameters and dealing with vanishing or exploding gradients during training.
Long Short-Term Memory (LSTM)
Convolution Neural Networks (CNN)
• CNNs are well suited to image classification and object recognition tasks.
• It works by transforming an input image into a feature map, which is then processed through
multiple convolutional and pooling layers to produce a predicted output
Convolution Layer Pooling Layer

Terjadi konvolusi memanfaatkan filter yang Untuk mereduksi input secara spasial dengan
memiliki tinggi, lebar, dan tebal tertentu. operasi down-sampling. Umumnya, metode
Filter ini diinisialisasi dengan nilai tertentu dan pooling yang digunakan adalah max pooling atau
nilainya akan di-update dalam proses learning. mengambil nilai terbesar dari bagian tersebut

Input Image Activation Map

Konvolusi
Filter/kernel
from keras.utils.np_utils import to_categorical
from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten, Conv2D, MaxPool2D # data augmentation to avoid overfitting problem
from keras.optimizers import RMSprop,Adam datagen = ImageDataGenerator(
from keras.preprocessing.image import ImageDataGenerator featurewise_center=False,
from keras.callbacks import ReduceLROnPlateau samplewise_center=False,
featurewise_std_normalization=False,
model = Sequential() samplewise_std_normalization=False,
# zca_whitening=False,
model.add(Conv2D(filters = 8, kernel_size = (5,5),padding = 'Same', rotation_range=5, zoom_range = 0.1,
activation ='relu', input_shape = (28,28,1))) width_shift_range=0.1,
model.add(MaxPool2D(pool_size=(2,2))) height_shift_range=0.1,
model.add(Dropout(0.25)) horizontal_flip=False,
# vertical_flip=False)
model.add(Conv2D(filters = 16, kernel_size = (3,3),padding = 'Same',
activation ='relu')) datagen.fit(X_train)
model.add(MaxPool2D(pool_size=(2,2), strides=(2,2)))
model.add(Dropout(0.25)) # Fit the model
# fully connected history = model.fit_generator(datagen.flow(X_train,Y_train,
model.add(Flatten()) batch_size=batch_size), epochs = epochs, validation_data =
model.add(Dense(256, activation = "relu")) (X_val,Y_val), steps_per_epoch=X_train.shape[0] // batch_size)
model.add(Dropout(0.5))
model.add(Dense(10, activation = "softmax")) Y_pred = model.predict(X_val)
# Compile the model
model.compile(optimizer = optimizer , loss = "categorical_crossentropy",
metrics=["accuracy"])

You might also like