Scribd
Upload
2 views

Lecture 4-Deep Learning and Cognitive Computing

This lecture focuses on deep learning and cognitive computing, outlining its significance in modern AI and its relationship with traditional machine learning. Key concepts include the structure and functioning of artificial neural networks (ANN), various deep learning methods such as convolutional and recurrent neural networks, and the role of cognitive computing in simulating human thought processes. Additionally, it discusses frameworks for implementing deep learning and the applications of cognitive computing across different industries.

Uploaded by

afranealfred40
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
2 views

Lecture 4-Deep Learning and Cognitive Computing

This lecture focuses on deep learning and cognitive computing, outlining its significance in modern AI and its relationship with traditional machine learning. Key concepts include the structure and functioning of artificial neural networks (ANN), various deep learning methods such as convolutional and recurrent neural networks, and the role of cognitive computing in simulating human thought processes. Additionally, it discusses frameworks for implementing deep learning and the applications of cognitive computing across different industries.

Uploaded by

afranealfred40
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 35

LECTURE 54

DEEP LEARNING AND COGNITIVE COMPUTING


LEARNING OBJECTIVES

i. Learn what deep learning is and how it is changing the world of computing
ii. Know the placement of deep learning within the broad family of AI learning
methods
iii. Understand how traditional “shallow” artificial neural networks (ANN) work
iv. Become familiar with the development and learning processes of ANN
v. Develop an understanding of the methods to shed light into the ANN black box
LEARNING OBJECTIVES

i. Know the underlying concept and methods for deep neural networks
ii. Become familiar with different types of deep learning methods
iii. Understand how convolutional neural networks (CNN), recurrent neural
networks (RNN), and long short-memory networks (LSTM) work
iv. Become familiar with the computer frameworks for implementing deep
learning
v. Know the foundational details about cognitive Computing and IBM
Watson
INTRODUCTION TO DEEP LEARNING

• Imaginative things in the SciFi movies are turning into realities-tanks to AI and
Machine Learning
• Siri, Google assistant, Alexa, Google home, …
• Deep learning is the newest member of the AI/Machine Learning family
• Learn better than ever before
• The reason for Deep Learning superiority
• Automatic feature extraction and representation
INTRODUCTION TO DEEP LEARNING
• The placement of Deep Learning within the overarching AI-based learning methods
INTRODUCTION TO DEEP LEARNING
• Differences between Classic Machine-Learning Methods and Representation
Learning/Deep Learning
BASICS OF “SHALLOW” LEARNING
• Artificial Neural Networks – abstractions of human brain and its complex biological
network of neurons
• Neurons = Processing Elements (PEs)
• Single-input and single-output neuron/PE
BASICS OF “SHALLOW” LEARNING
• Common transfer (activation) functions
BASICS OF “SHALLOW” LEARNING
• Typical multiple-input neuron with R individual inputs

n = w1,1 p1 + w1, 2 p2 + w1,3 p3 + ... + w1, R p R + b


n = Wp + b
BASICS OF “SHALLOW” LEARNING
• Typical Neural Network with three layers and eight neurons
PROCESS OF DEVELOPING NEURAL-NETWORK BASED SYSTEMS
• A process with constant feedbacks for changes and improvements!
LEARNING PROCESS IN ANN
1. Compute temporary outputs.
2. Compare outputs with desired targets.
3. Adjust the weights and repeat the
process.
BACKPROPAGATION FOR A NN TRAINING
1. Initialize weights with random values
2. Read in the input vector and the desired output
3. Compute the actual output via the calculations
4. Compute the error.
5. Change the weights by working backward
BACKPROPAGATION FOR A NN TRAINING
• Illustration of the Overfitting in ANN
ILLUMINATING THE BLACK BOX OF A NN
• ANN are typically known as black boxes
• Sensitivity analysis can shed light to the black-box
DEEP NEURAL NETWORKS

• Deep: more hidden layers


• In addition to CPU, it also uses GPU
• With programming languages like CUDA by NVIDIA
• Needs large datasets
• Deep learning uses tensors as inputs
• Tensor: N-dimensional arrays
• Image representation with 3-D tensors
• There are different types and capabilities of Deep Neural Networks for different tasks/purposes
DEEP NEURAL NETWORKS
Feedforward Multilayer Perceptron (MLP)-Type Deep Networks
• Most common type of deep networks
• Vector Representation of the First Three Layers in a Typical MLP Network.
DEEP NEURAL NETWORKS
• Impact of Random Weights in Deep MLP
• The Effect of Pre-training Network
Parameters on Improving Results of a
Classification-Type Deep Neural
Network.
• More hidden layers versus more
neurons?
CONVOLUTIONAL “DEEP” NEURAL
NETWORKS
• Most popular MLP-base D L method
• Used for image/video processing, text recognition
• Has at least one convolution weight function
• Convolutional layer
• Convolutional layer → Polling (sub-sampling)
• Consolidating the large tensors into one with a smaller size-and reducing the
number of model parameters while keeping only the important features
• There can be different types of polling layers
CONVOLUTION FUNCTION
• Typical Convolutional Network Unit

• Convolution of a 2 x 2 Kernel by a 3 x 6 Input Matrix


IMAGE PROCESSING USING CNN

• ImageNet (http://www.image-net.org)
• Architecture of AlexNet, a CNN for Image Classification
IMAGE PROCESSING USING CN N
• Conceptual Representation of the Inception Feature in GoogLeNet
IMAGE PROCESSING USING CN N
• Examples of Using the Google Lens
Figure 6.28 Two Examples of Using the Google Lens, a Service Based on
Convolutional Deep Networks for Image Recognition.

Source: ©2018 Google L LC, used with permission. Google and the Google logo are registered
trademarks of Google L LC.
TEXT PROCESSING USING CNN
• Google word2vec project
• Word embeddings
• Typical Vector Representation of Word Embeddings in a Two-Dimensional
Space
TEXT PROCESSING USING CNN
• CNN Architecture for Relation Extraction Task in Text Mining
RECURRENT NEURAL NETWORKS (RNN) &
LONG SHORT-TERM MEMORY (LSTM)

• RNN designed to process sequential inputs


• Typical recurrent unit
RECURRENT NEURAL NETWORKS (RNN) &
LONG SHORT-TERM MEMORY (LSTM)
• LSTM is a variant of RNN
• In a dynamic network, the weights are called the long-term
memory while the feedbacks role is the short-term memory

Typical Long
Short-Term
Memory (L ST M)
Network
Architecture
RECURRENT NEURAL NETWORKS (RNN)
& LONG SHORT-TERM MEMORY (LSTM)
• LSTM Network Applications

Example Indicating
the Close-to-
Human
Performance of the
Google Neural
Machine Translator
(G N MT)
COMPUTER FRAMEWORKS FOR
IMPLEMENTATION OF DEEP LEARNING
• Torch (http://www.torch.ch)
• ML with GPU

• Caffe (caffe.berkeleyvision.org)
• Facebook’s improved version (www.caffe2.ai)

• TensorFlow (www.tensorflow.org)
• Google - Tensor Processing Units (TPUs)

• Theano (deeplearning.net/software/theano)
• Deep Learning Group at the University of Montreal

• Keras (keras.io)
• Application Programming Interface
COGNITIVE COMPUTING
• Systems that use mathematical models to emulate (or partially simulate)
the human cognition process to find solutions to complex problems and
situations where the potential answers can be imprecise
• IBM Watson on Jeopardy!
• How does cognitive computing work?

• Adaptive • Data mining,


• Interactive • Pattern recognition,
• Iterative and stateful • Deep learning, and
• Contextual
• NLP
• Mimic the way the
human brain works
CONCEPTUAL FRAMEWORK FOR COGNITIVE
COMPUTING AND ITS PROMISES
COGNITIVE COMPUTING
• How does cognitive computing differ from AI?
Table 6.3 Cognitive Computing versus Artificial Intelligence (AI).
Characteristic Cognitive Computing Artificial Intelligence (AI)

Technologies used • Machine learning • Machine learning


• Natural language processing • Natural language processing
• Neural networks • Neural networks
• Deep learning • Deep learning
• Text mining
• Sentiment analysis

Capabilities offered Simulate human thought processes to assist Find hidden patterns in a variety of data
humans in finding solutions to complex sources to identify problems and provide
problems potential Solutions

Purpose Augment human capability Automate complex processes by acting like a


human in certain Situations

Industries Customer service, marketing, healthcare, Manufacturing, finance, healthcare, banking,


entertainment, service Sector securities, retail, government
COGNITIVE COMPUTING
• Cognitive computing use cases
• Development of smart and adaptive search engines
• Effective use of natural language processing
• Speech recognition
• Language translation
• Context-based sentiment analysis
• Face recognition and facial emotion detection
• Risk assessment and mitigation
• Fraud detection and mitigation
• Behavioral assessment and recommendations, …
• Cognitive analytics?
COGNITIVE SEARCH
• Can handle a variety of data types
• Can contextualize the search space
• Employ advanced AI technologies.
• Enable developers to build enterprise-specific search
applications
END OF LECTURE 4
• Questions / Comments
• Reading Assignment: TECHNOLOGY INSIGHT 6.1 Elements Of An
Artificial Neural Network. Page 330-336

You might also like