CP4252 ML Unit - V
CP4252 ML Unit - V
Basic Structure
Component Description
Input Layer Takes in raw features
Where:
• Xi : input
• Wi : weight
• b : bias
• ϕ : activation function
Activation Functions
Function Formula Use Case
Sigmoid Binary classification
Loss Functions
Task Type Loss Function
Regression Mean Squared Error (MSE)
Type Application
Disadvantages
• Requires large amount of data
• Computationally expensive
• Prone to overfitting if not regularized
Real-World Applications
• Image and speech recognition
• Natural language processing (NLP)
• Medical diagnosis
• Self-driving cars
• Fraud detection
2]. Biological Motivation for Neural Networks
Axon: Sends signal to next neuron Output: y=ϕ(z) ,an activation function
While Machine Learning (ML) is powerful, it has several limitations that affect
its reliability, scalability, and applicability in real-world scenarios.
3. Interpretability / Explainability
• Many ML models (e.g., neural networks, ensemble methods) are black
boxes.
• Hard to explain why a model gave a specific output.
• A challenge in critical applications like healthcare and law.
4. Generalization Problems
• A model trained on a specific dataset may:
o Overfit: Too specific to training data
o Underfit: Too simplistic to capture patterns
• Can struggle to generalize to new/unseen data.
Basic Architecture
• Input Layer: Raw data input
• Hidden Layers (multiple): Feature extraction
• Output Layer: Prediction (class, value, etc.)
Each layer is made of neurons (units), with weights and activation functions.
Advantages
• Best performance on unstructured data (images, text, audio)
• Learns complex patterns automatically
• Highly scalable with GPUs
Limitations
• Needs huge data and computing power
• Often a black-box
• Requires tuning many hyperparameters
5]. Convolutional Neural Network
• It's a type of artificial neural network that is particularly good at analyzing
visual imagery. In simpler terms, it's a deep learning algorithm specifically
designed to process and understand images.
6]. Recurrent Neural Networks (RNN)
• Recurrent Neural Networks (RNNs) are a class of artificial neural networks
where connections between nodes can create cycles, allowing the output
from previous steps to be used as input for the current step. This structure
enables RNNs to maintain a form of memory of previous inputs, making
them particularly effective for sequential data tasks like time series
prediction, speech recognition, and Natural Language Processing (NLP).
Hidden State Update
• ht: The hidden state at the current time step t. It's like the RNN's memory
of the sequence up to this point.
• xt: The current piece of input data (e.g., a word or character).
• ht-1: The hidden state from the previous time step t-1. This is the memory
from before.
• Wh: A weight matrix that helps combine the current input xt and the
previous hidden state ht-1.
• bh: A bias term that adds a bit of adjustment to the calculation.
• activation: An activation function (like tanh or ReLU) that processes the
combined input.
Output Calculation
Backward pass:
Gradient Descent
n is the learning rate, LOW is the gradient of the loss with respect to the weight
W, and Wnew is the updated weight.
7]. Common Use Cases of RNNs
RNNs capture context and word order — vital for understanding language.