Scribd
Upload
36 views18 pages

Lec 4

This document discusses activation functions in neural networks. It outlines 5 basic types of activation functions: 1) linear, 2) hard limiter, 3) saturating linear, 4) sigmoid, and 5) hyperbolic tangent. It provides examples of the output for each type of activation function. The document also includes two examples, one calculating the output of a neuron given input values and weights, and another solving for the threshold value that produces a given sigmoid output.

Uploaded by

mohammed.riad.bi.2020
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
36 views18 pages

Lec 4

This document discusses activation functions in neural networks. It outlines 5 basic types of activation functions: 1) linear, 2) hard limiter, 3) saturating linear, 4) sigmoid, and 5) hyperbolic tangent. It provides examples of the output for each type of activation function. The document also includes two examples, one calculating the output of a neuron given input values and weights, and another solving for the threshold value that produces a given sigmoid output.

Uploaded by

mohammed.riad.bi.2020
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 18

Part 3 :

Neural
Networks (NN)
Lecture 4
Presented by
Zainab Al-Qudsi
Objectives

❑Basic Activation Functions

❑Alternate nonlinear model of an ANN

❑Examples

2
2.5 Basic Activation Functions

The activation function (Sometimes called a


transfers function) shown in figure below can be a
linear or nonlinear function. There are many
different types of activation functions. Selection of
one type over another depends on the particular
problem that the neuron (or neural network) is to
solve. The most common types of activation
function are:-

3
Alternate nonlinear model of an
ANN

4
2.5 Basic Activation Functions

1- The first type is the linear (or identity) function.

5
2.5 Basic Activation Functions

2-The second type of activation function is a


hard limiter; this is a binary (or bipolar)
function that hard-limits the input to the
function to either a 0 or a 1 for the binary
type, and a -1 or 1 for the bipolar type. The
binary hard limiter is sometimes called the
threshold function, and the bipolar hard
limiter is referred to as the symmetric hard
limiter.
6
2.5 Basic Activation Functions

a- The o/p of the binary hard limiter:-

7
2.5 Basic Activation Functions

b-The o/p for the symmetric hard limiter


(shl):-

double side ‫ﺗﺳﻣﻰ اﯾﺿﺎ‬

8
2.5 Basic Activation Functions

3-The third type of basic activation function is


the saturating linear function or
threshold logic Unite (tLu) .
This type of function can have either a binary
or bipolar range for the saturation limits of
the output. The bipolar saturating linear
function will be referred to as the symmetric
saturating linear function.

9
2.5 Basic Activation Functions

a- The o/p for the saturating linear function


(binary o/p):-

10
2.5 Basic Activation Functions

11
2.5 Basic Activation Functions

4-The fourth type is sigmoid. Modern NN's use the


sigmoid nonlinearity which is also known as
logistic, semi linear, or squashing function.
‫ وﺑﮭﺎ ﻣروﻧﺔ‬1 ‫ و‬0 ‫ﻣﺣﺻورة ﺑﯾن‬

12
2.5 Basic Activation Functions

5-Hyperbollc tangent function is similar to sigmoid


in shape but symmetric about the origin. (tan h)

13
Ex.1

14
Ex.1

15
Ex2
Ex2:- (H.W) Find y for the following neuron if
x1 = 0.5, x2 = 1, x3 = -0.7
w1 = 0, w2 = -0.3, w3 = 0.6
Ɵ =1

16
Ex.3
Ex.3 The output of a simulated neural using a sigmoid
function is 0.5 find the value of threshold when the input x1
= 1, x2 = 1.5, x3 = 2.5. and have initial weights value = 0.2.

17
Ex.3

18

You might also like