Student Booklet 2024 Semester 2 - Computing V5
Student Booklet 2024 Semester 2 - Computing V5
Semester 2 - 2024
NAME:________________________________________
1
Getting Started _________________________________________________________________ 4
Setting up Visual Studio Code___________________________________________________________ 4
Installing Packages ___________________________________________________________________________ 4
Creating New Workspaces _____________________________________________________________________ 4
2
Lists ______________________________________________________________________________ 23
Tuples ____________________________________________________________________________ 25
Sets ______________________________________________________________________________ 26
Dictionaries ________________________________________________________________________ 27
Challenges 3 ___________________________________________________________________ 29
Student Grade Management System ____________________________________________________ 29
Class/Objects __________________________________________________________________ 30
Defining and Accessing Variables _______________________________________________________ 30
Default Functions ___________________________________________________________________ 30
Defining New Functions ______________________________________________________________ 31
Scope _____________________________________________________________________________ 32
Challenges 4 ___________________________________________________________________ 33
Library Management System __________________________________________________________ 33
Machine Learning ______________________________________________________________ 34
Basic Concepts _____________________________________________________________________ 34
Styles of Learning ___________________________________________________________________ 34
Supervised ________________________________________________________________________________ 35
Unsupervised ______________________________________________________________________________ 35
Reinforcement _____________________________________________________________________________ 35
3
Getting Started
Setting up Visual Studio Code
Visual Studio Code is a power, platform independent, programming environment
capable of containing entire projects in a large variety of programming languages.
VSCode can be downloaded from: Download Visual Studio Code - Mac, Linux,
Windows
Installing Packages
VSCode is also capable of automatically finding and downloading all the Python
libraries required for our work.
To do so, we access the powershell in VSCode and use the following command: pip
install “library name”. The powershell is located at the bottom of the screen. Here we
see the command for installing the numpy library, a very useful library for statistics.
Inside the folder you just made, create a new folder called
“Python Fundamentals”. This folder will contain your Python
files for the first couple of weeks of the course.
Inside your “Python Fundamentals” folder, create a new file called “comments.py”.
4
Python Fundamentals
Comments
Comments are a fundamental part of programming. Even simple programs need to be
commented to allow other users to be able to understand and read your code
effectively. It becomes especially important when you’re working in teams, both
locally and internationally.
Comments should be succinct, but detailed enough to tell the reader what your code
is doing. If you’re writing more than a few lines of comments for a single function, that
is probably too much. Larger comment blocks are reserved for explaining whole
programs, where shorter (1 – 2 sentences) are used for explaining functions or
complicated sections of your program.
Copy the following code into the file you created (comments.py) in the previous
section.
If you run the above code you will get the following output:
This is indicating there is a syntax error on line 6. VSCode is very good at identifying
errors and where they occur. Pay close attention to the error messages to help fix
broken code.
5
Syntax
Syntax is the structure of a language, just in the same way English, Japanese, Spanish,
etc all have a way to structure sentences to communicate, Python has a structure to
allow you to communicate with the computer. The syntax of Python will be revealed
the more you use it. How functions relate to objects and classes, how variables are
accessed with different scopes will be clearer when covered later in the booklet.
Create a new file in the Python Fundamentals folder and call it “syntax.py”, copy the
following code into it.
The above code will be explained in more detail in the Control Structures section of
this booklet.
VSCode will show you where the first error it encounters is located. Here it is
indicating an error on line 17 to say there is an indentation error with the specific
message of “after ‘if’ statement on line 16”.
6
Data Types
Data types are how Python distinguishes between things likes numbers, text, and the
different kinds of objects we’ll use. Each has their own rules that can be applied. The
function “type()” can be used to show to type of the data in question.
Create a new file called “DataTypes.py” and copy the following code:
7
Variables
Like Mathematics, variables in coding are used to reference a value. However, in
coding, these values can be small like individual numbers or larger and more
complicated and large things like DataFrames and even entire Neural Networks.
Note that the first time we initialised the variable ‘number’ it had a string value. It was
overwritten on line 7 to have an integer value.
We also overwrote the value of ‘number’ after the first print statement to have the
new integer value of 10.
8
Operators
Operators in Python are very similar to those used in mathematics, with a few
exceptions on how they’re written. A comprehensive list of operators can be found at:
Python Operators
Common Operators
Common Operators that are used in the same way as mathematics.
Operator Name Example
+ Addition x+y
- Subtraction x-y
* Multiplication x*y
/ Division x/y
!= Not equal x != y
>= / <= Less than or equal / greater than or equal x >= y or x <= y
9
Create a new file in the Python Fundamentals folder called “operators.py” and copy
the following code:
10
Control Structures
Conditionals
Python operates conditional statements with True and False conditions that result in
making the program flow in a particular way. We can examine the statement: I am
thirsty. If this is True, we should go and get a drink of water. If this is False, we don’t
need to do anything.
Greater than
Less than a<b a >= b
or equal
11
There are two basic ways to use conditionals, as an if-else or if-elif-else structure.
Above is an example of the if-else structure. Note the vertical lines under the “if” and
the “else” showing you the blocks of code that will get executed depending on the
conditional result of the question: is number greater than or equal to 6? If this is a
True statement, it will execute the top block. If the condition is false, it will execute
the bottom block.
We can also have an if-elif-else structure where the “elif” is abbreviated from “else if”
number = 5
number is
if number < 9 else less than or
equal to 4
number is number is
between 4 greater than
and 9 or equal to 9
12
Pros and Cons of Nested vs Linear Conditional Statements
Pros of Using Nested Conditionals
Hierarchical Decision-Making:
Nested conditionals allow for a more structured approach to decision-making,
enabling you to organize complex logic hierarchically. This is useful when certain
conditions only make sense to check after another condition has been satisfied.
Reduced Redundancy:
Nested conditionals can help reduce redundancy by avoiding repeated checks of the
same condition. Once a condition is met, subsequent nested conditions can assume
that the earlier conditions are true, streamlining the code.
Increased Complexity:
Nested conditionals can lead to increased complexity, making it difficult to debug
and test the code. The flow of execution can be harder to trace, and understanding
the full set of conditions under which certain parts of the code execute can be
challenging.
13
Loops
As the name suggests, a loop is a block of code that will repeat, either indefinitely or
until a particular condition is met. Python has two kinds of loops, each can be used to
perform the same tasks, but one or the other is usually better suited to the style of
loop.
While Loops
A while loop is designed to loop while a particular condition is true. This can be used
to look indefinitely or for an unknown number of times. This adds flexibility into the
loop to continue, even when you don’t have a finite end.
Create a file called “WhileOne.py” and copy the code below: do not run this code, it
is an infinite loop
14
For Loops
A for loop is designed to loop a known number of iterations. The known number of
iterations can actually be unknown, but a predetermined amount. An example of this
can be seen when counting the number of vowels in a sentence. The known number
of iterations is the number of letters in the sentence, but because we might not know
how big the sentence is, it is technically unknown.
Flowcharts
These are the basics of how the loops look, very similar, but the key difference is in
how the exit from the loop is handled.
15
User Input
Getting user input into a program is one way to make it more interactive. The simplest
way to get user input is through the terminal and having the user type in a response
or information. It is by no means the only way to get user input, for example, we could
ask the user to select a file to be read by the program (more on this later).
Standard user input is a String data type, but sometimes we want the user to input a
number. To convert the string input to a number we need to type cast it to an integer
or float as appropriate. The two examples below show the syntax for casting. If we
want to cast the variable it an integer, we use int(*).
Getting user input is another function like print(*) or int(*). The function is input(*)
where * is a string, telling the user what input you would like. The example below is
asking the user to type in their name and will store that text input into the variable
‘name’.
If the user types in a number in response to this, Python will attempt to store the
number as a string. This is because the input has not been type cast to an integer.
Create a new file “UserInput.py” and copy the following code:
16
Running this code you should get the following output (replacing your name with
mine):
Random
The Random library is the most common way to access a level of randomness in our
programs. Randomness is a vital component in understanding the future work of
Neural Networks and Genetic Algorithms.
The Random library is not included in Python's base installation and must be imported
using the ‘import’ keyword. If it's not installed, you need to install it via the terminal
before importing. You can add an alias, typically an abbreviation of the library's name,
to simplify code usage. While not required, an alias can make the code easier to read
and write.
Create a new file called “RandomExample.py” and copy the code below:
You should get out like this, but with different numbers:
17
Code Challenges 1
• Create another folder called “Code Challenges”.
• Where you see Last Name, replace it with your last name.
• Create a file called “Last Name CC1.py” – this is the file you’ll submit as
part of the assessment.
• You will write all the challenges into this one file.
• Have a couple of lines of spare space between each code challenge.
Challenge 1
• Write a comment to indicate the start of the first code challenge.
• Write a program that creates three variables named ‘word’, ‘number’, and
‘decimal’.
• Give each variable a value that is appropriate for the variable name.
• Print the type of each variable.
Challenge 2
• Write a comment to indicate the start of the second code challenge.
• Re-assign the variable called ‘number’ the value of 10.
• Use the appropriate assignment operator to add 6 to the variable.
• Print the result.
• Use the appropriate assignment operator to divide the variable by 2.
• Print the result.
Challenge 3
• Write a comment to indicate the start of the third code challenge.
• Write a if-elif-else block to determine if the variable ‘numberTwo’ is less
than or equal to 4, greater than 4 but less than 10, or greater then or equal
to 10.
Challenge 4
• Write a comment to indicate the start of the fourth code challenge.
• Print ‘number’
• Write a while loop that checks if ‘number’ is greater than 0.
• Inside the loop, use an assignment operator to subtract 1 from number.
• Print ‘number’ after the subtraction.
Challenge 5
• Write a for loop to iterate 5 times.
• The loop variable should start at 1.
• Create a variable called ‘result’ which holds the result of the multiplying 2
and the loop variable.
• Print the ‘result’ variable inside the loop
18
Code Challenges 2
Create a file called “Last Name – game name.py” – this is the file you’ll submit as
part of the assessment.
19
Functions
Create a new folder called “Functions”.
A function in Python is a reusable block of code that performs a specific task, defined
with the def keyword, which can accept inputs (arguments), execute a sequence of
statements, and optionally return a value. Functions help in organizing code, making
it more modular, readable, and easier to maintain.
Every function in Python operates in the same way. It will have a descriptive name
giving you an idea of what it does and a possible set of inputs that goes inside the
brackets.
Built-in
The following is a list of ten very commonly used functions within Python:
• * print() - Outputs text or other data to the console.
• * input() - Reads a line of input from the user.
• len() - Returns the number of items in an object, such as a list or string.
• * type() - Returns the type of an object.
• * int() - Converts a value to an integer.
• * float() - Converts a value to a floating-point number.
• str() - Converts a value to a string.
• list() - Creates a list.
• dict() - Creates a dictionary.
• * range() - Generates a sequence of numbers, often used in loops.
20
Inputs/Arguments and Outputs/returns
Functions are often given variable(s) as input(s) and then act on them to perform a
operations likely to be repeated many times. The variable(s) may be modified and then
returned to the user or the function may produce an output based on the inputs.
Below is an example of using the str() function to convert the integer value of 5 to the
string representation of “5”
Function
• int, 5 • string,
• str(int) "5"
Input(s) output(s)
When a function returns a value, we can assign the output of that function to a
variable, or the same one. Functions can have no input/outputs, or they could have
many.
Defining New
The power of functions comes from being able to define your own to perform the
repeatable instruction for your project. To define a new function, we need to use the
def keyword. This is short for define. When defining new functions, the name of the
function should allude to its purpose. There should also be comments in the code to
tell other users what the function is intended to do in terms of input(s) and output(s).
When writing a function that accepts arguments, the names of the arguments are
giving in the definition line of the function. These are variables to be used within the
function and do not need to be defined earlier in the program.
The above is an example of defining a new function called “isEven” that has a single
input. Inside the function we can refer to number as the input variable as it is being
defined within the function definition.
Functions can also be used inside of other functions, further simplifying code in the
main parts of a program.
21
Create a new file called “DefiningNew.py” and copy the code below:
# this file will demonstrate how to create new functions
# this function will take an integer input and print the numbers up to and
including the number
# it will return no value
def printUpTo(number):
for n in range(number):
print(n+1) # 1 is added so that we start printing from the value 1 as
the loop variable 'n' will start at 0.
printUpTo(5)
# this function will check if an input number is even and return the value of
True if the input is even
def isEven(number):
if number % 2 == 0:
return True
else:
return False
print(isEven(6))
print(isEven(7))
printEvens(10)
Lists
Lists in Python are versatile data structures used to store ordered collections of items.
They are mutable, allowing for dynamic modification such as adding, removing, or
changing elements. Lists can contain different data types, making them ideal for tasks
that require flexible data handling, such as storing sequences, iterating over data, and
managing collections that need frequent updates. Their ordered nature ensures that
elements are maintained in the sequence they were added, which is useful for
operations that depend on element positioning.
More information
Socratica - Lists
Python - List Methods (w3schools.com)
Create a new file called “ListBasics.py” and copy the following code.
# this file will demonstrate the creation of lists, indexing, and looping
print("\nloop example")
for fruit in fruits: # each iteration of 'fruit' in 'fruits' gets the next fruit in
the list
print(fruit)
print("\nloop with index")
for i in range(len(fruits)): # does the same, but uses the index of the fruits list
directly instead
23
print(fruits[i])
print("\nindexing examples")
print("The first item in the list at index 0:",fruits[0]) # will print the first
item in the list (apple)
print("The last item in the list at index 4:",fruits[4]) # will print the 5th item
(blue berries)
print("The last item in the list at index -1:",fruits[-1]) # the last item in the
list can also be accessed using an index of -1 instead
Create a new file called “ManipulatingLists.py” and copy the following code.
# this file will demonstrate some of the common ways to manipulate lists
24
print(fruits)
fruits.pop(5) # remove the item at index 5
print(fruits)
fruits.clear() # clear the entire list
print(fruits)
Run the code and take note of how each list method changes the original list. Pay close
attention to how the indexing on the list happens.
Tuples
Tuples in Python are used to store ordered collections of items, similar to lists, but
they are immutable, meaning their elements cannot be changed after creation. This
immutability makes tuples useful for storing data that should not be altered, ensuring
data integrity. They can hold different types of data and maintain the order of
elements. Tuples are often used for fixed data collections and can be used as keys in
dictionaries due to their hashable nature.
Socratica - Tuples
Python - Tuple Methods (w3schools.com)
Create a new file called “Tuples.py” and copy the following code.
# this file will demonstrate the creation and use of tuples
fruitsTuple = ("apple", "pear", "banana", "apple", "pear") # tuples can still have
duplicate items
print(fruitsTuple)
# to change items in a tuple (or other list related changes), you must convert it
to a list first, then convert back to a tuple
fruitsList = list(fruitsTuple)
25
fruitsList[2] = "grape"
fruitsTuple = tuple(fruitsList)
print(fruitsTuple)
Sets
Sets in Python are used to store unordered collections of unique elements. They
automatically remove duplicates and provide efficient methods for common set
operations like union, intersection, and difference. Sets are useful for tasks that
involve eliminating duplicate entries, membership testing, and mathematical
operations on collections. Their unordered nature makes them ideal for situations
where the order of elements is not important.
Socratica - Sets
Python - Set Methods (w3schools.com)
Create a new file called “Sets.py” and copy the following code.
# this file will demonstrate how to use sets and some of the basic functions
#Unordered means that the items in a set do not have a defined order.
#Set items can appear in a different order every time you use them, and cannot be
referred to by index or key. You can execute this program multiple times to see
that it can print differently each time.
fruitsSet = {"apple", "pear", "strawberry", "apple", "watermelon"}
print(fruitsSet) # note that the dupliocate of "apple" is not printed as it isn't
stored.
26
# We can loop through a set in the same way as a list, but cannot do it by index
for fruit in fruitsSet:
print(fruit)
Dictionaries
Dictionaries in Python store key-value pairs, allowing for efficient data retrieval by key.
They are ideal for situations where you need to associate unique keys with specific
values, like storing and accessing user information, configurations, or any dataset
where quick lookups are required. Dictionaries support fast insertion, deletion, and
retrieval operations, making them a versatile and powerful tool for managing and
organizing data. They do not allow duplicate keys. Dictionaries are changeable, allow
the adding, removing, and modifying of entries.
Socratica - Dictionaries
Python - Dictionary Methods (w3schools.com)
27
Create a new file called “Dictionaries.py” and copy the following code.
# this file will demonstrate the use of dictionaries and some of their functions
print(thisdict)
print(thisdict["brand"]) # access the 'brand' key and print its value pair
thisdict = {
"brand": "Toyota",
"model": "Tarago",
"year": 1986,
"year": 2005 # duplicate keys will override the previous key
}
print(thisdict)
# extracting the keys from a dictionary can be helpful in knowing what is available
in the dictionary
keys = thisdict.keys()
print(keys)
# we can add key:value pairs to the dictionary in a similar way to getting them.
# rather than selecting a key already present, we give a new key and assign it a
given value
thisdict["colour"] = "white"
print(thisdict)
# the same effect can be had with update({"":""})
# thisdict.update({"colour":"black"})
# changing values of keys in done in a similar way to changing the value of an item
in a list
28
# rather than giving an index, we give the key
thisdict["year"] = 2015
print(thisdict)
# looping through the various keys can be done just like a set or tuple
for x in thisdict:
print(x) # print the keys
print(thisdict[x]) # print the value of the key
# we can loop through each item and print both at the same time
for x, y in thisdict.items():
print(x, y)
Code Challenge 3
Student Grade Management System
Create a file called “Last Name - StudentGradeManagementSystem.py”
29
• Display the average grade, highest grade, and lowest grade.
Class/Objects
In Python, an object is an instance of a class, which serves as a blueprint for creating
objects. Objects encapsulate data (attributes) and behaviour (methods) relevant to
their class. They are created through class instantiation and can have both attributes
and methods. Additionally, objects support inheritance, allowing for the creation of a
hierarchical class structure, ie. a ‘chair’ object can inherit attributes from its parent
class, ‘furniture’. Encapsulation is a key concept, promoting the bundling of data and
methods and ensuring the protection of object integrity.
Create a new folder in the “2024 Semester 2 – Computing” folder called “Classes and
Objects”
Variables (object attributes) are accessed by calling the instance of the class, the
object, and then the name of the variable. Object attributes are normally accessed
and modified in this way.
Create a new file called “ClassBasics.py” and copy the following code.
class myClass:
number = 10
text = "hello world!"
Default Functions
Classes have two default functions:
• This is the constructor function which defines any starting
values for attributes when the object in initialised.
__init__(self, *):
• If this is not defined, the constructor for the object is seen
the same as above.
30
• We override this function when we want to instantiate an
object with predetermined values.
• This function is called when the object is referenced
inside a print() function.
• If not defined it will print the location in memory where
__str__(self):
he object is located.
• We override this function to have it print custom
information about the object.
# __str__ is a default function that we are overriding to say what we want when
we print the class
def __str__(self):
return f"The {self.style} chair has {self.legs} legs and is made of
{self.material}"
When defining new functions inside the class, we do so in the same way we did
previously and when overriding the default functions:
# __str__ is a default function that we are overriding to say what we want when
we print the class
def __str__(self):
output = f"The chair has the following features: {self.style}, {self.legs}
legs, {self.material}"
# if there are added features, add them to the output string
if(len(self.additionalFeatures)>0):
for i in range(len(self.additionalFeatures)):
output += ", "
output += str(self.additionalFeatures[i])
return output
print("\ncreate another chair with 3 attributes and then add the 'reclining'
feature")
chair = myChair("lounger", 4, "wood")
chair.add_feature("reclining")
chair.add_feature("foot rest")
chair.add_feature("cup holder")
print(chair)
# attributes of a class can be modified by calling the class and the attribute in a
similar way to using a function
print("\nchanging the material to 'steel'")
chair.material = "steel"
print(chair)
Scope
Scope describes the accessibility of variables and object parameters. Object
parameters can only be accessed by calling the object first. Scope allows the
abstraction of attributes and aids in code modularity. A global variable is one defined
32
outside of any object or function and should not be directly referenced inside an
object or function.
Code Challenge 4
Library Management System
Create a file called “Last Name - LibraryManagementSystem.py”
33
Machine Learning
The Concept
Machine learning is a branch of artificial intelligence that enables computers to learn
from data and improve their performance over time without being explicitly
programmed. By analysing patterns in data, machine learning algorithms can make
predictions, classify information, recognize patterns, and even make decisions. This
process involves training a model on a dataset and then using that model to make
predictions or decisions based on new data. Machine learning is widely used in various
applications, including recommendation systems, image recognition, natural language
processing, and autonomous vehicles.
Styles of Learning
https://www.slideshare.net/slideshow/big-data-and-machine-learning-for-businesses/75413684#12
34
Supervised
Supervised learning is a type of machine learning where the model is trained on a
labelled dataset. In this approach, the input data is paired with the correct output,
and the model learns to map inputs to outputs by identifying patterns in the data. The
goal is for the model to accurately predict the output for new, unseen data. Supervised
learning is commonly used for tasks like classification, where the model categorizes
data into predefined classes, and regression, where the model predicts continuous
values. Examples include spam detection in emails and predicting house prices based
on features like size and location.
Unsupervised
Unsupervised learning is a type of machine learning where the model is trained on
data without labelled outputs. The goal is to discover hidden patterns, structures, or
relationships within the data. Unlike supervised learning, the model doesn't know the
correct answer ahead of time; instead, it explores the data to find patterns on its own.
Common techniques in unsupervised learning include clustering, where data points
are grouped based on similarity, and dimensionality reduction, where the data is
simplified while retaining important information. Examples include customer
segmentation and anomaly detection in network security.
Reinforcement
Reinforcement learning is a type of machine learning where an agent learns to make
decisions by interacting with an environment. The agent takes actions to achieve a
goal, and based on the outcomes of these actions, it receives rewards or penalties.
The goal of the agent is to maximize its cumulative reward over time by learning the
best strategy or policy. Unlike supervised learning, where the correct answers are
provided, reinforcement learning involves trial and error, with the agent learning from
its experiences. This approach is used in applications like robotics, game playing, and
autonomous systems, where decision-making is crucial.
35
Neural Networks
A neural network is a computational model inspired by the human brain's structure
and function. It consists of interconnected layers of nodes, called neurons, which
process and transmit information. A neural network typically has an input layer, one
or more hidden layers, and an output layer. Each neuron receives input, applies a
mathematical function to it, and passes the output to the next layer of neurons. Neural
networks are particularly effective at learning complex patterns and relationships in
data, making them essential in tasks like image recognition, natural language
processing, and speech recognition. They form the foundation of many modern
machine learning techniques, including deep learning.
# Define the sigmoid function, which will be used as the activation function
# Sigmoid function maps any real value to a value between 0 and 1
def sigmoid(x):
return 1 / (1 + np.exp(-x))
36
# Training data: Input values for training the neural network
training_inputs = np.array([[0, 0, 0, 1], # Example 1: [0, 0, 0, 1]
[0, 1, 1, 1], # Example 2: [0, 1, 1, 1]
[1, 1, 0, 1], # Example 3: [1, 1, 0, 1]
[1, 0, 1, 1]]) # Example 4: [1, 0, 1, 1]
# Training process: Iterate over the training data 10,000 times to adjust weights
for i in range(10000):
# Forward propagation: Calculate the predicted output using the current
synaptic weights
input_layer = training_inputs # Input layer is the training data
outputs = sigmoid(np.dot(input_layer, synaptic_weights)) # Calculate output
using sigmoid activation
# Calculate the error (difference between the expected and predicted outputs)
err = training_outputs - outputs
# Calculate adjustments to the weights using the error and the sigmoid
derivative
# The sigmoid derivative helps to adjust weights in the right direction
adjustments = np.dot(input_layer.T, err * outputs * (1 - outputs))
# Print the output from the training data after the final iteration
print("Result")
print(outputs)
# Test the neural network with a new input to see how well it generalizes
print("Results from new input")
new_input = np.array([[1, 1, 0, 1]]) # New input to test the network
print(sigmoid(np.dot(new_input, synaptic_weights))) # Calculate and print the
output for the new input
37
Forwards/Backwards Propagation
Forward Propagation: In forward propagation, the input data is passed through the
neural network layer by layer, from the input layer to the output layer. Each neuron
processes the input it receives by applying a weight, adding a bias, and passing the
result through an activation function to produce an output. This output is then passed
on to the next layer. The final output of the network is compared to the actual target
values, and the difference (error) is calculated.
In the code example we wrote we can identify the forwards and backwards
propagation sections.
• The forwards propagation occurs as the dot product of the input_layer and the
current synaptic_weights is calculated. The result is passed through the
activation function to determine the final outputs. It is the calculation of the of
the networks outputs based on the current weights in the hidden layer (synaptic
weights).
• The backpropagation occurs as the error is calculated and the synaptic weights
are adjusted. It is the process of updating the weights based on the error to
improve the networks performance (the machine learning what to do).
Activation Function
An activation function in a neural network determines whether a neuron should be
activated or not. It introduces non-linearity into the model, allowing the network to
learn and perform more complex tasks. The sigmoid function is a common example of
an activation function. You aren’t required to understand how the mathematics of the
sigmoid function works, but to understand how it works as an activation function. The
sigmoid is not the only activation function available for use.
Sigmoid Function
The sigmoid function maps any real-valued number into a value between 0 and 1. It is
defined as:
1
𝜎𝜎(𝑥𝑥 ) =
1+𝑒𝑒 −𝑥𝑥
38
Figure 1: The Sigmoid Function
How it Works
Input Transformation:
The sigmoid function takes a real-valued input and transforms it into a value between
0 and 1. This is particularly useful for binary classification, where the output can be
interpreted as a probability.
Output Interpretation:
A value close to 1 indicates a high probability that the neuron should be activated,
while a value close to 0 suggests that it should not be activated.
Importance
Non-Linearity:
By introducing non-linearity, the sigmoid function allows neural networks to model
more complex relationships between input and output.
Smooth Gradient:
The sigmoid function has a smooth derivative, which makes it suitable for use in
backpropagation, allowing the network to update weights in a way that minimizes
error.
Limitations
Vanishing Gradient:
The sigmoid function can lead to the vanishing gradient problem during
backpropagation, especially in deep networks, where gradients become too small to
allow effective learning.
39
Genetic Algorithms (NEAT)
What is NEAT?
Analysing NEAT
41