0% found this document useful (0 votes)
48 views55 pages

Ai Class X Notes

The document provides an introduction to Artificial Intelligence (AI), defining various types of intelligence and outlining the characteristics and capabilities of AI systems. It explains the AI project cycle, which consists of stages such as problem scoping, data acquisition, data exploration, modeling, and evaluation, while also discussing different AI domains like data sciences, computer vision, and natural language processing. Additionally, it addresses ethical considerations, AI bias, data privacy, and the potential impact of AI on employment and society.

Uploaded by

Shikha Sharma
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
48 views55 pages

Ai Class X Notes

The document provides an introduction to Artificial Intelligence (AI), defining various types of intelligence and outlining the characteristics and capabilities of AI systems. It explains the AI project cycle, which consists of stages such as problem scoping, data acquisition, data exploration, modeling, and evaluation, while also discussing different AI domains like data sciences, computer vision, and natural language processing. Additionally, it addresses ethical considerations, AI bias, data privacy, and the potential impact of AI on employment and society.

Uploaded by

Shikha Sharma
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 55

Unit 1: Introduction to Artificial Intelligence

Intelligence

• Mathematical Logical Reasoning: A person's ability to


regulate, measure, and understand numerical symbols,
abstraction and logic.
• Linguistic Intelligence: Language processing skills both in
terms of understanding or implementation in writing or verbally.
• Spatial Visual Intelligence: It is defined as the ability to
perceive the visual world and the relationship of one object to
another.
• Kinaesthetic Intelligence: Ability that is related to how a
person uses his limbs in a skilled manner.
• Musical Intelligence: As the name suggests, this intelligence is
about a person's ability to recognize and create sounds, rhythms,
and sound patterns.
• Intrapersonal Intelligence: Describes how high the level of
self-awareness someone has is. Starting from realizing
weakness, strength, to his own feelings.
• Existential Intelligence: An additional category of intelligence
relating to religious and spiritual awareness.
• Naturalist Intelligence: An additional category of intelligence
relating to the ability to process the information on the
environment around us.
• Interpersonal intelligence: Interpersonal intelligence is the
ability to communicate with others by understanding other
people's feelings & influence on the person.

In other words, we may define intelligence as:

• Ability to interact with the real world


o To perceive, understand and act
▪ Example: Speech Recognition – Understanding and
synthesis
▪ Example: Image Recognition
▪ Example: Ability to take action: to have an effect
• Reasoning and planning
o Modelling the external world, given input
▪ Solving new problems, planning and making
decisions
▪ Ability to deal with unexpected problems,
uncertainties
• Learning and adaptation
o Continuous learning and adapting graph
▪ Our internal models are always being updated
▪ Example: Baby learning to categorize and recognise
animals

How do you make decisions?


The basis of decision-making depends upon the availability of
information and how we experience and understand it. For the
purposes of this article, ‘information’ includes our past experience,
intuition, knowledge, and self-awareness.
What is Artificial Intelligence?
When a machine possesses the ability to mimic human traits, i.e.,
make decisions, predict the future, learn and improve on its own, it is
said to have artificial intelligence.
How do machines become Artificially Intelligent?
Bachines also become intelligent once they are trained with some
information that helps them achieve their tasks. AI machines also
keep updating their knowledge to optimise their output.
NITI Aayog: AI refers to the ability of machines to perform cognitive
tasks like thinking, perceiving, learning, problem-solving and
decision-making.
World Economic Forum: Artificial intelligence (AI) is the software
engine that drives the Fourth Industrial Revolution. Its impact can
already be seen in homes, businesses and political processes. In its
embodied form of robots, it will soon be driving cars, stocking
warehouses and caring for the young and elderly.

• AI is a form of Intelligence; a type of technology and a field of


study.
• AI theory and the development of computer systems (both
machines and software) enable machines to perform tasks that
normally require human intelligence.
• Artificial Intelligence covers a broad range of domains and
applications and is expected to impact every field in the future.
Overall, its core idea is building machines and algorithms
capable of performing computational tasks that would otherwise
require human-like brain functions.

AI, ML & DL
Artificial Intelligence (AI): Refers to any technique that enables
computers to mimic human intelligence. It gives the ability to
machines to recognize a human’s face; to move and manipulate
objects, understand human voice commands, and do other tasks. The
AI-enabled machines think algorithmically and execute what they
have been asked for intelligently.
Machine Learning (ML): It is a subset of Artificial Intelligence that
enables machines to improve at tasks with experience (data). The
intention of Machine Learning is to enable machines to learn by
themselves using the provided data and make accurate Predictions/
Decisions.
Deep Learning (DL): It enables software to train itself to perform
tasks with vast amounts of data. In Deep Learning, the machine is
trained with huge amounts of data which helps it in training itself
around the data. Such machines are intelligent enough to develop
algorithms for themselves. Deep Learning is the most advanced form
of Artificial Intelligence out of these three.
AI Domains
With respect to the type of data fed in the AI model, AI models can be
broadly categorized into three domains:

• Data Sciences
• Computer Vision
• Natural Language Processing

Data Sciences
Data sciences is a domain of AI related to data systems and processes,
in which the system collects numerous data, maintains data sets and
derives meaning/sense out of them. The information extracted through
data science can be used to make a decision about it.

Computer Vision
Computer Vision, abbreviated as CV, is a domain of AI that depicts
the capability of a machine to get and analyse visual information and
afterwards predict some decisions about it. The entire process
involves image acquiring, screening, analysing, identifying and
extracting information. This extensive processing helps computers to
understand any visual content and act on it accordingly.

Natural Language Processing


Natural Language Processing, abbreviated as NLP, is a branch of
artificial intelligence that deals with the interaction between
computers and humans using natural language. Natural language
refers to language that is spoken and written by people, and natural
language processing (NLP) attempts to extract information from the
spoken and written word using algorithms.

• AI ethics is a system of moral principles and techniques


intended to inform the development and responsible use of
artificial intelligence technology.
• AI ethics is a set of guidelines that advise on the design and
outcomes of artificial intelligence. Human beings come with
all sorts of cognitive biases, such as recency and confirmation
bias, and those inherent biases are exhibited in our behaviours
and subsequently, our data.

AI Bias means favouring someone or something. AI bias focuses


upon training the machines with unbiased data, when Bias Data is fed
to an AI Machine while creating the Model then the machine will also
be biased. The quality of your model is usually a direct result of the
quality and quantity of your data.

Data privacy: Generally, means the ability of a person to determine


for themselves when, how, and to what extent personal information
about them is shared with or communicated to others. This personal
information can be one's name, location, contact information, or
online or real-world behaviour.

AI Access: Since Artificial Intelligence is still a budding technology,


not everyone has the opportunity to access it. The people who can
afford AI enabled devices make the most of it while others who
cannot are left behind. Because of this, a gap has emerged between
these two classes of people and it gets widened with the rapid
advancement of technology.

AI creates unemployment : AI is making people’s lives easier. Most


of things nowadays are done in just a few clicks. In no time AI will
manage to do all the laborious tasks we humans have been doing for a
long time. Maybe in the coming years, AI-enabled machines will
replace all the people who work as labourers. This may start an era of
mass unemployment where people having little or no skills may be
left without jobs and others who keep up with their skills according to
what is required, will flourish.

AI for kids: As we all can see, kids nowadays are smart enough to
understand technology from a very early age. As their thinking
capabilities increase, they start becoming techno-savvy and eventually
they learn everything more easily than an adult.
Unit 2: AI Project Cycle
AI Project Cycle
The AI Project Cycle provides us with an appropriate framework that
can lead us towards the goal. The AI Project Cycle mainly has 5
stages:

AI Project Cycle

• You need to acquire data that will become the base of your
project as it will help you in understanding the parameters that
are related to problem scoping.
• You go for data acquisition by collecting data from various
reliable and authentic sources. Since the data you collect would
be in large quantities, you can try to give it a visual image of
different types of representations like graphs, databases, flow
charts, maps, etc. This makes it easier for you to interpret the
patterns which your acquired data follows.
• After exploring the patterns, you can decide upon the type of
model you would build to achieve the goal. For this, you can
research online and select various models which give a suitable
output.
• You can test the selected models and figure out which is the
most efficient one.
• The most efficient model is now the base of your AI project and
you can develop your algorithm around it.
• Once the modelling is complete, you now need to test your
model on some newly fetched data. The results will help you in
evaluating your model and improve it.
• Finally, after evaluation, the project cycle is now complete and
what you get is your AI project.

Stage1: Problem Scoping


Scoping a problem is not that easy as we need to have a deeper
understanding around it so that the picture becomes clearer while we
are working to solve it. Hence, we use the 4Ws Problem Canvas to
help us out.

Who?
The “Who” block helps in analysing the people getting affected
directly or indirectly due to it. Under this, we find out who the
‘Stakeholders’ of this problem are and what we know about them.
Stakeholders are the people who face this problem and would be
benefitted from the solution.

What?
Under the “What” block, you need to look into what you have on
hand. At this stage, you need to determine the nature of the problem.
What is the problem and how do you know that it is a problem?
Under this block, you also gather evidence to prove that the problem
you have selected actually exists. Newspaper articles, Media,
announcements, etc are some examples.

Where?
Now that you know who is associated with the problem and what the
problem actually is; you need to focus on the
context/situation/location of the problem. This block will help you
look into the situation in which the problem arises, its context of it,
and the locations where it is prominent.
Why?
You have finally listed down all the major elements that affect the
problem directly. Now it is convenient to understand who the people
that would be benefitted by the solution are; what is to be solved; and
where will the solution be deployed.

Problem Statement
The Problem Statement Template helps us to summarise all the key
points into one single Template so that in the future, whenever there is
a need to look back at the basis of the problem, we can take a look at
the Problem Statement Template and understand the key elements of
it.

Stage 2: Data Acquisition


As the term clearly mentions, this stage is about acquiring data for the
project. Data can be a piece of information or facts and statistics
collected together for reference or analysis. Whenever we want an AI
project to be able to predict an output, we need to train it first using
data.
For any AI project to be efficient, the training data should be
authentic and relevant to the problem statement scoped.

Data Features
Data features refer to the type of data you want to collect. There can
be various ways in which you can collect data. Some of them are:

• Surveys
• Web Scraping
• Sensors
• Cameras
• Observations
• API (Application Program Interface)

Stage 3: Data Exploration


To analyse the data, you need to visualise it in some user-friendly
format so that you can:
• Quickly get a sense of the trends, relationships and patterns
contained within the data.
• Define the strategy for which model to use at a later stage.
• Communicate the same to others effectively. To visualise data,
we can use various types of visual representations.

Stage 4: Modelling
The ability to mathematically describe the relationship between
parameters is the heart of every AI model. Thus, whenever we talk
about developing AI models, it is the mathematical approach to
analyzing data that we refer to. Generally, AI models can be classified
as follows:

Rule Based Approach


It refers to the AI modelling where the rules are defined by the
developer. The machine follows the rules or instructions mentioned
by the developer and performs its task accordingly. For example, we
have a dataset that tells us about the conditions on the basis of which
we can decide if an elephant may be spotted or not while on safari.
The parameters are Outlook, Temperature, Humidity and Wind.
This is known as a rule-based approach because we fed the data
along with rules to the machine and the machine after getting trained
on them is now able to predict answers for the same.
A drawback/feature of the rule-based approach is that the learning
is static. The machine once trained, does not take into consideration
any changes made in the original training dataset. That is, if you try
testing the machine on a dataset that is different from the rules and
data you fed it at the training stage, the machine will fail and will not
learn from its mistake. Once trained, the model cannot improvise
itself on the basis of feedback.
Learning Based Approach
It refers to AI modelling where the machine learns by itself. Under the
Learning Based approach, the AI model gets trained on the data fed to
it and then is able to design a model which is adaptive to the change
in data.
If the machine learning based model is trained with X type of data
and the machine designs the algorithm around it, the model would
modify itself according to the changes which occur in the data so that
all the exceptions are handled in this case.

The learning-based approach can further be divided into three


parts:

• Supervised Learning
• Unsupervised Learning
• Reinforcement Learning

Supervised Learning
In a supervised learning model, the dataset which is fed to the
machine is labelled. In other words, we can say that the dataset is
known to the person who is training the machine only then he/she is
able to label the data.

There are two types of Supervised Learning models:


1. Classification:
Where the data is classified according to the labels. For
example, in the grading system, students are classified on the
basis of the grades they obtain with respect to their marks in the
examination. This model works on a discrete dataset which
means the data need not be continuous.
2. Regression:
Such models work on continuous data. For example, if you wish
to predict your next salary, then you would put in the data of
your previous salary, any increments, etc., and would train the
model. Here, the data which has been fed to the machine is
continuous.

Unsupervised Learning
An unsupervised learning model works on unlabelled dataset. This
means that the data which is fed to the machine is random and there is
a possibility that the person who is training the model does not have
any information regarding it. The unsupervised learning models are
used to identify relationships, patterns and trends out of the data
which is fed into it. It helps the user in understanding what the data is
about and what are the major features identified by the machine in it.

Unsupervised learning models can be further divided into two


categories:
1. Clustering:
Refers to the unsupervised learning algorithm which can
cluster the unknown data according to the patterns or
trends identified out of it. The patterns observed might be
the ones that are known to the developer or it might even
come up with some unique patterns out of it.

2. Dimensionality Reduction:
We humans are able to visualise upto 3-Dimensions only
but according to a lot of theories and algorithms, there are
various entities that exist beyond 3-Dimensions. For
example, in Natural language Processing, the words are
considered to be N-Dimensional entities. Which means
that we cannot visualise them as they exist beyond our
visualisation ability. Hence, to make sense out of it, we
need to reduce their dimensions. Here, a dimensionality
reduction algorithm is used.
Stage 5: Evaluation
Once a model has been made and trained, it needs to go through
proper testing so that one can calculate the efficiency and
performance of the model. Hence, the model is tested with the help of
Testing Data (which was separated out of the acquired dataset at the
Data Acquisition stage) and the efficiency of the model is calculated
on the basis of the parameters mentioned below:

Neural Networks
Neural networks are loosely modelled by how neurons in the human
brain behave. The key advantage of neural networks are that they are
able to extract data features automatically without needing the input
of the programmer.
A neural network is essentially a system of organizing machine
learning algorithms to perform certain tasks. It is a fast and efficient
way to solve problems for which the dataset is very large, such as in
images.

Some of the features of a Neural Network are listed below:


Unit 3: Advance Python
Jupyter Notebook
The Jupyter Notebook is an incredibly powerful tool for interactively
developing and presenting AI related projects. The Jupyter project is
the successor to the earlier IPython Notebook, which was first
published as a prototype in 2010.
Jupyter Notebook is an open-source web application that can be
used to create and share documents that contain live code, equations,
visualizations, and text.
The easiest way to install and start using Jupyter Notebook is
through Anaconda. Anaconda is the most widely used Python
distribution for data science and comes pre-loaded with all the most
popular libraries and tools.
With Anaconda, comes the Anaconda Navigator through which we
can scroll around all the applications which come along with it.
Jupyter notebook can easily be accessed using the Anaconda
Prompt with the help of a local host.
To work with Jupyter Notebook, it is necessary to have a kernel on
which it operates. A kernel provides programming language support
in Jupyter. IPython is the default kernel for Jupyter Notebook.
A virtual environment is a tool that helps to keep dependencies
required by different projects separated, by creating isolated Python
virtual environments for them. This is one of the most important tools
that most of Python developers use.
Different projects developed in the system might have another
environment to keep their dependencies isolated from each other.

Python is a programming language that was created by Guido Van


Rossum in Centrum Wiskunde & Informatica. The language was
publicly released in 1991 and it got its name from a BBC comedy
series from 1970s – ‘Monty Python’s Flying Circus’.
There are various programming languages like Lisp, Prolog, C++,
Java and Python, which can be used for developing applications of
AI.
Why Python in AI?

• Easy to learn, read and maintain: Python has few keywords, a


simple structure and a clearly defined syntax. Python allows
anyone to learn the language quickly. A program written in
Python is fairly easy-to-maintain.
• A Broad Standard library: Python has a huge bunch of
libraries with plenty of built-in functions to solve a variety of
problems.
• Interactive Mode: Python has support for an interactive mode
that allows interactive testing and debugging of snippets of
code.
• Portability and Compatibility: Python can run on a wide
variety of operating systems and hardware platforms and has the
same interface on all platforms.
• Extendable: We can add low-level modules to the Python
interpreter. These modules enable programmers to add to or
customize their tools to be more efficient.
• Databases and Scalable: Python provides interfaces to all
major open source and commercial databases along with a better
structure and support for much larger programs than shell
scripting.

Applications of Python
Python Statements
Instructions written in the source code to execute are known as
statements. These are the lines of code that we write for the computer
to work upon. For example, if we wish to print the addition of two
numbers, say 5 and 10, we would simply write: print(5+10)
Python Comments
Comments are the statements that are incorporated in the code to give
a better understanding of code statements to the user. To write a
comment in Python, one can use # and then write anything after it.
For example:
# This is a comment and will not be read by the machine.
print(5+10) # This is a statement and the machine will print the
summation.
In Python, there exist some words which are pre-defined and carry a
specific meaning for the machine by default. These words are known
as keywords. Keywords cannot be changed at any point in time and
should not be used any other way except the default one, otherwise,
they create confusion and might result in ambiguous outputs.

Keywords in Python

Identifier
An identifier is any word that is variable. Identifiers can be declared
by the user as per their convenience of use and can vary according to
the way the user wants. These words are not defined and can be used
in any way. Keywords cannot be used as identifiers. Some examples
of keywords can be: count, interest, x, ai_learning, Test, etc.
Identifiers are also case-sensitive hence an identifier named as Test
would be different from an identifier named test.

Variable
A variable is a named location used to store data in memory. It is
helpful to think of variables as a container that holds data that can be
changed later throughout programming. Just like in Mathematics, in
Python too we can use variables to store values in it. The difference
here is, that in Python, the variables not only store numerical values
but can also contain different types of data.

Variable Examples:

• X = 10 # X variable contains numerical data


• Letters = ‘XYZ’ # Letters variable contains alphabetic data
• number = 13.95 # number variable contains a decimal value
• word = ‘k’ # word variable contains a character

Datatype
All variables contain different types of data in them. The type of data
is defined by the term datatype in Python. There can be various types
of data that are used in Python programming. Hence, the machine
identifies the type of variable according to the value which is stored
inside it.
Python inputs
In Python, not only can we display the output to the user, but we can
also collect data from the user and can pass it on to the Python script
for further processing. To collect the data from the user at the time of
execution, input() function is used.
While using the input function, the datatype of the expected input is
required to be mentioned so that the machine does not interpret the
received data in an incorrect manner as the data taken as input from
the user is considered to be a string (sequence of characters) by
default.
Python Operators
Operators are special symbols that represent computation. They are
applied to operand(s), which can be values or variables. The same
operators can behave differently on different data types. Operators
when applied to operands form an expression. Operators are
categorized as Arithmetic, Relational, Logical and Assignment. Value
and variables when used with operators are known as operands.
Conditional Operators
Operator Meaning Expression Result
20 > 10 True
> Greater Than
15 > 25 False
20 < 45 True
< Less Than
20 < 10 False
5 == 5 True
== Equal To
5 == 6 False
67 != 45 True
!= Not Equal to
35 != 35 False
45 >= 45 True
>= Greater than or Equal to
23 >= 34 False
13 <= 24 True
<= Less than or equal to
13 <= 12 False
Arithmetic Operators
Operator Meaning Expression Result
+ Addition 10 + 20 30
- Subtraction 30 - 10 20
* Multiplication 30 * 100 300
/ Division 30 / 10 20.0
// Integer Division 25 // 10 2
% Remainder 25 % 10 5
** Raised to power 3 ** 2 9

Logical Operators
Operator Meaning Expression Result
True and True True
and And operator
True and False False
True or False True
or Or operator
False or False False
not False True
not Not Operator
not True False

Assignment Operators
Operator Expression Equivalent to
= X=5 X=5
+= X +=5 X=X+5
-= X -= 5 X=X-5
*= X *= 5 X=X*5
/= X /= 5 X=X/5

Conditional Statements
Conditional statements help the machine in taking a decision
according to the condition which gets fulfilled. There exist different
types of conditional statements in Python. Some of them are:

• If statement
• If-else statement
• If-else ladder
Looping
The loop statements help in iterating statements or a group of
statements as many times as it is asked for. In this case, we will
simply write a loop that would start counting from 1 to 10. At every
count, it will print hello once on the screen and as soon as it reaches
10, the loop will stop executing. All this can be done by just one loop
statement. Various types of looping mechanisms are available in
Python. Some of them are:

• For Loop
• While Loop
• Do-While Loop

Python Packages
A package is nothing but a space where we can find codes or
functions or modules of similar type. There are various packages
readily available to use for free (perks of Python being an open-
sourced language) for various purposes.
• NumPy
- A package created to work around numerical arrays in python.
- Handy when it comes to working with large numerical databases and
calculations around it.
• OpenCV
- An image processing package that can explicitly work around
images and can be used for image manipulation and processing like
cropping, resizing, editing, etc.
• NLTK
- NLTK stands for Natural Language Tool Kit and it helps in tasks
related to textual data.
- It is one of the most commonly used packages for Natural Language
Processing.
• Pandas
- A package that helps in handling 2-dimensional data tables in
python.
- It is useful when we need to work with data from excel sheets and
other databases.
Unit 4: Data Science

Data Sciences
It is a concept to unify statistics, data analysis, machine learning and
their related methods in order to understand and analyse actual
phenomena with data. It employs techniques and theories drawn from
many fields within the context of Mathematics, Statistics, Computer
Science, and Information Science.
Applications of Data Sciences –
Fraud and Risk Detection
Over the years, banking companies learned to divide and conquer data
via customer profiling, past expenditures, and other essential variables
to analyse the probabilities of risk and default. Moreover, it also
helped them to push their banking products based on customers’
purchasing power.

Applications of Data Sciences –


Genetics & Genomics
Data Science applications also enable an advanced level of treatment
personalization through research in genetics and genomics. The goal
is to understand the impact of the DNA on our health and find
individual biological connections between genetics, diseases, and
drug response.
Applications of Data Sciences –
Internet Search
All search engines (including Google) make use of data science
algorithms to deliver the best result for our searched query in a
fraction of a second. Considering the fact that Google processes more
than 20 petabytes of data every day, had there been no data science,
Google wouldn’t have been the ‘Google’ we know today.

Applications of Data Sciences –


Targeted Advertising
If you thought Search would have been the biggest of all data science
applications, here is a challenger – the entire digital marketing
spectrum. Starting from the display banilrs on various websites to the
digital billboards at the airports – almost all of them are decided by
using data science algorithms.

Applications of Data Sciences –


Website Recommendations
Internet giants like Amazon, Twitter, Google Play, Netflix, LinkedIn,
IMDB and many more use this system to improve the user
experience. The recommendations are made based on previous search
results for a user.

Applications of Data Sciences –


Airline Route Planning
Now, while using Data Science, airline companies can:

• Predict flight delay • Decide which class of airplanes to buy


• Whether to directly land at the destination or take a halt in
between
• Effectively drive customer loyalty programs

Data Science is a combination of Python and Mathematical concepts


like Statistics, Data Analysis, probability, etc. Concepts of Data
Science can be used in developing applications around AI as it gives a
strong base for data analysis in Python.
The Scenario
Every day, restaurants prepare food in large quantities keeping in
mind the probable number of customers walking into their outlet. But
if the expectations are not met, a good amount of food gets wasted
which eventually becomes a loss for the restaurant as they either have
to dump it or give it to hungry people for free. And if this daily loss is
taken into account for a year, it becomes quite a big amount.
Problem Scoping

The Problem statement template leads us towards the goal of our


project which can now be stated as: “To be able to predict the quantity
of food dishes to be prepared for everyday consumption in restaurant
buffets.”
Data Acquisition

For this problem, a dataset covering all the elements mentioned above
is made for each dish prepared by the restaurant over a period of 30
days.
Data Exploration
After creating the database, we now need to look at the data collected
and understand what is required out of it. In this case, since the goal
of our project is to be able to predict the quantity of food to be
prepared for the next day, we need to have the following data:
Modelling
Once the dataset is ready, we train our model on it. In this case, a
regression model is chosen in which the dataset is fed as a dataframe
and is trained accordingly. Regression is a Supervised Learning
model which takes in continuous values of data over a period of time.
Evaluation
Once the model has been trained on the training dataset of 20 days, it
is now time to see if the model is working properly or not. Once the
model is able to achieve optimum efficiency, it is ready to be
deployed in the restaurant for real-time usage.
Data Collection
Data collection is nothing new that has come up in our lives. It has
been in our society for ages. Even when people did not have a fair
knowledge of calculations, records were still maintained in some way
or the other to keep an account of relevant things.
Data collection is an exercise that does not require even a tiny bit of
technological knowledge. But when it comes to analysing the data, it
becomes a tedious process for humans as it is all about numbers and
alpha-numerical data. That is where Data Science comes into the
picture.
Data Science not only gives us a clearer idea of the dataset but also
adds value to it by providing deeper and clearer analyses around it.
And as AI gets incorporated in the process, predictions and
suggestions by the machine become possible on the same.

Sources of Data
Offline Data Collection: Sensors, Surveys, Interviews, Observations.
Online Data Collection: Open-sourced Government Portals, Reliable
Websites (Kaggle), World Organisations’ open-sourced statistical
Observations websites
While accessing data from any of the data sources, the following
points should be kept in mind:

i. Data that is available for public usage only should be taken up.
ii. Personal datasets should only be used with the consent of the
owner.
iii. One should never breach someone’s privacy to collect data.
iv. Data should only be taken from reliable sources as the data
collected from random sources can be wrong or unusable.
v. Reliable sources of data ensure the authenticity of data which
helps in the proper training of the AI model.

Types of Data
For Data Science, usually, the data is collected in the form of tables.
These tabular datasets can be stored in different formats.

CSV
CSV stands for comma-separated values. It is a simple file format
used to store tabular data. Each line of this file is a data record and the
reach record consists of one or more fields that are separated by
commas. Since the values of records are separated by a comma, hence
they are known as CSV files.

Spreadsheet
A Spreadsheet is a piece of paper or a computer program that is used
for accounting and recording data using rows and columns into which
information can be entered. Microsoft Excel is a program that helps in
creating spreadsheets.

SQL
SQL is a programming language also known as Structured Query
Language. It is a domain-specific language used in programming and
is designed for managing data held in different kinds of DBMS
(Database Management Systems) It is particularly useful in handling
structured data.
Data Access
After collecting the data, to be able to use it for programming
purposes, we should know how to access the same in Python code. To
make our lives easier, there exist various Python packages which help
us in accessing structured data (in tabular form) inside the code.

NumPy
NumPy, which stands for Numerical Python, is the fundamental
package for Mathematical and logical operations on arrays in Python.
It is a commonly used package when it comes to working around
numbers. NumPy gives a wide range of arithmetic operations around
numbers giving us an easier approach to working with them. NumPy
also works with arrays, which are nothing but a homogenous
collection of Data.

An array is nothing but a set of multiple values which are of the


same data type. They can be numbers, characters, booleans, etc. but
only one datatype can be accessed through an array. In NumPy, the
arrays used are known as ND-arrays (N-Dimensional Arrays) as
NumPy comes with a feature of creating n-dimensional arrays in
Python.

NumPy Arrays

i. Homogenous collection of Data.


ii. Can contain only one type of data, hence not flexible with
datatypes.
iii. Cannot be directly initialized. Can be operated with Numpy
package only.
iv. Direct numerical operations can be done. For example, dividing
the whole array by 3 divides every element by 3 .
v. Widely used for arithmetic operations.
vi. Arrays take less memory space.
vii. Functions like concatenation, appending, reshaping, etc are not
trivially possible with arrays.
viii. Example: To create a numpy array ' A ':
import numpy
A= numpy.array ([1,2,3,4,5,6,7,8,9,0])

Lists

i. Heterogenous collection of Data.


ii. Can contain multiple types of data, hence flexible with
datatypes.
iii. Can be directly initialized as it is a part of Python syntax.
iv. Direct numerical operations are not possible. For example,
dividing the whole list by 3 cannot divide every element by 3 .
v. Widely used for data management.
vi. Lists acquire more memory space.
vii. Functions like concatenation, appending, reshaping, etc are
trivially possible with lists.
viii. Example: To create a list:
A = [1,2,3,4,5,6,7,8,9,0]

Pandas
Pandas is a software library written for the Python programming
language for data manipulation and analysis. In particular, it offers
data structures and operations for manipulating numerical tables and
time series. The name is derived from the term "panel data", an
econometrics term for data sets that include observations over
multiple time periods for the same individuals.
Pandas is well suited for many different kinds of data:

• Tabular data with heterogeneously-typed columns, as in an SQL


table or Excel spreadsheet
• Ordered and unordered (not necessarily fixed-frequency) time
series data.
• Arbitrary matrix data (homogeneously typed or heterogeneous)
with row and column labels
• Any other form of observational / statistical data sets. The data
actually need not be labelled at all to be placed into a Pandas
data structure
The two primary data structures of Pandas, Series (1-dimensional)
and DataFrame (2-dimensional), handle the vast majority of typical
use cases in finance, statistics, social science, and many areas of
engineering.
pandas does well:

• Easy handling of missing data (represented as NaN) in floating


point as well as non-floating point data
• Size mutability: columns can be inserted and deleted from
DataFrame and higher dimensional objects
• Automatic and explicit data alignment: objects can be explicitly
aligned to a set of labels, or the user can simply ignore the labels
and let Series, DataFrame, etc. automatically align the data for
you in computations
• Intelligent label-based slicing, fancy indexing, and subsetting of
large data sets
• Intuitive merging and joining data sets
• Flexible reshaping and pivoting of data sets

Matplotlib
Matplotlib is an amazing visualization library in Python for 2D plots
of arrays. Matplotlib is a multiplatform data visualization library built
on NumPy arrays.
Matplotlib comes with a wide variety of plots. Plots help to
understand trends, and patterns, and to make correlations. They’re
typically instruments for reasoning quantitative information.
Some types of graphs that we can make with this package are listed
below:

Not just plotting, but you can also modify your plots the way you
wish. You can stylise them and make them more descriptive and
communicable.
Basic Statistics with Python
Data Visualisation
Analysing the data collected can be difficult as it is all about tables
and numbers. While machines work efficiently on numbers, humans
need a visual aid to understand and comprehend the information
passed. Hence, data visualisation is used to interpret the data collected
and identify patterns and trends out of it.
Issues we can face with data:

• Erroneous Data: There are two ways in which the data can be
erroneous:
Incorrect values: The values in the dataset (at random places)
are incorrect.
Invalid or Null values: In some places, the values get corrupted
and hence they become invalid.
• Missing Data: In some datasets, some cells remain empty.
• Outliers: Data that do not fall in the range of a certain element
are referred to as outliers.

In Python, Matplotlib package helps in visualising the data and


making some sense out of it. As we have already discussed before,
with the help of this package, we can plot various kinds of graphs.
Scatter plots
Scatter plots are used to plot discontinuous data; that is, the data
which does not have any continuity in flow is termed as
discontinuous. There exist gaps in data that introduce discontinuity. A
2D scatter plot can display information maximum upto 4 parameters.

Bar Chart
It is one of the most commonly used graphical methods. From
students to scientists, everyone uses bar charts in some way or the
other. It is very easy-to-draw yet informative graphical representation.
Various versions of bar chart exist like a single bar chart, double bar
chart, etc
Histograms
Histograms are the accurate representation of continuous data. When
it comes to plotting the variation in just one entity of a period of time,
histograms come into the picture. It represents the frequency of the
variable at different points of time with the help of the bins.
Box Plots
When the data is split according to its percentile throughout the range,
box plots come in Haman. Box plots also known as box and whiskers
plot conveniently display the distribution of data throughout the range
with the help of 4 quartiles.

K-Nearest Neighbour
The k-nearest neighbours (KNN) algorithm is a simple, easy-to-
implement supervised machine learning algorithm that can be used to
solve both classification and regression problems. The KNN
algorithm assumes that similar things exist in close proximity. In
other words, similar things are near to each other as the saying goes
“Birds of a feather flock together”.
Some features of KNN are:

• The KNN prediction model relies on the surrounding points or


neighbours to determine its class or group
• Utilises the properties of the majority of the nearest points to
decide how to classify unknown points
• Based on the concept that similar data points should be close to
each other

KNN tries to predict an unknown value on the basis of the known


values. The model simply calculates the distance between all the
known points with the unknown point (by distance we mean to say
the difference between two values) and takes up K number of points
whose distance is minimum. And according to it, predictions are
made.
As we decrease the value of K to 1, our predictions become less
stable. Just think for a minute, imagine K=1 and we have X
surrounded by several greens and one blue, but the blue is the single
nearest neighbour. Reasonably, we would think X is most likely
green, but because K=1, KNN incorrectly predicts that it is blue.
Inversely, as we increase the value of K, our predictions
become more stable due to majority voting / averaging, and thus,
more likely to make more accurate predictions (up to a certain point).
Eventually, we begin to witness an increasing number of errors. It is
at this point we know we have pushed the value of K too far.
In cases where we are taking a majority vote (e.g. picking the mode
in a classification problem) among labels, we usually make K an odd
number to have a tiebreaker.
Unit 5:Computer Vision
Introduction
The Computer Vision domain of Artificial Intelligence, enables
machines to see through images or visual data, process and analyse
them on the basis of algorithms and methods in order to analyse
actual phenomena with images.
Applications of Computer Vision - Facial Recognition
With the advent of smart cities and smart homes, Computer Vision
plays a vital role in making the home smarter. Security being the most
important application involves the use of Computer Vision for facial
recognition. It can be either guest recognition or log maintenance of
the visitors. It also finds its application in schools for an attendance
system based on the facial recognition of students.

Applications of Computer Vision - Face Filters


Modern-day apps like Instagram and Snapchat have a lot of features
based on the usage of computer vision. The application of face filters
is one among them. Through the camera, the machine or the algorithm
is able to identify the facial dynamics of the person and applies the
facial filter selected.
Applications of Computer Vision - Google’s Search by Image
The maximum amount of searching for data on Google’s search
engine comes from textual data, but at the same time, it has an
interesting feature of getting search results through an image. This
uses Computer Vision as it compares different features of the input
image to the database of images and gives us the search result while at
the same time analysing various features of the image.
Applications of Computer Vision - Retail
The retail field has been one of the fastest growing fields and at the
same time is using Computer Vision for making the user experience
more fruitful. Retailers can use Computer Vision techniques to track
customers’ movements through stores, analyse navigational routes
and detect walking patterns.
Applications of Computer Vision - Inventory Management
Through security camera image analysis, a Computer Vision
algorithm can generate a very accurate estimate of the items available
in the store. Also, it can analyse the use of shelf space to identify
suboptimal configurations and suggest better item placement.

Applications of Computer Vision –


Self Driving Cars
Computer Vision is the fundamental technology behind developing
autonomous vehicles. Most leading car manufacturers in the world are
reaping the benefits of investing in artificial intelligence for
developing on-road versions of hands-free technology. This involves
the process of identifying the objects, getting navigational routes and
also at the same time environment monitoring.

Applications of Computer Vision –


Medical Imaging
For the last decades, computer-supported medical imaging application
has been a trustworthy help for physicians. It doesn’t only create and
analyse images, but also becomes an assistant and helps doctors with
their interpretation. The application is used to read and convert 2D
scan images into interactive 3D models that enable medical
professionals to gain a detailed understanding of a patient’s health
condition.

Applications of Computer Vision –


Google Translate App
All you need to do to read signs in a foreign language is to point your
phone’s camera at the words and let the Google Translate app tell you
what it means in your preferred language almost instantly. By using
optical character recognition to see the image and augmented reality
to overlay an accurate translation, this is a convenient tool that uses
Computer Vision.
Computer Vision Tasks: The various applications of Computer
Vision are based on a certain number of tasks that are performed to
get certain information from the input image which can be directly
used for prediction or forms the base for further analysis. The tasks
used in a computer vision application are :

Classification: The image Classification problem is the task of


assigning an input image one label from a fixed set of categories. This
is one of the core problems in CV that, despite its simplicity, has a
large variety of practical applications.

Classification + Localisation: This is the task that involves both


processes of identifying what object is present in the image and at the
same time identifying at what location that object is present in that
image. It is used only for single objects.

Object Detection: Object detection is the process of finding instances


of real-world objects such as faces, bicycles, and buildings in images
or videos. Object detection algorithms typically use extracted features
and learning algorithms to recognize instances of an object category.
It is commonly used in applications such as image retrieval and
automated vehicle parking systems.
Instance Segmentation: Instance Segmentation is the process of
detecting instances of the objects, giving them a category and then
giving each pixel a label on the basis of that. A segmentation
algorithm takes an image as input and outputs a collection of regions
(or segments).

Basics of Pixels: The word “pixel” means a picture element. Every


photograph, in digital form, is made up of pixels. They are the
smallest unit of information that make up a picture. Usually round or
square, they are typically arranged in a 2-dimensional grid.

Resolution: The number of pixels in an image is sometimes called the


resolution. When the term is used to describe pixel count, one
convention is to express resolution as the width by the height, for
example, a monitor resolution of 1280×1024. This means there are
1280 pixels from one side to the other, and 1024 from top to bottom.
Another convention is to express the number of pixels as a single
number, like a 5 megapixel camera (a megapixel is a million pixels).
This means the pixels along the width multiplied by the pixels along
the height of the image taken by the camera equals 5 million pixels. In
the case of our 1280×1024 monitors, it could also be expressed as
1280 x 1024 = 1,310,720, or 1.31 megapixels.

Pixel value: Each of the pixels that represent an image stored inside a
computer has a pixel value that describes how bright that pixel is,
and/or what colour it should be. The most common pixel format is the
byte image, where this number is stored as an 8-bit integer giving a
range of possible values from 0 to 255. Typically, zero is to be taken
as no colour or black and 255 is taken to be full colour or white.

Grayscale Images: Grayscale images are images that have a range of


shades of gray without apparent colour. The darkest possible shade is
black, which is the total absence of colour or zero value of a pixel.
The lightest possible shade is white, which is the total presence of
colour or 255 value of a pixel. Intermediate shades of gray are
represented by equal brightness levels of the three primary colours. A
grayscale has each pixel of size 1 byte having a single plane of 2d
array of pixels. The size of a grayscale image is defined as the Height
x Width of that image.

RGB Images: All the images that we see around are coloured images.
These images are made up of three primary colours Red, Green and
Blue. All the colours that are present can be made by combining
different intensities of red, green and blue.

Image Features: In computer vision and image processing, a feature


is a piece of information that is relevant for solving the computational
task related to a certain application. Features may be specific
structures in the image such as points, edges or objects.
In image processing, we can get a lot of features from the image. It
can be either a blob, an edge or a corner. These features help us to
perform various tasks and then get the analysis done on the basis of
the application. Now the question that arises is which of the following
are good features to be used? As you saw in the previous activity, the
features having corners are easy to find as they can be found only at a
particular location in the image, whereas the edges are spread over a
line or an edge look the same all along. This tells us that the corners
are always good features to extract from an image followed by the
edges.

OpenCV or Open Source Computer Vision Library is a tool that


helps a computer extract these features from the images. It is used for
all kinds of image and video processing and analysis. It is capable of
processing images and videos to identify objects, faces, or even
handwriting.

Convolution: Different filters applied to an image change the pixel


values evenly throughout the image with the help of the process of
convolution and the convolution operator which is commonly used to
create these effects. As we change the values of these pixels, the
image changes. This process of changing pixel values is the base of
image editing.
We all use a lot of image editing software like photoshop and at the
same time use apps like Instagram and Snapchat, which apply filters
to the image to enhance the quality of that image.

Convolution: Convolution is a simple Mathematical operation that is


fundamental to many common image-processing operators.
Convolution provides a way of `multiplying together' two arrays of
numbers, generally of different sizes, but of the same dimensionality,
to produce a third array of numbers of the same dimensionality. An
(image) convolution is simply an element-wise multiplication of
image arrays and another array called the kernel followed by a sum.

What is a Kernel?
A Kernel is a matrix, which is slid across the image and multiplied
with the input such that the output is enhanced in a certain desirable
manner. Each kernel has a different value for different kind of effects
that we want to apply to an image.
Convolution

i. Convolution is a common tool used for image editing.


ii. It is an element-wise multiplication of an image and a kernel to
get the desired output.
iii. In computer vision applications, it is used in Convolutional
Neural Network (CNN) to extract image features.

What is a Convolutional Neural Network?


A Convolutional Neural Network (CNN) is a Deep Learning
algorithm that can take in an input image, assign importance
(learnable weights and biases) to various aspects/objects in the image
and be able to differentiate one from the other.
A convolutional neural network consists of the following layers:
1) Convolution Layer 2) Rectified linear Unit (ReLU) 3) Pooling
Layer 4) Fully Connected Layer

Convolution Layer
It is the first layer of a CNN. The objective of the Convolution
Operation is to extract the high-level features such as edges, from the
input image. CNN need not be limited to only one Convolutional
Layer. Conventionally, the first Convolution Layer is responsible for
capturing the Low-Level features such as edges, colour, gradient
orientation, etc. With added layers, the architecture adapts to the
High-Level features as well, giving us a network that has a
wholesome understanding of images in the dataset.
Rectified Linear Unit Function
The next layer in the Convolution Neural Network is the Rectified
Linear Unit function or the ReLU layer. After we get the feature map,
it is then passed onto the ReLU layer. This layer simply gets rid of all
the negative numbers in the feature map and lets the positive number
stay as it is.
Pooling Layer
Similar to the Convolutional Layer, the Pooling layer is responsible
for reducing the spatial size of the Convolved Feature while still
retaining the important features. There are two types of pooling which
can be performed on an image.

i. Max Pooling : Max Pooling returns the maximum value from


the portion of the image covered by the Kernel.
ii. Average Pooling: Max Pooling returns the maximum value from
the portion of the image covered by the Kernel.

The pooling layer is an important layer in the CNN as it performs a


series of tasks which are as follows :

• Makes the image smaller and more manageable


• Makes the image more resistant to small transformations,
distortions and translations in the input image.

Fully Connected Layer


The final layer in the CNN is the Fully Connected Layer (FCP). The
objective of a fully connected layer is to take the results of the
convolution/pooling process and use them to classify the image into a
label.
Unit 6: Natural Language Process
Natural Language Processing, or NLP, is the sub-field of AI that is
focused on enabling computers to understand and process human
languages. AI is a subfield of Linguistics, Computer Science,
Information Engineering, and Artificial Intelligence concerned with
the interactions between computers and human (natural) languages, in
particular how to program computers to process and analyse large
amounts of natural language data.
Applications of Natural Language Processing

• Automatic Summarization: Automatic summarization is


relevant not only for summarizing the meaning of documents
and information, but also to understand the emotional meanings
within the information, such as in collecting data from social
media.
• Sentiment Analysis: The goal of sentiment analysis is to
identify sentiment among several posts or even in the same post
where emotion is not always explicitly expressed.
• Text classification: Text classification makes it possible to
assign predefined categories to a document and organize it to
help you find the information you need or simplify some
activities. For example, an application of text categorization is
spam filtering in email.
• Virtual Assistants: Nowadays Google Assistant, Cortana, Siri,
Alexa, etc have become an integral part of our lives. Not only
can we talk to them but they also have the ability to make our
lives easier. By accessing our data, they can help us in keeping
notes of our tasks, make calls for us, send messages and a lot
more.

Natural Language Processing: Natural Language Processing is all


about how machines try to understand and interpret human language
and operate accordingly.
AI Project Cycle in Natural Language Processing
• The Scenario: We get to hear a lot of cases where people are
depressed due to reasons like peer pressure, studies, family
issues, relationships, etc. To overcome this, cognitive
behavioural therapy (CBT) is considered to be one of the best
methods.
• Problem Scoping: CBT is a technique used by most therapists
to cure patients out of stress and depression. But it has been
observed that people do not wish to seek the help of a
psychiatrist willingly.
• Data Acquisition: To understand the sentiments of people, we
need to collect their conversational data so the machine can
interpret the words that they use and understand their meaning.
• Data Exploration: Once the textual data has been collected, it
needs to be processed and cleaned so that an easier version can
be sent to the machine.
• Modelling: Once the text has been normalised, it is then fed to
an NLP based AI model. Note that in NLP, modelling requires
data pre-processing only after which the data is fed to the
machine.
• Evaluation: The model trained is then evaluated and the
accuracy for the same is generated on the basis of the relevance
of the answers which the machine gives to the user’s responses.

Chatbots
Script-bot Smart-bot
Script bots are easy to make Smart bots are flexible and powerful
Script bots work around a script that is Smart bots work on bigger databases
programmed into them and other resources directly
Mostly they are free and are easy to
Smart bots learn with more data
integrate into a messaging platform
Coding is required to take this up on
No or little language processing skills
board
Limited functionality Wide functionality
All the assistants like Google Assistant, Alexa, Cortana, Siri, etc. can
be taken as smart bots as not only can they handle the conversations
but can also manage to do other tasks which makes them smarter.
Human Language VS Computer Language

• Humans communicate through language which we process all


the time. Our brain keeps on processing the sounds that it hears
around us and tries to make sense of them all the time.
• On the other hand, the computer understands the language of
numbers. Everything that is sent to the machine has to be
converted to numbers.

Use cases:

• Arrangement of the words and meaning


• Multiple Meanings of a word
• Perfect Syntax, no Meaning

Data Processing

• It's how Natural Language Processing makes it possible for


machines to understand and speak in Natural Languages just
like humans.
• Since we all know that the language of computers is Numerical,
the very first step that comes to our mind is to convert our
language to numbers.

Text Normalisation: In Text Normalisation, we undergo several


steps to normalize the text to a lower level. Text Normalisation helps
in cleaning up the textual data in such a way that it comes down to a
level where its complexity is lower than the actual data.
Sentence Segmentation: Under sentence segmentation, the whole
corpus is divided into sentences. Each sentence is taken as different
data so now the whole corpus gets reduced to sentences.
Tokenisation: After segmenting the sentences, each sentence is then
further divided into tokens. The token is a term used for any word or
number or special character occurring in a sentence. Under
tokenisation, every word, number and special character is considered
separately and each of them is now a separate token.
Removing Stopwords, Special Characters and Numbers: In this
step, the tokens which are not necessary are removed from the token
list. Stopwords are the words which occur very frequently in the
corpus but do not add any value to it. Humans use grammar to make
their sentences meaningful for the other person to understand. But
grammatical words do not add any essence to the information which
is to be transmitted through the statement hence they come under
stopwords.

Converting text to a common case: After the stopwords removal, we


convert the whole text into a similar case, preferably lower case. This
ensures that the case-sensitivity of the machine does not consider
same words as different just because of different cases.

Stemming: In this step, the remaining words are reduced to their root
words. In other words, stemming is the process in which the affixes of
words are removed and the words are converted to their base form.
Lemmatization: Stemming and lemmatization both are alternative
processes to each other as the role of both processes is the same –
removal of affixes. But the difference between both of them is that in
lemmatization, the word we get after affix removal (also known as
lemma) is a meaningful one. Lemmatization makes sure that a lemma
is a word with meaning and hence it takes a longer time to execute
than stemming.
Bag of Words: Bag of Words is a Natural Language Processing
model which helps in extracting features out of the text which can be
helpful in machine learning algorithms. In bag of words, we get the
occurrences of each word and construct the vocabulary for the corpus.
TFIDF: Term Frequency & Inverse Document Frequency
The bag of words algorithm gives us the frequency of words in each
document we have in our corpus. It gives us an idea that if the word is
occurring more in a document, its value is more for that document.
For example, if I have a document on air pollution, air and pollution
would be the words that occur many times in it. And these words are
valuable too as they give us some context around the document.
Term Frequency: Term frequency is the frequency of a word in one
document. Term frequency can easily be found from the document
vector table as in that table we mention the frequency of each word of
the vocabulary in each document.
Inverse Document Frequency: Now, let us look at the other half of
TFIDF which is Inverse Document Frequency. For this, let us first
understand what does document frequency mean. Document
Frequency is the number of documents in which the word occurs
irrespective of how many times it has occurred in those documents.
Finally, the formula of TFIDF for any word W becomes:
TFIDF(W) = TF(W) * log( IDF(W) )
Here, the log is to the base of 10.
Summarising the TFIDF concept, we can say that:

1. Words that occur in all the documents with high term


frequencies have the least values and are considered to be
stopwords.
2. For a word to have high TFIDF value, the word needs to have a
high term frequency but less document frequency which shows
that the word is important for one document but is not a
common word for all documents.
3. These values help the computer understand which words are to
be considered while processing the natural language. The higher
the value, the more important the word is for a given corpus.

Applications of TFIDF
TFIDF is commonly used in the Natural Language Processing
domain. Some of its applications are:

• Document Classification: Helps in classifying the type and


genre of a document.
• Topic Modelling: It helps in predicting the topic for a corpus.
• Information Retrieval System: To extract the important
information out of a corpus.
• Stop word filtering: Helps in removing unnecessary words
from a text body.
Unit 7: Evaluation
Introduction: In the Evaluation stage, we will explore different
methods of evaluating an AI model. Model Evaluation is an integral
part of the model development process. It helps to find the best model
that represents our data and how well the chosen model will work in
the future.
What is evaluation?
Evaluation is the process of understanding the reliability of any AI
model, based on outputs by feeding the test datasets into the model
and comparing it with actual answers.
It’s not recommended to use the data we used to build the model to
evaluate it. This is because our model will simply remember the
whole training set, and will therefore always predict the correct label
for any point in the training set. This is known as overfitting.

Here, we can see in the picture that a forest fire has broken out in the
forest. The model predicts a Yes which means there is a forest fire.
The Prediction matches with the Reality. Hence, this condition is
termed as True Positive.
Here there is no fire in the forest hence the reality is No. In this case,
the machine too has predicted it correctly as a No. Therefore, this
condition is termed as True Negative.
Here the reality is that there is no forest fire. But the machine has
incorrectly predicted that there is a forest fire. This case is termed as
False Positive.

Here, a forest fire has broken out in the forest because of which the
Reality is Yes but the machine has incorrectly predicted it as a No
which means the machine predicts that there is no Forest Fire.
Therefore, this case becomes False Negative.

Confusion matrix: The result of the comparison between the


prediction and reality can be recorded in what we call the confusion
matrix. The confusion matrix allows us to understand the prediction
results. Note that it is not an evaluation metric but a record that can
help in evaluation.
Prediction and Reality can be easily mapped together with the help of
this confusion matrix.
Accuracy is defined as the percentage of correct predictions out of all
the observations.

Here, total observations cover all the possible cases of prediction that
can be True Positive (TP), True Negative (TN), False Positive (FP)
and False Negative (FN).
But this parameter is useless for us as the actual cases where the fire
broke out are not taken into account. Hence, there is a need to look at
another parameter that takes account of such cases as well.
Precision: Precision is defined as the percentage of true positive
cases versus all the cases where the prediction is true. That is, it takes
into account the True Positives and False Positives.
If Precision is high, this means the True Positive cases are more,
giving lesser False alarms.
Recall
Another parameter for evaluating the model’s performance is Recall.
It can be defined as the fraction of positive cases that are correctly
identified. It majorly takesinto account the true reality cases where in
Reality there was a fire but the machine either detected it correctly or
it didn’t. That is, it considers True Positives (There was a forest fire in
reality and the model predicted a forest fire) and False Negatives
(There was a forest fire and the model didn’t predict it).

Now as we notice, we can see that the Numerator in both Precision


and Recall is the same: True Positives. But in the denominator,
Precision counts the False Positives while Recall takes False
Negatives into consideration.
Which Metric is Important?
Choosing between Precision and Recall depends on the condition in
which the model has been deployed. In a case like Forest Fire, a False
Negative can cost us a lot and is risky too. Imagine no alert being
given even when there is a Forest Fire. The whole forest might burn
down.
Which Metric is Important?
Another case where a False Negative can be dangerous is Viral
Outbreak. Imagine a deadly virus has started spreading and the model
which is supposed to predict a viral outbreak does not detect it. The
virus might spread widely and infect a lot of people.
Which Metric is Important?
On the other hand, there can be cases in which the False Positive
condition costs us more than False Negatives. One such case is
Mining. Imagine a model telling you that there exists treasure at a
point and you keep on digging there but it turns out that it is a false
alarm. Here, False Positive case (predicting there is treasure but there
is no treasure) can be very costly.

F1 Score
F1 score can be defined as the measure of the balance between
precision and recall.

An ideal situation would be when we have a value of 1 (that is 100%)


for both Precision and Recall. In that case, the F1 score would also be
an ideal 1 (100%). It is known as the perfect value for F1 Score. As
the values of both Precision and Recall ranges from 0 to 1, the F1
score also ranges from 0 to 1.
In conclusion, we can say that a model has good performance if the
F1 Score for that model is high.

You might also like