ARTIFICIAL INTELLIGENCE

COURSE TITLE: THEORIES OF COMPUTATION AND ALGORITHM

COURSE CODE: AIT 314

MATHEMATICAL MODELS OF COMPUTATIONS

Theories of computation and algorithms are fundamental concepts in computer science,
providing the theoretical foundation for understanding the nature and limits of
computation, as well as designing efficient algorithms to solve computational problems.
Here's an overview of one of the key aspects related to theories of computation and
algorithms:
Theory of Computation:
 Automata Theory: This branch studies abstract machines and their computational
capabilities. Finite Automata, Pushdown Automata, and Turing Machines are
examples of abstract models used to understand different levels of computation.
 Formal Languages: Describes various types of languages and grammars that can be
used to define and represent computations. Regular languages, context-free
languages, and recursively enumerable languages are commonly studied.
1. Automata Theory:
Definition: Automata theory deals with/studies the abstract machines as well as the
computational models to understand the capabilities and limitations of computation. These
abstract machines are defined by a set of rules that determine how they transition between
states based on input.
Key Concepts:
 Finite Automata (FA): These are machines with a finite number of states and
operate on an input alphabet. They are often used to recognize regular languages.
 Pushdown Automata (PDA): Extends the capabilities of finite automata by adding
a stack, enabling the recognition of context-free languages.
 Turing Machines (TM): The most powerful and widely studied model. It includes
an infinite tape and a read/write head, allowing it to perform any computation that
can be algorithmically described.

2. Formal Languages:
Definition: Formal languages are sets of strings with well-defined rules for determining
which strings belong to a language and which one doesn’t and formal language theory
provides tools to define and describe these languages.
Key Concepts:
Regular Languages: Can be recognized by finite automata and described by regular
expressions.
Context-Free Languages: Described by context-free grammars and recognized by
pushdown automata.
Recursively Enumerable Languages: Recognizable by Turing machines, which can accept
any language for which a Turing machine can be constructed to accept it.
Formal Language vs. Automata Model:
1. Definition:
 Formal Language: A formal language is a set of strings over a finite alphabet.
It is a well-defined, structured collection of strings that follow certain rules.
 Automata Model: An automata model is an abstract machine or
computational model that defines how computations or processes can be
performed on strings. Automata models are used to recognize or generate
formal languages.
2. Nature:
 Formal Language: Describes a set of strings without specifying how those
strings are generated or recognized. It focuses on the structure and
membership of the strings.
 Automata Model: Describes a computational device or machine that
processes strings according to a set of rules. It defines how strings are
recognized or generated by the machine.
3. Representation:
 Formal Language: Represented by a set of rules or specifications, often
through formal grammars (such as context-free grammars) or regular
expressions.
 Automata Model: Represented by the structure and behavior of an abstract
machine, such as a Finite Automaton (FA), Pushdown Automaton (PDA), or
Turing Machine (TM).
4. Purpose:
 Formal Language: Focuses on characterizing and specifying languages,
allowing for precise description and understanding of sets of strings.
  Automata Model: Focuses on computation and the processing of strings. It
provides a way to recognize, accept, or generate strings based on a set of
rules defined by the automaton.
5. Examples:
 Formal Language: Examples include regular languages, context-free
languages, context-sensitive languages, and recursively enumerable
languages.
 Automata Model: Examples include Finite Automaton (FA), Pushdown
Automaton (PDA), and Turing Machine (TM), each designed to recognize
different classes of languages.
6. Relationship:
 Formal Language and Automata Model: There is a close relationship between
formal languages and automata models. Automata are used to define and
recognize formal languages, and formal languages help describe the sets of
strings that can be processed by automata.
In summary, a formal language is a set of strings defined by specific rules, while an
automata model is an abstract machine that processes strings according to a set of rules.
Formal languages and automata models are interconnected, with automata providing a
way to recognize or generate strings described by formal languages.
FORMAL LANGUAGE THEORY
Formal language theory is a branch of theoretical computer science that deals with the
study of formal languages, grammars, and automata. It provides a mathematical framework
for defining and analyzing languages, as well as the automata that recognize these
languages. Here's a detailed explanation of formal language theory:
1. Alphabet and Strings:
 Alphabet (Σ): A finite set of symbols. Formal languages are constructed over
an alphabet.
 String (w): A finite sequence of symbols from an alphabet.
2. Formal Language (L):
 Formal Language: A set of strings over a specified alphabet. Formal
languages can be finite or infinite, and they are often defined by certain rules
or patterns.
3. Grammar:
  Grammar (G): A set of rules that defines how strings in a formal language can
be generated. Grammars are used to describe the syntax and structure of
languages.
4. Formal Grammar Types:
 Regular Grammar: Corresponds to regular languages and is described by
regular expressions. Regular grammars are recognized by finite automata.
 Context-Free Grammar (CFG): Corresponds to context-free languages.
Context-free grammars are recognized by pushdown automata.
 Context-Sensitive Grammar (CSG): Corresponds to context-sensitive
languages. Context-sensitive grammars are recognized by linear-bounded
automata.
 Recursively Enumerable Grammar: Corresponds to recursively enumerable
languages. These grammars are recognized by Turing machines.
5. Chomsky Hierarchy:
 Chomsky Hierarchy: Classifies formal languages into four types (regular,
context-free, context-sensitive, and recursively enumerable) based on the
types of grammars that generate them. Each type of grammar corresponds to
a specific level of computational power.
Others are Decidability, Closure Properties, Pumping Lemma etc.
COMPUTABILITY THEORY
Computability theory, also known as recursion theory, is a branch of theoretical computer
science that focuses on understanding the concept of computability and the limits of what
can be computed algorithmically. It investigates questions such as which problems can be
solved by algorithms, which problems cannot be solved, and how we can reason about
computation itself. Here's a detailed explanation of computability theory:
1. Decision Problems:
 Decision Problem: A problem where the answer is either "yes" or "no." In
computability theory, the focus is often on decision problems, which can be
formulated as determining whether a given input satisfies a certain property.
2. Turing Machines:
 Turing Machine (TM): An abstract mathematical model introduced by Alan
Turing in the 1930s. A Turing machine consists of an infinite tape, a
read/write head, and a finite set of states. It can simulate any algorithmic
process and serves as a fundamental model for studying computability.
3. Church-Turing Thesis:
  Church-Turing Thesis: Proposes that anything computable by an algorithm
can be computed by a Turing machine. This thesis serves as the foundation
for defining computability and understanding the limits of what can be
computed.
4. Computable Functions:
 Computable Function: A function for which there exists an algorithm (a
Turing machine) that computes its values. Computable functions are also
known as recursive or effectively calculable functions.
5. Decidability:
 Decidable Problem: A decision problem for which there exists an algorithm
that can always provide a correct answer for all possible inputs. Decidability
is a key concept in computability theory, and the study of decidable problems
helps characterize the scope of computability.
6. Undecidability:
 Undecidable Problem: A decision problem for which there is no algorithm
that can always provide a correct answer for all possible inputs. The
existence of undecidable problems, such as the Halting Problem,
demonstrates the limitations of computation.
7. Halting Problem:
 Halting Problem: The decision problem of determining, given a description
of an arbitrary computer program and an input, whether the program will
eventually halt or continue running indefinitely. The Halting Problem is a
classic example of an undecidable problem.
8. Gödel's Incompleteness Theorems:
 Gödel's Incompleteness Theorems: A set of results by Kurt Gö del that
demonstrates the inherent limitations of formal mathematical systems. One
consequence is the existence of true but unprovable statements within such
systems, highlighting the limits of formal reasoning.
COMPLEXITY THEORY
Complexity theory, also known as computational complexity theory, is a branch of
theoretical computer science that focuses on the study of the resources required to solve
computational problems. It investigates questions related to the efficiency of algorithms,
the classification of problems based on their computational difficulty, and the inherent
limits of what can be efficiently computed. Here's an explanation of complexity theory:
1. Time Complexity:
  Time Complexity: Measures the amount of time an algorithm takes to
complete as a function of the size of its input. It provides insights into how
the running time of an algorithm scales with the size of the problem.

2. Space Complexity:
 Space Complexity: Measures the amount of memory space (or other
resources) an algorithm requires as a function of the size of its input. It
provides insights into the memory requirements of an algorithm.
3. Computational Classes:
 P (Polynomial Time): Class of problems that can be solved in polynomial
time by a deterministic Turing machine. Algorithms in P are considered
efficient.
 NP (Nondeterministic Polynomial Time): Class of problems for which a
solution can be verified in polynomial time by a nondeterministic Turing
machine. The question of whether P equals NP is a major open problem in
complexity theory.
 EXP (Exponential Time): Class of problems that can be solved in
exponential time. Problems in EXP are generally considered intractable for
large inputs.
4. Reduction:
 Reduction: A technique used to show that one problem is as hard as another.
Polynomial-time reductions are often employed to establish the complexity
of problems by reducing them to known problems.
5. Completeness:
 NP-Completeness: A class of problems in NP that are among the most
difficult problems in NP. If any NP-complete problem has a polynomial-time
algorithm, then all problems in NP have polynomial-time algorithms (implies
P = NP).
6. Hardness:
 Hardness: Describes the difficulty of a problem. NP-hard problems are at
least as hard as the hardest problems in NP.
7. Oracle Machines:
 Oracle Machines: Extensions of Turing machines that can query an "oracle"
for answers to specific decision problems. Complexity classes involving
 oracle machines are used to study the limits of computation beyond standard
models.
8. Quantum Complexity:
 Quantum Complexity Theory: Extends classical complexity theory to
quantum computation. Quantum complexity classes, such as BQP (bounded-
error quantum polynomial time), consider the power of quantum computers.

9. Randomized Complexity:
 Randomized Complexity: Studies algorithms that use randomization to
solve problems. Complexity classes like BPP (bounded-error probabilistic
polynomial time) involve randomized algorithms.
10. Average-Case Complexity:
 Average-Case Complexity: Studies the expected behavior of algorithms on
average inputs. It complements worst-case complexity by considering the
average behavior of algorithms over a distribution of inputs.
Comparison of the computational capabilities of Finite Automata (FA), Pushdown
Automata (PDA), and Turing Machines (TM) based on various aspects:
Aspect Finite Automata (FA) Pushdown Automata Turing Machine (TM)
(PDA)
Computational Limited Intermediate Universal
Capability
Language Class Regular Context-Free Recursively Enumerable
Memory Finite (States) Stack Infinite Tape
Power Limited for regular Can handle nested Universal Turing
languages structures machine
Expressiveness Limited More expressive than Most expressive
FA
Decidability Decidable Decidable Semidecidable (Halting
Problem)
Determinism Can be deterministic or Can be deterministic or Can be deterministic or
non-deterministic non-deterministic non-deterministic

FINITE AUTOMATA
Finite automata (FA) are computational models used to recognize patterns within strings of
symbols from a finite alphabet. Finite automata are characterized by a finite set of states
and transitions between these states based on input symbols. It is used to characterize a
Regular Language.
Key components of a finite automaton include:
1. Alphabet (Σ): The finite set of symbols from which input strings are composed.
2. States (Q): Are finite set of states representing the internal configuration of the
automaton in which each state indicates the current mode or situation of the
machine.
3. Transitions (δ): A transition function is a function that maps a state and an input
symbol to a new state. It defines how the automaton moves from one state to
another based on the input symbol read.
4. Initial State (q0): This is the starting state from which the automaton begins
processing input.
5. Accepting States (F): They are set of states indicating acceptance criteria. If the
automaton enters one of these states after processing the input, the input is
considered accepted.
Finite automata can be further classified into two main types based on their behavior:
1. Deterministic Finite Automaton (DFA): In a DFA, for each state and input symbol,
there is exactly one next state. It means that the transition from one state to another
is uniquely determined by the current state and the input symbol. DFAs are suited
for recognizing regular languages and can be represented using transition diagrams
or tables.
2. Nondeterministic Finite Automaton (NFA): In an NFA, for a given state and input
symbol, there can be multiple possible next states or even transitions that consume
no input. NFAs allow for more flexibility in modeling certain patterns but require
additional computational resources for processing. They are also capable of
recognizing regular languages and can be represented using transition diagrams or
tables.
REGULAR LANGUAGES
A regular language is a language that can be expressed with a regular expression or a
deterministic or non-deterministic finite automata or state machine. A language is a set of
strings which are made up of characters from a specified alphabet, or set of symbols.
CONCEPTS OF NON-COMPUTABILITY AND UNDECIDABILITY
Non-computability and undecidability are fundamental concepts in theoretical computer
science that highlight the limits of what can be achieved algorithmically.
1. Non-computability: Refers to functions or problems for which no algorithm can
compute a correct solution for all possible inputs. It highlights tasks that cannot be
accomplished algorithmically.
 2. Undecidability: Refers to decision problems for which there is no algorithm that
always provides a correct answer for all inputs. It reveals questions that cannot be
definitively answered by any algorithm.
Together, these concepts demonstrate the inherent limitations of computation, shaping our
understanding of what can and cannot be achieved algorithmically in computer science.
ALGORITHM
An algorithm is a step-by-step procedure designed to solve a specific problem. It is a finite
sequence of well-defined instructions that, when followed, leads to the solution of a
problem or the achievement of a goal. Algorithms can be implemented in various forms,
including as computer programs, mathematical formulas, flowcharts, or natural language
descriptions. Algorithms are used in a wide range of applications, from simple tasks like
sorting a list of numbers to complex operations like image recognition and natural
language processing. They are the building blocks of programs and software, dictating how
a computer should process information and perform tasks. The role of algorithms in
problem-solving is multifaceted.
Firstly, they provide a systematic approach to solving problems. This means that they
break down complex problems into manageable steps, making it easier to understand and
solve the problem. This systematic approach also ensures that the same problem can be
solved consistently, regardless of who or what is implementing the algorithm.

Secondly, algorithms help in optimising the problem-solving process. There are often
many ways to solve a problem, but not all solutions are created equal. Some solutions may
be faster, more efficient, or use fewer resources than others. Algorithms allow us to
compare different solutions and choose the most optimal one. This is particularly
important in computer science, where resources like time and memory are often limited.

Lastly, algorithms are reusable and universal. Once an algorithm has been developed to
solve a particular problem, it can be used again to solve similar problems. This saves time
and effort in the long run. Moreover, algorithms are not tied to a specific programming
language or platform. They are a universal language of problem-solving that can be
implemented in any programming environment. In conclusion, algorithms are an essential
tool in problem-solving. They provide a systematic, optimised, and reusable approach to
solving problems, making them a fundamental part of computer science and programming.