M.

Tayyab
 Automata Theory is a branch of the Theory of Computation.
 It deals with the study of abstract machines and their capacities for
computation.
 An abstract machine is called the automata.
 It includes the design and analysis of automata, which are
mathematical models that can perform computations on strings of
symbols according to a set of rules.
 RC OPTIMIALLY UNDERSTOOD MQM

 Regular Expressions (RE) : Used for pattern matching in

Linux/Unix command prompt, programming languages.
 Finite Automata in Modeling Systems : Used in designing/checking
models and electronic circuits that operate based on certain rules.
 Context-Free Grammars (CFG) : Used in Compiler / Programming
Language design to describe syntax and NLP to describe structure.
 Mathematical Models : Mathematical understanding of computing
devices by mathematically modeling them.
 Quantum Computing: Turing Machines are a fundamental building
block for understanding quantum computation models.
 Optimizing Algorithm: Classify problems (e.g., P, NP, NP-complete,
and NP-hard), proving that some problems have no efficient solutions.
 Understanding Computability : Study of which problems can be
solved using algorithms, essentially defining the boundaries of what a
computer can calculate.
The fundamental concepts of strings can be classified into the following
three parts

 Symbol: Basic building block of a string. We can compare this as

letters in an alphabet.
 Alphabet (Σ): All possible symbols used in a language are termed as
alphabet.
 String (w): The collection of letters or symbols which are present in
the alphabet set is termed as strings. In automata theory, we use the
notation ‘w’ to represent strings. Strings would be finite in length.
TOA focuses on how these strings are built and recognized.
 String Length: The number of Valid letters/symbols.

 Finiteness: In automata theory we always use finite strings.

 Concatenation: We can combine multiple strings together through
concatenation operations like "ab" is concatenated with "cd" will
make "abcd".
 Empty String (ε): The idea of empty string or ε is unique in automata
theory which is like a placeholder which contains nothing, not a single
symbol.
 Substring: Given a string s, substring of s is any part of the string s
means w is a substring of s, if there exist strings x and y (either or
both possibly null) such that s = xwy.
 An alphabet is a finite, nonempty set of symbols. By convention
we use the symbol  for an alphabet.
 Normally our alphabet consist of individual characters.
  = {0,1} the binary alphabet
  = {a,b, … z} the set of all lowercase letters
 string (or sometimes a word)
◦ A finite sequence of symbols chosen from an alphabet. For
example, 010101010 is a string chosen from the binary alphabet, as
is the string 0000 or 1111.
 The empty string is the string with zero occurrences of symbols.
This string is denoted ε and may be chosen from any alphabet.
 The power notation is used to represent multiple occurrences of a
string; e.g. a3 = aaa, a2 = aa, etc.
 Powers of an alphabet
◦ If  is an alphabet, we can express the set of all strings of a certain
length from that alphabet by using an exponential notation.
◦ k is defined to be the set of strings of length k belonging from.
◦ For example, given the alphabet  = {0,1,2} then:
 0 = {ε}  2 = {00,01,02,10,11,12,20,21,22}
 1 = {0,1,2}  3 = {000,001,002,... 222}

 Note that  and 1 are different.

 The set of all strings over an alphabet is denoted by *.
* = 0 1 2 ……
Sometimes it is useful to exclude the empty string from the set of
strings. The set of nonempty strings from the alphabet is denoted
by +.
 To concatenate strings, we put them right next to one another.
 If x and y are strings, where x=001 and y=111 then xy = 001111
 A set of strings all of which are chosen from some * is called a
language.
If  is an alphabet and L is a subset of * then L is a language
over .
Note that a language need not include all strings in *.

 The language of all strings consisting of n 0’s followed by n 1’s, for

some n0: { ε, 01, 0011, 000111, …}
 Ø is the empty language, which is a language over any alphabet.
 {ε} is the language consisting of only the empty string. Note that
this is not the same as example #2, the former has no strings and the
latter has one string.
In automata, the grammars are formal systems for describing the
structure of languages. In grammar, there are set of rules for
generating valid strings in a language.
Formally, we can define grammar like this. A grammar is a
tuple G = (V, Σ, R, S), where:

 V is a finite set of variables (non-terminal symbols)

 Σ is a finite set of terminal symbols (the alphabet)

 R is a finite set of production rules

 S is the start symbol (S V)

Grammars are used to generate all valid strings in a language, it also
provides a structural description of the language and serve as a basis for
parsing and syntax analysis. Let us see the following table to
understand different components of a grammar clearly.
Component Description Example
Non-terminal
Variables (V, N) A, B, C
symbols
Terminals Symbols in the
a, b, c, 0, 1
(T or ∑) alphabet
Rules for string
Production rules A → aB, B → bC
generation
Initial variable for
Start symbol S
derivations
({S, A, B}, {a, b}, S, {S → AB, A → a,
Example
B → b})
Regular language (RL) is a type of formal language in theoretical
computer science and automata theory that can be expressed using a
Regular Expression(RE) and recognized by a Finite Automaton.
Characteristics of Regular Languages:
 1. Recognizable by Finite Automata(FA): RL can be recognized by
a FA, which is a machine with a finite number of states.
 2. Expressible Using RE: These languages can be defined using REs,
which describe patterns of strings.
 3. Closure Properties: Regular languages are closed under
operations like Union, Concatenation, Intersection, Complement
 4. Pumping Lemma: The Pumping lemma provides a way to prove
whether a language is not regular by identifying inconsistencies in its
structure.
Examples of Regular Languages:
◦ 1. Strings over the alphabet `{0, 1}` that contain an even number of
`0`s.
◦ 2. Strings that start with `a` and end with `b` over the alphabet `{a,
b}`.
◦ 3. Strings that match the regular expression `(ab)*`, representing
zero or more occurrences of "ab".
A notation used to specify all the strings that belong to a language
Components of RE
 Dot Operator (`.`):The `.` (dot) operator represents **any single
symbol from the alphabet.
◦ if your alphabet is ∑= {a, b}, then `.` matches either `a` or `b`.
 Round Braces (`()`) :Parentheses are used for grouping expressions,
ensuring that the operations inside them are treated as a single unit.
◦ `(a|b)c` matches either `ac` or `bc`.
◦ `(ab)*` matches any number of repetitions of the string `ab`.
 Union Operator (`|`) :As mentioned before, the `|` operator denotes
a choice between options.
◦ `a|b` matches either `a` or `b`.
 4. Concatenation: Writing two symbols or expressions together
implies concatenation, where one directly follows the other.
◦ `ab` matches the string `ab`.
 Kleene Star (`*`) : Represents zero or more repetitions of the
preceding element.
◦ `a*` matches `ε`, `a`, `aa`, `aaa`, etc.
◦ `(ab)*` matches `ε`, `ab`, `abab`, `ababab`, etc.
 Kleene Plus (`+`) : Similar to `*`, but requires at least one occurrence
of the preceding element.
 Empty String (`ε`): Represents the empty string, a valid element in
regular expressions when nothing is matched.
 Square Brackets (`[]`) : Denote a set or range of symbols.
◦ `[a, b]` matches either `a` or `b`.
◦ `[a-z]` matches any lowercase letter from `a` to `z`.
 Optional (`?`) :Indicates that the preceding element is optional (can
appear zero or once).
◦ `a?b` matches either `b` or `ab`.
 Caret (^):When placed at the beginning of a regular expression, ^
asserts that the match must occur at the start of the string.
^a matches any string that starts with a, such as apple but not
banana.
 Curly Braces ({}):Curly braces are used to specify the exact number
of repetitions of the preceding element
a{3} matches exactly three as, so it matches aaa.
a{2,5} matches between two and five as, so it matches aa, aaa, aaaa,
and aaaaa.
 Backslash (\):The backslash is used as an escape character to give
special meaning or to match special characters literally.
\d matches any digit (0-9).
\w matches any word character (alphanumeric and underscore).
\s matches any whitespace character (spaces, tabs, line breaks).
To match a literal dot, you would use \. because . alone matches
any character.
 Dollar Sign ($):The dollar sign asserts position at the end of a
string.
a$ matches any string that ends with a, so it matches pizza and
pasta but not apples.
Regular grammar generates regular language. They have a single non-
terminal on the left-hand side and a right-hand side consisting of a
single terminal or single terminal followed by a non-terminal.
The productions must be in the form:
A xB
A x
A Bx
where A, B Variable(V) and x T* i.e. string of terminals.
Types of regular grammar

Left Linear grammar(LLG)

Right linear grammar(RLG)

A CFL is a type of formal language that can be generated by CFG and
recognized by a PDA. CFL play a key role in parsing and programming
language design.
Characteristics of Context-Free Languages.
 Generated by Context-Free Grammar (CFG)
 Recognized by Pushdown Automaton (PDA)

 Hierarchical and Recursive Structures

 Closure Properties
Union, Concatenation,
They are not closed under intersection or complementation.
Examples of Context-Free Languages
 Palindromes: Strings that read the same forwards and backwards.

CFG: S → aSa | bSb | a | b | ε

 Matched `a`s and `b`s: {anbn n≥0}

CFG: S → aSb | ε
A grammar is said to be the Context-free grammar if every production is
in the form of: G -> (V T)*, where G V

 V (Variables/Non-terminals): These are symbols that can be replaced

using production rules. (e.g., S, A, B).
 T (Terminals): These are symbols that appear in the final strings of the
language and cannot be replaced further. (e.g., a, b, c).
 The left-hand side can only be a Variable, it cannot be a terminal.

 But on the right-hand side here it can be a Variable or Terminal or

both combination of Variable and Terminal.
The above equation states that every production which contains any
combination of the ‘V’ variable or ‘T’ terminal is said to be a context-
free grammar.
A context-sensitive language (CSL) It is defined by a context-
sensitive grammar (CSG) and can be recognized by a linear bounded
automaton (LBA), which is a restricted form of a Turing machine.
Characteristics of Context-Sensitive Languages
Generated by Context-Sensitive Grammars (CSGs)
Recognized by Linear Bounded Automata (LBA)
Ability to Handle Context
CSL Examples:
: Strings with equal numbers of as, bs, and cs, such as abc,
aabbcc, or aaabbbccc.
:Strings where the second half is identical to the first half, such as
abab, 001001, or xyxy.
Natural Constraints: CSLs can model constraints in real-world
systems that require matching or equal distributions, which context-
free grammars cannot.
A context-sensitive grammar consists of production rules of the form:
 αAβ→αγβ where:
 A is a non-terminal.
 α,β,γ are strings of terminals and/or non-terminals.
 The length of γ is greater than or equal to the length of A
(ensuring non-shrinking derivations).
CSG is a formal grammar in which the left-hand sides and right-
hand sides of any production rules may be surrounded by a
context of terminal and nonterminal symbols.
Grammar Language Automata
Type-0 Recursively enumerable Turing machine
Linear-Bounded
Type-1 Context-sensitive
Automata

Type-2 Context-free Push Down Automata

Type-3 Regular Finite state automata

Regular languages are the simplest class of languages and are defined
by finite automata or regular expressions. While they are very useful in
many contexts, they have notable limitations:
 No Memory for Nested Structures: Regular languages cannot
handle constructs that involve matching nested elements (e.g.,
parentheses or nested function calls). For example, the language {an
bn | n ≥ 0} (equal numbers of as followed by bs) is not regular.
 Limited Expressiveness: They cannot express dependencies between
parts of the input. For example, they fail with languages where
relationships span across input, such as ensuring variable declarations
before use.
 Finite State: Regular languages are recognized by finite state
machines that lack memory or stack mechanisms, making them
unsuitable for contexts requiring more complex state management.
Context-free languages are more powerful than regular languages and
are defined by pushdown automata or context-free grammars. They can
handle nesting and hierarchical structures but still face limitations:
Cannot Represent Cross-Dependencies: Context-free languages
cannot handle languages where two parts must "agree" on values or
lengths. For instance, {an bn cn | n ≥ 0} is not context-free because the
dependency between b and c extends beyond what a single stack can
manage.
Single Stack Constraint: Pushdown automata only have one stack,
limiting their ability to store and manage multiple independent states.
Ambiguity: Many CFLs are ambiguous, meaning a string can have
multiple valid parse trees. Resolving ambiguity often requires extra
work in parsing.
Context-sensitive languages are even more powerful and are defined by
linear-bounded automata. While they overcome the limitations of CFLs,
they too have their challenges:
 Complexity: Parsing context-sensitive languages is computationally
expensive, often requiring exponential time in the worst case. This
makes them impractical for many applications.
 Memory Bounds: Linear-bounded automata must operate within
memory proportional to the input size, which imposes constraints on
certain computations.
 Difficult Grammar Design: Designing context-sensitive grammars
for languages is far more challenging than for regular or context-free
grammars
 Language Class Strength Limitation
Fast, efficient for Cannot handle nesting or
Regular Languages
simple syntax dependencies
Fails with complex
Context-Free Handles nested
dependencies or cross-
Languages structures
referencing
Computationally
Context-Sensitive Handles more
expensive, hard to
Languages complex structures
implement and design