Automata: The Methods and The

Madness
Course: Theory of Computation (CSE 2205)

Prepared By
Noshin Tahsin
Lecturer, Department of Computer Science and Engineering
BSMRU, Kishoreganj
Course Contents
CSE 2205: Theory of Computation (3 Credits)
Language theory; Finite automata: deterministic finite automata, nondeterministic
finite automata, equivalence and conversion of deterministic and nondeterministic
finite automata, pushdown automata; Context-free languages; Context-free
grammars; Turing Machines: basic machines, configuration, computing with Turing
machines, combining Turing machines; Undecidability.
Course Contents
Language Theory
● What It Means: Imagine languages like English or Bangla but for computers.
Language theory is about creating "rules" that computers understand, just like
grammar helps us write correct sentences.
● Why It’s Useful: These rules let us write programs, talk to computers, and
make sure computers respond correctly.
● Real-Life Example: When you type a search query in Google or click buttons
in apps, language rules ensure the system understands your instructions.
Course Contents
Language Theory (A More Formal Definition)
● Implications: Language theory defines how formal languages (sets of strings)
are structured and how they can be represented. This underpins many areas
of computer science, including programming languages and compilers.

● Applications:

○ Designing and analyzing programming languages.

○ Defining protocols for data communication (e.g., XML, JSON).

○ Building compilers and interpreters to translate high-level code into

machine code.
Course Contents
Finite Automata (FA)

● What It Means: Think of FA like a very basic robot. It follows simple

instructions to check if something matches a pattern.
● Why It’s Useful: FA helps computers recognize patterns or simple commands.
● Real-Life Example:
○ When you use Ctrl+F to find a word in a document.
○ Password validation on websites.
Course Contents
Finite Automata (A More Formal Definition)

Implications: Finite automata are simple models of computation that help us

understand pattern recognition and string processing. DFAs and NFAs demonstrate
how simple machines can solve regular problems.

Applications:

● Implementing lexical analyzers in compilers (e.g., recognizing keywords in

source code).
● Designing efficient pattern-matching algorithms (e.g., regular expressions in
text editors, search engines).
● Network protocol validation (e.g., HTTP requests).
● Analyzing and modeling sequential circuits in hardware.
Course Contents
Pushdown Automata (PDA)

● What It Means: Imagine a robot that can also remember things using a sticky note
(called a stack). PDAs can handle more complex tasks that need memory.
● Why It’s Useful: They help computers understand structures like nested
parentheses or matching HTML tags.
● Real-Life Example:
○ When your browser checks if all the <title> and </title> tags are
correctly paired on a web page.
○ When your calculator processes operations like (3 + 5) * 2.
Course Contents
Pushdown Automata (A More Formal Definition)
Implications: Pushdown automata extend finite automata by using a stack, making them
powerful enough to recognize context-free languages. This models computations that
require memory for nested structures.

Applications:

● Parsing programming languages and constructing syntax trees in compilers.

● Modeling natural language grammar for speech processing.
● Validating nested structures like parentheses in code or HTML/XML tags.
Course Contents
Context-Free Languages (CFLs) and Grammars

● What It Means: CFLs are the "grammar rules" for programming languages. They
describe how programs should be written so the computer can understand them.
● Why It’s Useful: They form the backbone of programming languages like Python,
Java, or C++.
● Real-Life Example:
○ How compilers and interpreters understand and run your code.
○ When your IDE (like VS Code) shows errors in your code as you type.
Course Contents
Context-Free Languages (CFLs) and Grammars (A more formal definition)

Implications: CFLs are the foundation for describing the syntax of most programming
languages. Context-free grammars define the rules for structuring statements and
expressions in languages.

Applications:

● Building parsers for programming languages.

● Generating compilers for various programming languages.
● Implementing auto-completion and syntax checking in modern IDEs.
● Modeling and analyzing query languages like SQL.
Course Contents
Turing Machines (TM)

● What It Means: A Turing Machine is like the smartest robot ever imagined—it can do
any calculation or task if it’s possible to do. It’s a model of how computers work.
● Why It’s Useful: It helps us understand what problems computers can solve and
what’s impossible for them.
● Real-Life Example:
○ Every computer or smartphone is inspired by the idea of a Turing Machine.
○ Helps us understand the limits of apps and programs, like why we can’t make
an app to predict the future perfectly.
Course Contents
Turing Machines (TM) (A more formal definition)
Implications: Turing Machines are the most powerful model of computation, capable of
simulating any computation. They are the theoretical basis for understanding what
computers can do and the concept of "algorithm".

Applications:

● Evaluating the computational complexity of algorithms (e.g., what can and cannot be
computed efficiently).
● Modeling and proving the correctness of software systems.
● Designing advanced algorithms and understanding their limits.
● Building the foundation for research in artificial intelligence, machine learning, and
cryptography.
Course Contents
Undecidability

● What It Means: Some problems are like puzzles that no computer can ever solve,
no matter how smart or fast it is.
● Why It’s Useful: Knowing these limits helps us avoid wasting time trying to solve the
impossible and focus on things that can actually be done.
● Real-Life Example:
○ Proving whether a program will run forever or stop can’t always be solved—it’s
called the "Halting Problem."
○ Helps in cryptography to make systems more secure.
Course Contents
What Is the Halting Problem?

● The Halting Problem is a classic question in computer science that asks:

● Given a program and an input, can you determine whether the program will
eventually stop (halt) or run forever?
● In 1936, Alan Turing proved that there is no general algorithm that can solve this
problem for all possible programs and inputs. This means that for some programs, it
is undecidable whether they will halt or continue running indefinitely.

Why This Is Mind-Blowing?

Turing’s proof shows there are limits to what computers can do. No matter how powerful a
computer is, some problems are fundamentally unsolvable. This idea laid the foundation
for modern computer science and our understanding of computation.
Course Contents
How Does Undecidability Help in Cryptography?
Creates Problems That Are Impossible to Reverse

Cryptographic systems rely on problems that are either undecidable or so hard to compute
that they seem impossible to solve within a reasonable amount of time.

● Example: Imagine you lock your data with a code (encryption). The only way to
unlock it (decryption) is by using a specific key. An attacker would need to try every
possible key, which could take millions of years. This makes it practically
impossible to break.
Course Contents
How Does Undecidability Help in Cryptography?

One-Way Functions

A one-way function is easy to compute in one direction but practically impossible to reverse.

● Example: Hashing
○ Hashing is like turning a piece of data (e.g., a password) into a scrambled number or
string (e.g., abc123 → E99A18C428CB38D5F260853678922E03).
○ It’s easy to hash a password but almost impossible to figure out the original password
from the hash. Unlike encryption, there’s no key to "unlock" the hash. To find the
original input, you would have to try every possible combination until you find one that
produces the same hash—a process known as a brute-force attack.

These functions work because reversing them is undecidable or computationally infeasible.

Course Contents
How Does Undecidability Help in Cryptography?
Protects Against Brute-Force Attacks

Brute force means trying every possible solution until the right one is found. For many
cryptographic problems:

● Even if it’s technically solvable (not undecidable), it would take so long to compute
(thousands of years) that it’s effectively impossible. This concept is inspired by
undecidability.
Course Contents
How Does Undecidability Help in Cryptography?
Unbreakable Foundations

Cryptographic systems often rely on problems rooted in undecidability or computational

infeasibility:

● RSA Encryption: Based on the difficulty of factoring very large numbers.

Course Contents
Undecidability (A more formal definition)
Implications: Some problems cannot be solved by any algorithm or machine,
regardless of time or resources. Recognizing undecidability prevents wasted effort
on unsolvable problems.
Applications:
● Identifying unsolvable problems in software verification (e.g., the Halting
Problem).
● Understanding limits in areas like automated theorem proving, formal
verification, and AI.
● Designing approximate or heuristic solutions for problems where exact
solutions are undecidable.
Course Contents
What Is an Approximate or Heuristic Solution?

Imagine you’re solving a really tough problem, like finding the shortest route to visit 100
cities. Solving it perfectly might take forever, so instead, you come up with a "good
enough" solution that works quickly. That’s an approximate solution or a heuristic
solution.

It’s like saying:

“I don’t need the absolute best answer; I just need one that’s close enough and works for
me right now.”
Why Do We Need Them?

Some Problems Are Just Too Hard:

● For some problems, like deciding the shortest route between many cities, finding
the perfect solution might take years with even the best computers.
● Instead, we find a solution that’s close to the best in seconds or minutes.

Time Is Limited:

● In the real world, you don’t always have the luxury of waiting. For example:
○ GPS systems don’t calculate the perfect route—they calculate a good route
fast enough for you to use.

We Don’t Always Need Perfection:

● Sometimes, a solution doesn’t need to be perfect—just good enough to solve the

problem practically.
Approximate or Heuristic Solution

Examples You Can Relate To

Finding a Shortest Route:

● Let’s say you’re delivering pizzas to 10 locations. Calculating the absolutely shortest route could
take forever, so instead, you use a rule of thumb:
○ Visit the closest house next each time.
○ This might not be the shortest route, but it’s good enough and gets the job done!

Playing a Game:

● In chess or strategy games, your computer opponent doesn’t calculate every possible move (it
would take too long). Instead, it looks a few moves ahead and picks what seems to be the best
choice. This is a heuristic.
Approximate or Heuristic Solution

How Do They Work?

Heuristic solutions use shortcuts or rules of thumb to find an answer faster:

1. Greedy Approach: Always pick the best option at each step.

○ Example: When delivering pizzas, go to the nearest house next.
2. Guess and Improve:
○ Example: Start with a random solution, then keep tweaking it to make it better.

Approximate or heuristic solutions are like smart shortcuts: they don’t give perfect
answers, but they’re good enough and much faster. In computer science and real life,
they’re super useful for solving problems efficiently when perfection isn’t necessary and
when you need answers quickly and can’t afford to spend forever calculating.
Why These Matter to You as a Computer Science Student

Building Better Software: These concepts help you design tools like
programming languages, text editors, and compilers that let other people create
amazing software.

Understanding How Computers Think: You’ll learn how computers "decide" what
to do, from solving a math problem to running your favorite app.

Pushing the Limits of Technology: By understanding what’s possible and

impossible, you’ll know where to innovate and where to create alternatives.
Why These Matter to You as a Computer Science Student

Think of this course as learning the blueprints of how computers and software
are built. Just like an architect needs to understand the physics of
buildings, you need these theories to become a computer scientist who can build
reliable and efficient systems so that by the end of this course, you’ll not only
understand how these ideas shape computing but also feel confident in applying
them to solve real-world problems!
What is Automata Theory

Automata theory is the study of abstract computing devices or

machines

Abstract computing devices or machines are simplified, theoretical models of

computation that represent how computers and other systems process information. These
are not physical machines but mathematical models that help us understand the
principles of computation.

● They are not real-world computers like your laptop or phone.

● They are simplified models that strip away unnecessary details, focusing only on
the essential aspects of computation.
● Their goal is to study what is computationally possible and classify problems based
on their computational complexity.
Examples of Abstract Computing Devices
❏ Finite Automata (FA): A simple model of computation with limited memory. Used to recognize regular
languages. Example: Think of a vending machine. It processes a sequence of inputs (like coins) and
makes decisions (like dispensing snacks) based on its current state.

❏ Pushdown Automata (PDA): An automaton with a stack as its memory. Used to recognize
context-free languages, such as those used in programming languages. Example: Think of
balancing parentheses in code: A stack is used to keep track of opening and closing parentheses.

❏ Turing Machines: A more powerful model of computation that can simulate any algorithm. Used to
define what is computable (anything a computer can compute). Example: Imagine an infinite piece of
paper where a machine writes, reads, and erases symbols. This machine can represent any computer
program.

❏ Linear Bounded Automata (LBA): A Turing machine with a restricted amount of memory. Used to
recognize context-sensitive languages.

❏ Lambda Calculus: A mathematical model for defining computation using functions. Forms the basis
for functional programming languages like Haskell.
Alan Turing
English mathematician, computer
scientist, logician, cryptanalyst,
philosopher, and theoretical biologist.

● Before there were computers in the 1930’s, A.

Turing studied an abstract machine that had
all the capabilities of todays computers, at
least as far as in what they could compute

● Turings goal was to describe precisely the

boundary between what a computing machine
could do and what it could not do

● His conclusions apply not only to his abstract

Turing machines but to todays real machines
The Imitation Game
● A biographical drama about the life and work of Alan
Turing that highlights Turing's crucial role in breaking
the German Enigma code during World War II and
his contributions to modern computing

● Alan Turing, recruited by the British government,

joins a team of cryptographers at Bletchley Park, the
British codebreaking center. Their mission was to
crack the Enigma code, an encryption system used
by the German military to send classified messages.
The code was considered unbreakable because it
changed daily.

● Turing designs a machine (an early prototype of the

modern computer) called Christopher to automate
the decryption process.

“Only Machine Can Beat Machine”

The Imitation Game

“Only Machine Can Beat Machine”

Alan Turing
English mathematician, computer scientist,
logician, cryptanalyst, philosopher, and
theoretical biologist.

● Before there were computers in the 1930’s, A.

Turing studied an abstract machine that had
all the capabilities of todays computers, at
least as far as in what they could compute

● Turings goal was to describe precisely the

boundary between what a computing machine
could do and what it could not do

● His conclusions apply not only to his abstract

Turing machines but to todays real machines
Is Breaking the Enigma Code Related to Automata?
Yes, breaking the Enigma code is indirectly related to automata theory, as it involves concepts of
computation, algorithm design, and pattern recognition, which are foundational to automata.

The Challenge:

● The German Enigma machine was an encryption device that scrambled messages using a
complex system of rotors and plugboard settings.
● The settings changed daily, making it nearly impossible to decode messages manually.

Turing’s Approach:

● Turing and his team at Bletchley Park designed a machine called the Bombe to automate the
process of breaking the code.
● The Bombe used an algorithmic approach to test possible rotor settings systematically,
significantly speeding up the decryption process.
Is Breaking the Enigma Code Related to Automata?
Relation to Automata:

● The Bombe was a mechanical automaton that followed predefined steps to solve a
problem. This idea aligns with automata theory, which studies machines (abstract or
physical) that follow rules to compute solutions.

● While not a direct application of automata theory as we know it today, the Bombe and its
algorithms were precursors to the concepts of finite state machines and sequential
logic, which are part of automata.
Is the Turing Test Related to Automata?
❏ The Turing Test: Proposed by Alan Turing in 1950, the test evaluates whether a
machine can imitate human intelligence so convincingly that a human evaluator cannot
reliably distinguish the machine from another human during a conversation.

❏ Conceptually related to automata theory because it deals with the capabilities of

machines, but it is more focused on artificial intelligence (AI) and the question of
whether a machine can exhibit intelligent behavior indistinguishable from a human.

❏ Breaking the Enigma code is related to automata theory because it used systematic
computation (an automaton-like machine) to solve a problem.

❏ The Turing Test goes beyond automata by exploring whether a machine can exhibit
human-like intelligence.
Formal Grammars, Brain Functions,
and Chomsky
❏ In the 1940’s and 1950’s simpler kinds of
machines, which we today call finite automata,
were studied by a number of researchers,
originally proposed to model brain function

❏ In the late 1950’s, the linguist Noam Chomsky

began the study of formal grammars. While not
strictly machines, these grammars have close
relationships to abstract automata and serve American professor and public intellectual known for his
work in linguistics, political activism, and social criticism.
today as the basis of some important software
Sometimes called "the father of modern linguistics",
components, including parts of compilers
Chomsky is also a major figure in analytic philosophy and
one of the founders of the field of cognitive science. He is a
laureate professor of linguistics at the University of Arizona
and an institute professor emeritus at the Massachusetts
Institute of Technology (MIT)
Introduction to Finite Automata

❏ In 1969, Stephen Cook extended Turing's study of what could and what could not
be computed.

❏ Cook was able to separate those problems that can be solved efficiently by
computer from those that can, in principle, be solved but, in practice, take so
much time that computers are useless for all but very small instances of the
problem. The latter class of problems is called intractable or NP-hard.

❏ It is highly unlikely that even the exponential improvement in computing speed that
computer hardware has been following (Moores Law) will have significant impact on
our ability to solve large instances of intractable problems
Task
❏ Difference between NP, NP-Complete, and NP-Hard problems with example.
❏ Intractable problems with examples.
❏ Difference between the language in automata theory and programming languages

Write a short two-page report on the above topics.

Deadline: 24 November 2024 (Sunday)
Introduction to Finite Automata
❏ There are many systems or components that may be viewed as being at all times in
one of a finite number of states

❏ The purpose of a state is to remember the relevant portion of the systems history

❏ Since there are only a finite number of states, the entire history generally cannot be
remembered so the system must be designed carefully to remember what is
important and forget what is not

❏ The advantage of having only a finite number of states is that we can implement the
system with a fixed set of resources.

❏ For example, we could implement it in hardware as a circuit or as a simple form of

program that can make decisions looking only at a limited amount of data or using the
position in the code itself to make the decision
Introduction to Finite Automata
Introduction to Finite Automata

● As for all finite automata, the states are represented by circles.

● Arcs between states are labeled by inputs, which represent external influences on
the system
● Here both arcs are labeled by the input Push which represents a user pushing the
button The intent of the two arcs is that whichever state the system is in, when the
Push input is received, it goes to the other state
Introduction to Finite Automata

● One of the states is designated the start state, the state in which the system is
placed initially
● In our example, the start state is off and we conventionally indicate the start state
by the word Start and an arrow leading to that state
Introduction to Finite Automata
● It is often necessary to indicate one or more states as final or accepting states.
Entering one of these states after a sequence of inputs indicates that the input
sequence is good in some way.
● For instance, we could have regarded the state on in Fig 1.1 as accepting because
in that state the device being controlled by the switch will operate.
● It is conventional to designate accepting states by a double circle, although we
have not made any such designation in Fig 1.1.
Introduction to Finite Automata
Example 1.2 Sometimes what is remembered by a state can be much more complex than an ono
choice. Figure 1.2 shows another finite automaton that could be part of a lexical analyzer. The job of this
automaton is to recognize the keyword, then. It thus needs five states, each of which represents a
different position in the word then that has been reached so far. These positions correspond to the
prefixes of the word, ranging from the empty string (i.e., nothing of the word has been seen so far) to
the complete word
Introduction to Finite Automata
The five states are named by the prefix of then seen so far. Inputs correspond to letters. We may
imagine that the lexical analyzer examines one character of the program that it is compiling at a time
and the next character to be examined is the input to the automaton. The start state corresponds to the
empty string and each state has a transition on the next letter of then to the state that corresponds to
the next larger prefix. The state named then is entered when the input has spelled the word then.

Since it is the job of this automaton to recognize when then has been seen, we could consider that
state the lone accepting state.
Structural Representations
❏ Grammars: Formal systems used to define languages, which are sets of strings
(sequences of symbols) over a given alphabet.
❏ Regular Expressions: Denote the structure of data, especially text strings

[A-Z][a-z]*[][A-Z][A-Z]: Represents capitalized words followed by a

space and two capital letters. This expression represents patterns in text that
could be a city and state, e.g., Ithaca NY.

It misses multiword city names, such as Palo Alto CA, which could be
captured by the more complex expression
[A-Z][a-z]*([][A-Z][a-z]*)*[][A-Z][A-Z]
Regular Expressions
● When interpreting such expressions, we only need to know that [A-Z] represents a
range of characters from capital A to capital Z, i.e., any capital letter, and [ ] is used
to represent the blank character alone.
● The * symbol represents any number of the preceding expression.
● Parentheses are used to group components of the expression; they do not
represent characters of the text described.
The Central Concepts of Automata Theory
From the textbook
The Central Concepts of Automata Theory
Problems as Decision Questions (Yes/No Questions):

In complexity theory, problems are typically defined as decision problems: You are asked
whether a given input belongs to a particular language (a set of valid strings).

Real-World Tasks Are Often More Complex:

● Real-world tasks (like compiling a C program) involve more than just deciding "yes"
or "no."
● Example: A C compiler not only checks if a program is valid but also generates a
parse tree, populates a symbol table, and produces object code.
● These tasks are transformative: they involve producing something (e.g., object
code) rather than just answering a question
The Central Concepts of Automata Theory
Decision Problems vs. Practical Problems:

● Complexity theory focuses on decision problems because they allow us to study

computational limits systematically.

● Remarkably, if the decision problem (e.g., "Is this a valid C program?") is hard, the
corresponding practical problem (e.g., "Compile this C program into object code") is
also hard.

● This connection is established through a concept called reductions.

○ A reduction is a technique to show that one problem is as hard as another.
○ If we can solve Problem A efficiently, and Problem B can be transformed into
Problem A, then Problem B must also be solvable efficiently.
Why Define Problems as Languages?
Simplicity in Complexity Theory:

● Defining problems as "questions about membership in a language" simplifies the

analysis of computational complexity.
● It allows us to focus on proving whether problems can or cannot be solved within
certain time limits (e.g., polynomial time).

Hardness Applies to Both Forms:

● The decision version (e.g., "Is this string valid?") and the transformative version
(e.g., "Generate object code") are equally hard.