Scribd
Upload
8 views

parsing

Lexical analysis converts source code into tokens for easier processing by compilers and interpreters, and is utilized in text editors, code analysis tools, and natural language processing. Parsing, the next step, analyzes the structure of these tokens to ensure grammatical correctness and creates a parse tree for further processing. There are two main types of parsing: top-down and bottom-up, each with distinct methods for constructing the parse tree.

Uploaded by

Sathyavathi Ramkumar
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
8 views

parsing

Lexical analysis converts source code into tokens for easier processing by compilers and interpreters, and is utilized in text editors, code analysis tools, and natural language processing. Parsing, the next step, analyzes the structure of these tokens to ensure grammatical correctness and creates a parse tree for further processing. There are two main types of parsing: top-down and bottom-up, each with distinct methods for constructing the parse tree.

Uploaded by

Sathyavathi Ramkumar
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 12

Lexical Analysis

Lexical analysis is the process of converting a sequence of characters in a


source code file into a sequence of tokens that can be more easily processed
by a compiler or interpreter. It is often the first phase of the compilation
process and is followed by syntax analysis and semantic analysis.
Applications of Lexical Analysis:
•Compilers: Lexical analysis is an important part of the compilation process,
as it converts the source code of a program into a stream of tokens that can
be more easily processed by the compiler.

•Interpreters: Lexical analysis is also used in interpreters, which execute a


program directly from its source code without the need for compilation.

•Text editors: Many text editors use lexical analysis to highlight keywords
and other elements of the source code in different colors, making it easier for
programmers to read and understand the code.
Contd.
• Code analysis tools: Lexical analysis is used by tools that
analyze the source code of a program for errors, security
vulnerabilities, and other issues.

• Natural language processing: Lexical analysis is also used


in natural language processing (NLP) to break down natural
language text into individual words and phrases that can be
more easily processed by NLP algorithms.

• Information retrieval: Lexical analysis is used in information


retrieval systems, such as search engines, to index and search
for documents based on the words they contain.
PARSING
• In Natural Language Processing (NLP), parsing analyzes
the structure of text (or speech) to understand its
grammatical elements and relationships, aiding in tasks
like machine translation and text summarization.
Contd..
Parsing, also known as syntactic analysis, is the process of
analyzing a sequence of tokens to determine the grammatical
structure of a program. It takes the stream of tokens, which are
generated by a lexical analyzer or tokenizer, and organizes them
into a parse tree or syntax tree.
• The parse tree visually represents how the tokens fit together
according to the rules of the language’s syntax. This tree structure
is crucial for understanding the program’s structure and helps in
the next stages of processing, such as code generation or
execution. Additionally, parsing ensures that the sequence of
tokens follows the syntactic rules of the programming language,
making the program valid and ready for further analysis or
execution.
What is the Role of Parser?
• A parser performs syntactic and semantic analysis of source code,
converting it into an intermediate representation while detecting and
handling errors.

• Context-free syntax analysis: The parser checks if the structure of the code
follows the basic rules of the programming language (like grammar rules).
It looks at how words and symbols are arranged.
• Guides context-sensitive analysis: It helps with deeper checks that depend
on the meaning of the code, like making sure variables are used correctly.
For example, it ensures that a variable used in a mathematical operation,
like x + 2, is a number and not text.
• Constructs an intermediate representation: The parser creates a simpler
version of your code that’s easier for the computer to understand and work
with.
• Produces meaningful error messages: If there’s something wrong in your
code, the parser tries to explain the problem clearly so you can fix it.
• Attempts error correction: Sometimes, the parser tries to fix small mistakes
in your code so it can keep working without breaking completely.
Types of Parsing

The parsing is divided into two types, which are as


follows:
• Top-down Parsing

• Bottom-up Parsing
Top-Down Parsing

Top-down parsing is a method of building a parse tree from the start symbol
(root) down to the leaves (end symbols). The parser begins with the highest-
level rule and works its way down, trying to match the input string step by step.
• Process: The parser starts with the start symbol and looks for rules that can
help it rewrite this symbol. It keeps breaking down the symbols (non-terminals)
into smaller parts until it matches the input string.

• Leftmost Derivation: In top-down parsing, the parser always chooses the


leftmost non-terminal to expand first, following what is called leftmost
derivation. This means the parser works on the left side of the string before
moving to the right.

• Other Names: Top-down parsing is sometimes called recursive parsing or


predictive parsing. It is called recursive because it often uses recursive
functions to process the symbols.
Bottom-Up Parsing
• Bottom-up parsing is a method of building a parse tree starting
from the leaf nodes (the input symbols) and working towards the
root node (the start symbol). The goal is to reduce the input string
step by step until we reach the start symbol, which represents the
entire language.
• Process: The parser begins with the input symbols and looks for
patterns that can be reduced to non-terminals based on the
grammar rules. It keeps reducing parts of the string until it forms
the start symbol.

• Rightmost Derivation in Reverse: In bottom-up parsing, the


parser traces the rightmost derivation of the string but works
backwards, starting from the input string and moving towards the
start symbol.

• Shift-Reduce Parsing: Bottom-up parsers are often called shift-


1. LR parsing/Shift Reduce
Parsing:
Shift reduce Parsing is a process of parsing a string to
obtain the start symbol of the grammar.
• LR(0)

• SLR(1)

• LALR

• CLR
2. Operator Precedence
Parsing:
• The grammar defined using operator grammar is known
as operator precedence parsing. In
operator precedence parsing there should be no null
production and two non-terminals should not be
adjacent to each other.

You might also like