Scribd
Upload
766 views17 pages

Lexical Analysis & Lex Tool

Uploaded by

growmorewithfaq
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
766 views17 pages

Lexical Analysis & Lex Tool

Uploaded by

growmorewithfaq
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 17

LEXICAL ANALYSIS &

LEX TOOL

Subject : Compiler Design


By
Dr. P. P. Deshmukh
LEXICAL ANALYSIS
 It is the first step of compiler design, it takes the
input as a stream of characters and gives the output
as tokens also known as tokenization. The tokens can
be classified into identifiers, Separators, Keywords,
Operators, Constant and Special Characters.
 It has three phases:
 Tokenization: It takes the stream of characters and
converts it into tokens.
 Error Messages: It gives errors related to lexical
analysis such as exceeding length, unmatched string,
etc.
 Storing identifiers with its attributes in symbol
table.
 Eliminate Comments: Eliminates all the spaces,
blank spaces, new lines, and indentations.
LEX TOOL
 Lex is a tool or a computer program that generates
Lexical Analyzers (converts the stream of characters
into tokens).
 The Lex tool itself is a compiler.

 The Lex compiler takes the input and transforms


that input into input patterns.
 It is commonly used with YACC(Yet Another
Compiler Compiler). It was written by Mike Lesk
and Eric Schmidt. New tool is called flex tool- fast
lexical analyzer.
FUNCTION OF LEX
 In the first step the source code which is in the Lex
language having the file name ‘File.l’ gives as input
to the Lex Compiler commonly known as Lex to get
the output as lex.yy.c.
 After that, the output lex.yy.c will be used as input to
the C compiler which gives the output in the form of
an ‘a.out’ file,
 and finally, the output file a.out will take the stream
of character and generates tokens as output.
 lex.yy.c: It is a C program.
File.l: It is a Lex source program
a.out: It is a Lexical analyzer
1. Compilation of
lex program

2. Building of
compiler

3.Execution of
lexical analyzer
LEX FILE FORMAT
A Lex program consists of three parts and is separated
by %% delimiters:-
Declarations
%%
Translation rules
%%
Auxiliary procedures

 Declarations: The declarations include declarations


of variables. Header files inclusion
 Transition rules: These rules consist of Pattern and
Action.
 Auxiliary procedures: The Auxiliary section holds
auxiliary functions used in the actions. Like main
function
For example:
%{
declaration
#include<stdio.h>
int c=0;
%}
%%
pattern {action}
if {return (IF);}
%%
auxiliary function
int main()
{
}
LEX TOOL RULES
 abc  Match abc
 [a-z]  Match small a to z

 [a-z]*  small letters strings with null or more than


1 characters
 [a-z]+  one or more characters

 [A-Z a-z]+  at least one symbol

 ^a  a should come at start

 a$  a should come at end

 [0-9]+  one or more digit

 [A-za-z0-9]+ One or more alphanumeric characters

 abc*  ababcabccabccc…
LEX PROGRAM IMPORTANT FUNCTIONS

 yywrap() – Called by lex tool when


input is exhuasted , it returns 1 if
input is finished else 0

 yylex()
– Reads the input stream and
generates token according to the
regular expression.

 yytext – pointer to the input string


MORE FUNCTIONS
Name Functions

intyylex(void) call to invoke lexer,returns token

char *yytext pointer to matched string

Yyleng length of matched string

Yylval value associated with token

intyywrap(void) wrapup,return 1 if done,0 if not done

FILE *yyout output file

FILE *yyin input file

INITIAL initial start condition

BEGIN condition switch start condition

ECHO write matched string


LEX TOOL PROGRAMS
%{
#include<stdio.h> // declaration section
%}
%%
“hi” { printf(“Bye”); }
.* { printf(“wrong”);} // Rules
%%
int main()
{
printf(“enter input”); // Auxiliary functions
yylex();
return 0;
}
int yywrap()
{return 1;
}
AIM: IMPLEMENT A LEX PROGRAM TO VERIFY THE PARENTHESIS OF A GIVEN
EXPRESSION IS BALANCED.

%{
int flag=0;
%}
%%
[(] {flag++;}
[)] {flag--;}
[\n] {if(flag==0)
printf("\t Well formed parathensis"); else if(flag>0)
printf("\t Closing parathens missing"); else
printf("\topening parathens missing");
}
%%
int main(void)
{
printf("enter the expression");
yylex();
return(0);
}
int yywrap(void)
{
return 0;
}
int yyerror(void)
{
printf("Error\n"); exit(1);
}
PROGRAMS COUNTS THE NUMBER OF LINES, WORDS AND CHARACTERS IN A
TEXT FILE.

%{
#include<stdio.h>
int lines=0, words=0,s_letters=0,c_letters=0, num=0,
spl_char=0,total=0;
%}
%%
\n { lines++; words++;}
[\t ' '] words++;
[A-Z] c_letters++;
[a-z] s_letters++;
[0-9] num++;
. spl_char++;
%%
main(void)
{
yyin= fopen("myfile.txt","r");
yylex();
total=s_letters+c_letters+num+spl_char;
printf(" This File contains ...");
printf("\n\t%d lines", lines);
printf("\n\t%d words",words);
printf("\n\t%d small letters", s_letters);
printf("\n\t%d capital letters",c_letters);
printf("\n\t%d digits", num);
printf("\n\t%d special characters",spl_char);
printf("\n\tIn total %d characters.\n",total);
}

int yywrap()
{
return(1);
}
OUTPUT
Let the 'myfile.txt' contains this.
This is my 1st lex program!!!
Cheers!! It works!!:)

The output will be


This file contains..
2 lines
9 words
30 small letters
3 capital letters
1 digits
9 special characters
In total 43 characters.
HOW TO RUN LEX PROGRAM

Commands for execution of lex program


1. lex sample.l
2. gcc lex.yy.c
3. ./a.out

You might also like