LEXICAL

ANALYZER
(Part 3)
COMPILATION SEQUENCE
BUILDING A COMPILER WITH
LEX/YACC
THE LEX AND FLEX
SCANNER GENERATORS
 Lex and its newer cousin flex are scanner generators
 Systematically translate regular definitions into C source code for efficient
scanning
 Generated code is easy to integrate in C applications
 Follow the following link for downloading and installing FLEX tool:
 https://www.youtube.com/watch?v=ilwXAchl4uw
 https://simran2607.medium.com/compiler-design-using-flex-and-bison-in-
windows-a9642ebd0a43
CREATING A LEXICAL
ANALYZER WITH LEX AND
FLEX
lex
source lex or flex
lex.yy.c
program compiler
lex.l

C
lex.yy.c a.exe
compiler

input sequence
a.exe
stream of tokens
FINITE STATE AUTOMATON
(WITH CODE)
PATTERN MATCHING
PRIMTIVES (IN LEX)
LEX SPECIFICATION
LEX SPECIFICATION
(CONTD.)
 A lex specification consists of three parts:
regular definitions, C declarations in %{ %}
%%
translation rules
%%
user-defined auxiliary procedures
 The translation rules are of the form:
p1 { action1 }
p2 { action2 }
…
pn { actionn }
LEX PREDEFINED VARIABLES
REGULAR EXPRESSIONS IN
LEX x
\.
match the character x
match the character .
“string” match contents of string of characters
. match any character except newline
^ match beginning of a line
$ match the end of a line
[xyz] match one character x, y, or z (use \ to escape -)
[^xyz]match any character except x, y, and z
[a-z] match one of a to z
r* closure (match zero or more occurrences)
r+ positive closure (match one or more occurrences)
r? optional (match zero or one occurrence)
r1r2 match r1 then r2 (concatenation)
r1|r2 match r1 or r2 (union)
( r ) grouping
r1\r2 match r1 when followed by r2
{d} match the regular expression defined by d
%{
int n = 0 ;
%}
%%

"while"|"if"|"else" {n++;printf("\t keywords : %s", yytext);}

"int"|"float" {n++;printf("\t keywords : %s", yytext);}

[a-zA-Z_][a-zA-Z0-9_]* {n++;printf("\t identifier : %s", yytext);}
"<="|"=="|"="|"++"|"-"|"*"|"+" {n++;printf("\t operator : %s",
yytext);}
[(){}|, ;] {n++;printf("\t separator : %s", yytext);}
[0-9]*"."[0-9]+ {n++;printf("\t float : %s", yytext);}
[0-9]+ {n++;printf("\t integer : %s", yytext);}
. ;
%%
int yywrap()
{
return 1;
}
int main()
{

yylex();

printf("\n total no. of token = %d\n", n);

}
EXAMPLE LEX
SPECIFICATION 1
Contains
the matching
%{ lexeme
Translation #include <stdio.h>
rules %}
%%
[0-9]+ { printf(“%s\n”, yytext); }
.|\n { } Invokes
%% the lexical
main() analyzer
{ yylex();
}

lex spec.l
gcc lex.yy.c
a.exe
EXAMPLE LEX
SPECIFICATION 2
%{
#include <stdio.h> Regular
int ch = 0, wd = 0, nl = 0; definition
Translation %}
rules delim [ \t]+
%%
\n { ch++; wd++; nl++; }
^{delim} { ch+=yyleng; }
{delim} { ch+=yyleng; wd++; }
. { ch++; }
%%
main()
{ yylex();
printf("%8d%8d%8d\n", nl, wd, ch);
}
EXAMPLE LEX
SPECIFICATION 3
%{
#include <stdio.h> Regular
%} definitions
Translation digit [0-9]
rules letter [A-Za-z]
id {letter}({letter}|{digit})*
%%
{digit}+ { printf(“number: %s\n”, yytext); }
{id} { printf(“ident: %s\n”, yytext); }
. { printf(“other: %s\n”, yytext); }
%%
main()
{ yylex();
}
EXAMPLE LEX
SPECIFICATION 4
%{ /* definitions of manifest constants */
#define LT (256)
…
%}
delim [ \t\n]
ws {delim}+
letter [A-Za-z] Return
digit [0-9] token to
id {letter}({letter}|{digit})* parser
number {digit}+(\.{digit}+)?(E[+\-]?{digit}+)?
%%
{ws} { } Token
if {return IF;}
then {return THEN;} attribute
else {return ELSE;}
{id} {yylval = install_id(); return ID;}
{number} {yylval = install_num(); return NUMBER;}
“<“ {yylval = LT; return RELOP;}
“<=“ {yylval = LE; return RELOP;}
“=“ {yylval = EQ; return RELOP;}
“<>“ {yylval = NE; return RELOP;}
“>“ {yylval = GT; return RELOP;}
“>=“ {yylval = GE; return RELOP;} Install yytext as
%% identifier in symbol table
int install_id()
…
EXAMPLE LEX
SPECIFICATION 5
EXAMPLE LEX
SPECIFICATION 6
THANK
YOU!!!