Scribd Logo
Upload

Welcome to Scribd!

  • 84 views

    How To Create A Recursiver Parser

    Uploaded by

    Bcalh3
    This document discusses recursive descent parsers and how to implement them for LL(1) grammars. It covers: 1) What makes a grammar LL(1) - grammars where there is at most one way to parse the next input symbol. 2) Calculating First and Follow sets to determine lookahead sets and parsing options. 3) Translating an LL(1) grammar into a recursive descent parser by making methods for each production using lookahead sets. 4) Adding evaluation to the parser by changing the methods to return values and calculate expression results.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    How To Create A Recursiver Parser

    Uploaded by

    Bcalh3
    84 views3 pages
    This document discusses recursive descent parsers and how to implement them for LL(1) grammars. It covers: 1) What makes a grammar LL(1) - grammars where there is at most one way to parse the next input symbol. 2) Calculating First and Follow sets to determine lookahead sets and parsing options. 3) Translating an LL(1) grammar into a recursive descent parser by making methods for each production using lookahead sets. 4) Adding evaluation to the parser by changing the methods to return values and calculate expression results.

    Original Description:

    Theory behind programming parsers

    Original Title

    How to create a recursiver parser

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    This document discusses recursive descent parsers and how to implement them for LL(1) grammars. It covers: 1) What makes a grammar LL(1) - grammars where there is at most one way to parse the next input symbol. 2) Calculating First and Follow sets to determine lookahead sets and parsing options. 3) Translating an LL(1) grammar into a recursive descent parser by making methods for each production using lookahead sets. 4) Adding evaluation to the parser by changing the methods to return values and calculate expression results.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    84 views3 pages

    How To Create A Recursiver Parser

    Uploaded by

    Bcalh3
    This document discusses recursive descent parsers and how to implement them for LL(1) grammars. It covers: 1) What makes a grammar LL(1) - grammars where there is at most one way to parse the next input symbol. 2) Calculating First and Follow sets to determine lookahead sets and parsing options. 3) Translating an LL(1) grammar into a recursive descent parser by making methods for each production using lookahead sets. 4) Adding evaluation to the parser by changing the methods to return values and calculate expression results.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 3

    7

    How to implement a recursive descent parser

    A parser is a program which processes input defined by a context-free grammar.


    The translation given in the previous section is not very useful in the design
    of such a program because of the non-determinism. Here I show how for a
    certain class of grammars this non-determinism can be eliminated and using
    the example of arithmetical expressions I will show how a JAVA-program can
    be constructed which parses and evaluates expressions.

    We calculate First and Follow in a similar fashion:


    First(a) = {a} if a .
    If A B1 B2 . . . Bn and there is an i n s.t. 1 k < i.Bk  then
    we add First(Bi ) to First(A).
    And for Follow:

    7.1

    What is a LL(1) grammar ?

    The basic idea of a recursive descent parser is to use the current input symbol
    to decide which alternative to choose. Grammars which have the property that
    it is possible to do this are called LL(1) grammars.
    First we introduce an end marker $, for a given G = (V, , S, P ) we define the
    augmented grammar G$ = (V 0 , 0 , S 0 , P 0 ) where
    / V ,
    V 0 = V {S 0 } where S 0 is chosen s.t. S 0
    0 = {$} where $ is chosen s.t. $
    / V ,
    P 0 = P {S 0 S$}
    The idea is that
    L(G$ ) = {w$ | w L(G)}
    Now for each nonterminal symbol A V 0 0 we define
    First(A) = {a | a A a}
    Follow(A) = {a | a S 0 Aa}
    i.e. First(A) is the set of terminal symbols with which a word derived from A
    may start and Follow(A) is the set of symbols which may occur directly after
    A. We use the augmented grammar to have a marker for the end of the word.
    For each production A P we define the set Lookahead(A ) which
    are the set of symbols which indicate that we are in this alternative.
    [
    Lookahead(A B1 B2 . . . Bn ) = {First(Bi ) | 1 k < i.Bk }
    
    Follow(A) if B1 B2 . . . Bk 

    otherwise
    We now say a grammar G is LL(1), iff for each pair A , A P with
    6= it is the case that Lookahead(A ) Lookahead(A ) =

    7.2

    How to calculate First and Follow

    We have to determine whether A . If there are no -production we know


    that the answer is always negative, otherwise
    If A  P we know that A .
    If A B1 B2 . . . Bn where all Bi are nonterminal symbols and for all
    1 i n: Bi  then we also know A .
    39

    $ Follow(S) where S is the original start symbol.


    If there is a production A B then everything in First() is in
    Follow(B).
    If there is a production A B with  then everything in
    Follow(A) is also in Follow(B).

    7.3

    Constructing an LL(1) grammar

    Lets have a look at the grammar G for arithmetical expressions again. G =


    ({E, T, F }, {(, ), a, +, }, E, P ) where
    P = {E T | E + T
    T F |T F
    F a | (E)
    We dont need the Follow-sets in the moment because the empty word doesnt
    occur in the grammar. For the nonterminal symbols we have
    First(F ) = {a, (}
    First(T ) = {a, (}
    First(E) = {a, (}
    and now it is easy to see that most of the Lookahead-sets agree, e.g.
    Lookahead(E T ) = {a, (}
    Lookahead(E E + T ) = {a, (}
    Lookahead(T F ) = {a, (}
    Lookahead(T T F ) = {a, (}
    Lookahead(F a) = {a}
    Lookahead(F (E)) = {(}
    Hence the grammar G is not LL(1).
    However, luckily there is an alternative grammar G0 which defines the same
    language: G0 = ({E, E 0 , T, T 0 , F }, {(, ), a, +, }, E, P 0 ) where
    P 0 = {E T E 0
    E 0 +T E 0 | 
    T FT0
    T 0 *F T 0 | 
    F a | (E)
    40

    Since we have -productions we do need the Follow-sets.


    First(E) = First(T ) = First(F ) = {a, (}
    First(E 0 ) = {+}
    First(T 0 ) = {*}
    Follow(E) = Follow(E 0 ) = {), $}
    Follow(T ) = Follow(T 0 ) = {+, ), $}
    Follow(F ) = {+, *, ), $}
    Now we calculate the Lookahead-sets:
    Lookahead(E T E 0 ) = {a, (}
    Lookahead(E 0 +T E 0 ) = {+}
    Lookahead(E 0 ) = Follow(E 0 ) = {), $}
    Lookahead(T +F T 0 ) = {a, (}
    Lookahead(T 0 *F T 0 ) = {*}
    Lookahead(T 0 ) = Follow(T 0 ) = {+, ), $}
    Lookahead(F a) = {a}
    Lookahead(F (E)) = {(}

    Hence the grammar G0 is LL(1).

    7.4

    try {
    curr=st.nextToken().intern();
    } catch( NoSuchElementException e) {
    curr=null;
    }
    }
    We also implement a convenience method error(String) to report an error
    and terminate the program.
    Now we can translate all productions into methods using the Lookahead sets to
    determine which alternative to choose. E.g. we translate
    E 0 +T E 0 | 
    into (using E1 for E 0 to follow JAVA rules):
    static void parseE1() {
    if (curr=="+") {
    next();
    parseT();
    parseE1();
    } else if(curr==")" || curr=="$" ) {
    } else {
    error("Unexpected :"+curr);
    }
    The basic idea is to

    How to implement the parser

    We can now implement a parser - one way would be to construct a deterministic


    PDA. However, using JAVA we can implement the parser using recursion - here
    the internal JAVA stack plays the role of the stack of the PDA.
    First of all we have to separate the input into tokens which are the terminal symbols of our grammar. To keep things simple I assume that tokens are separated
    by blanks, i.e. one has to type
    ( a + a ) * a
    for (a+a)*a. This has the advantage that we can use java.util.StringTokenizer.
    In a real implementation tokenizing is usually done by using finite automata.
    I dont want to get lost in java details - in the main program I read a line and
    produce a tokenizer:
    String line=in.readLine();
    st = new StringTokenizer(line+" $");
    The tokenizer st and the current token are static variables. I implement the
    convenience method next which assigns the next token to curr.
    static StringTokenizer st;
    static String curr;

    Translate each occurrence of a non terminal symbol into a test that this
    symbol has been read and a call of next().
    Translate each nonterminal symbol into a call of the method with the same
    name.
    If you have to decide between different productions use the lookahead sets
    to determine which one to use.
    If you find that there is no way to continue call error().
    We initiate the parsing process by calling next() to read the first symbol and
    then call parseE(). If after processing parseE() we are at the end marker,
    then the parsing has been successful.
    next();
    parseE();
    if(curr=="$") {
    System.out.println("OK ");
    } else {
    error("End expected");
    }
    The complete parser can be found at
    http://www.cs.nott.ac.uk/~txa/g51mal/ParseE0.java.
    Actually, we can be a bit more realistic and turn the parser into a simple evaluator by
    42

    static void next() {


    41

    Replace a by any integer. I.e. we use


    Integer.valueOf(curr).intValue();

    to translate the current token into a number. JAVA will raise an exception
    if this fails.
    Calculate the value of the expression read. I.e. we have to change the
    method interfaces:
    static
    static
    static
    static
    static

    int
    int
    int
    int
    int

    parseE()
    parseE1(int x)
    parseT()
    parseT1(int x)
    parseF()

    7.5

    Beyond LL(1) - use LR(1) generators

    The restriction to LL(1) has a number of disadvantages: In many case a natural


    (and unambiguous) grammar like G has to be changed. There are some cases
    where this is actually impossible, i.e. although the language is deterministic
    there is no LL(1) grammar for this.
    Luckily, there is a more powerful approach, called LR(1). LL(1) proceeds from
    top to bottom, when we are looking at the parse tree, hence this is called topdown parsing. In contrast LR(1) proceeds from bottom to top, i.e. it tries to
    construct the parse tree from the bottom upwards.
    The disadvantage with LR(1) and the related approach LALR(1) (which is
    slightly less powerful but much more efficient) is that it is very hard to construct
    LR-parsers from hand. Hence there are automated tools which get the grammar
    as an input and which produce a parser as the output. One of the first of those
    parser generators was YACC for C. Nowadays one can find parser generators
    for many languages such as JAVA CUP for Java [Hud99] and Happy for Haskell
    [Mar01].

    The idea behind parseE1 and parseT1 is to pass the result calculated
    so far and leave it to the method to incorporate the missing part of the
    expression. I.e. in the case of parseE1
    static int parseE1(int x) {
    if (curr=="+") {
    next();
    int y = parseT();
    return parseE1(x+y);
    } else if(curr==")" || curr=="$" ) {
    return x;
    } else {
    error("Unexpected :"+curr);
    return x;
    }
    }

    Here is the complete program with evaluation


    http://www.cs.nott.ac.uk/~txa/g51mal/ParseE.java.
    We can run the program and observe that it handles precedence of operators
    and brackets properly:
    [txa@jacob misc]$ java ParseE
    3 + 4 * 5
    OK 23
    [txa@jacob misc]$ java ParseE
    ( 3 + 4 ) * 5
    OK 35

    44

    43

    You might also like