Principles of Bottom-Up Parsing

(上向き構文解析の原理)

9th lecture, June 3, 2016

Language Theory and Compilers

http://www.sw.it.aoyama.ac.jp/2016/Compiler/lecture9.html

Martin J. Dürst

© 2005-16 Martin J. Dürst 青山学院大学

Today's Schedule

• Remainders and summary from last lecture
• Left recursion and EBNF
• Limits of top-down parsing
• Bottom-up parsing
• bison

Summary of Last Lecture

• General top-down parsing: Backtracking
• Recursive descent parsing
  • Concept: A function for each nonterminal symbol
  • Problem: Left recursion

Problems with Top-Down Parsing

• For efficient parsing, the grammar has to be restricted quite a bit
• Grammars where lookahead is necessary are difficult to deal with
  Example:
  

  Factor → FunctionCall | Variable | ArrayElement
  FunctionCall → Identifier '(' Parameters ')'
  Variable → Identifier
  ArrayElement → Identifier '[' Expression ']'

  (it is unclear which rule should be selected in Factor when the next token is Identifier)

Additional Requirements for Grammars

Depending on parsing method, additional requirements become necessary:

• Efficiency
• Recognition power
• Guarantee to terminate (example: no left recursion)
• No ambiguity

⇒ A grammar that just produces/recognizes 'words' is not enough

Conclusion

• With recursive descent parsing, it is easily possible to convert grammar rules to a parser by hand
• However, this requires changes and limitations to the grammar
• A parsing method with less limitations is desirable
• Automatic implementation is desirable

→ Bottom-up parsing)

Top-Down and Bottom-Up Parsing

Top-down parsing:

The parse tree is constructed starting at the top (start symbol)

Bottom-up parsing:

The parse tree is constructed from the bottom (terminal symbols)

During parsing, there may be multiple small parse trees

Top-Down and Bottom-Up Parsing

bison Overview

• yacc: Yet Another Compiler Compiler (1975)
• Became popular together with Unix
• Compiler Compiler: A compiler that constructs a compiler
• Yet Another: one more
  (it was very popular to create compiler compilers, and difficult to find a good name)
  (due to yacc, names starting with YA.. became widely used)
• bison is the gnu free software version of yacc

Exercise: A simple Pocket Calculator

Files to start with: makefile, calc.y, calc.lex

Use of make

• It is difficult not to forget flex, bison, and gcc
• The make command follows the instructions in the makefile, executing a minimum of commants to bring everything up to date
• If necessary, please add the make command to your cygwin installation
• Syntax of makefile (→: tabulator; other whitespace characters will not work!):
  target: input1 input2 input3 ...
→command to construct target
• Running make without arguments will construct the first target in the makefile
• Running make target will construct target
• Caution: The file name has to be exactly makefile, without extension
  (makefile.txt will not work!)

Example of bison Input Format

%{
#include <stdio.h>
#define YYSTYPE double
int yylex (void);
void yyerror (char const *);
%}
%token NUM PLUS

%%
statement: exp { printf ("Result is %g\n", $1); }
;

exp:       exp PLUS exp { $$ = $1 + $3; }
         | NUM { $$ = $1; }
;

%%
int main (void)
{    return yyparse ();  }
     
void yyerror (char const *s)
{    fprintf (stderr, "%s\n", s);  }

Structure of bison Input Format

bison 関係の宣言など
%{ C 言語の宣言など %}
%%
書換規則    { 実行文 (C 言語) }
書換規則    { 実行文 (C 言語) }
書換規則    { 実行文 (C 言語) }
%%
関数など (C 言語)
関数など (C 言語)

Structure of bison Input Format

Mixture of bison-specific directives and C program fragments

There are three main parts, separated by %%:

1. Preparation/settings:
   Settings for using bison
   C #include statements, definition/initialization of global variables,...
   C parts have to be surrounded by %{ ... %}
2. Rewriting rules and (in { ... }) program fragments that get executed when a rule is matched
   (the first nonterminal symbol is the start symbol)
3. Rest of C program (functions,...)

Newlines and indentation can be significant

Example of bison Rewriting Rule

exp: exp PLUS exp { $$ = $1 + $3; }

   | NUM { $$ = $1; }

;

• Colon (:) instead of arrow (→)
• Semicolon (;) at the end

Attribute(d) Grammar

• Simple parsing will only accept or reject the input
• Attribute grammars add attributes to each (non)terminal symbol
• For each rewriting rule, rules for how to calculate attribute values are defined
• Order of calculating attribute values:
  Usually: bottom-up
  A combination of both directions is also possible
• Example rewriting rule: E0 → E1 '-' E3
  Rule for calculating attribute values: S(E0) = S(E1) - S(E3)
  Notation in bison: $$ = $1 - $3
• Typical attribute types:
  • Hight of tree, length of input
  • Result of expression evaluation
  • Partial parse tree (or syntax tree)
  • Generated code

bison Manual

• English:
  http://www.gnu.org/software/bison/manual/
  
• Japanese:
  http://www.geocities.co.jp/SiliconValley-Oakland/3432/man/bison/bison-ja.html

Hints for Developping bison Programs

• Prepare a very simple input file for testing
• Prepare a correct output file for testing
• Automatically compile/execute the program and compare the output using the diff command
• If there is no output from diff, the test is successful
• Test First: Add a new test, check that it fails, then work on making the test successful

Homework

Deadline: June 9, 2016 (Thursday), 19:00

Where to submit: Box in front of room O-529 (building O, 5th floor)

Complete calc.y so that with test.in as an input, it produces test.check

Submit calc.y only, A4 single page (using both sides is okay; NO cover page, staple in top left corner if more than one page is necessary), printout (no wrapping lines), name (kanji and kana) and student number in comment at the top right

If there are differences with newlines, make sure that all files use Unix line ending convention
(In Notepad2, choose File → Line Endings → Unix (LF))

Glossary

factor

因子

tabulator

タブ文字

attribute(d) grammar

属性文法