Вы находитесь на странице: 1из 110

Describing Syntax and Semantics

Introduction
The General Problem of Describing
Syntax
Formal Methods of Describing Syntax
Attribute Grammars
Describing the Meanings of
Programs: Dynamic Semantics

Describing Syntax and


Semantics
Syntax
the form or structure of the expressions, statements, and
program units
describes how programs *look*: their form and structure

Semantics
the meaning of the expressions, statements, and
program units
which describes what language constructs *do*

Example:
For example, the syntax of a Java while statement
is
while (boolean_expr) statement

The semantics of this statement form


is that when the current value of the
Boolean expression is true, the
embedded statement is executed.
Otherwise, control continues after
the while construct. Then control
implicitly returns to the Boolean
expression to repeat the process.

Steps in a compiler:

Both lexical analysis and syntactic analysis deal with syntax,


why separate them in two steps in the compiler?
- Efficiency:
- design of the lexer is based on a finite state automaton,
while the parser is a pushdown automaton, which is more
complex
- for a non-optimizing compiler, about 75% of compile time
is spent in the lexer
- History:
- prior to ASCII, computers had their own character sets
- still different character sets today actually
- it used to be hat the end-of-line character was very O/S
dependent

The General Problem of Describing


Syntax
Who must use language definitions?
1. Other language designers
2. Implementers
3. Programmers (the users of the
language)
A sentence is a string of characters
over some
alphabet
A language is a set of sentences

A lexeme is the lowest level syntactic unit of a


language (e.g., *, sum, begin)
A token is a category of lexemes (e.g., identifier)
Example
index = 2 * count + 17;
The lexemes and tokens of this statement are
Lexemes Tokens
index identifier
= equal_sign
2 int_literal
* mult_op
Count identifier
+ plus_op
17 int_literal
; semicolon

Formal approaches to describing syntax:


1. Recognizers - used in compilers
Suppose we have a language L that uses an alphabet of
characters.
To define L formally using the recognition method, we would
need to construct a mechanism R, called a recognition
device, capable of reading strings of characters from the
alphabet .
R would indicate whether a given input string was or was
not in L

The syntax analysis part of a compiler is a recognizer for


the language the compiler translates.
The recognizer need not test all possible strings of
characters from some set to determine whether each is in
the language, it need only determine whether given
programs are in the language.
The syntax analyzer then determines whether the given
programs are syntactically correct.
The structure of syntax analyzers, also known as parsers

2. Generators
A language generator is a device that
can be used to generate the
sentences of a language.

Formal Methods of Describing


Syntax
discusses the formal languagegeneration
mechanisms,
called
grammars, that are commonly used
to
describe
the
syntax
of
programming languages

Context-Free Grammars
Developed by Noam Chomsky in the
mid-1950s
Language generators, meant to
describe the syntax of natural
languages
Define a class of languages called
context-free languages

Context-Free Grammar
In linguistics and computer science, a context-free
grammar (CFG) is a formal grammar in which every
production rule is of the form
Vw
where V is a non-terminal symbol and w is a string
consisting of terminals and/or non-terminals.
The term "context-free" expresses the fact that the nonterminal V can always be replaced by w, regardless of
the context in which it occurs.
A formal language is context-free if there is a contextfree grammar that generates it.

context-free grammar
(CFG)
Is a set of recursive rewriting rules (or productions) used to
generate patterns of strings.
A CFG consists of the following components:
a set of terminal symbols, which are the characters of the
alphabet that appear in the strings generated by the
grammar. {0, 1}
a set of nonterminal symbols, which are placeholders for
patterns of terminal symbols that can be generated by the
nonterminal symbols. {q, f,}
a set of productions, which are rules for replacing (or
rewriting) nonterminal symbols (on the left side of the
production) in a string with other nonterminal or terminal
symbols (on the right side of the production).
a start symbol, which is a special nonterminal symbol that
appears in the initial string generated by the grammar.

The syntax of whole programming


languages, with minor exceptions,
can be described by context-free
grammars

Backus Normal Form (1959)


Invented by John Backus to describe Algol
58
BNF is equivalent to context-free grammars
BNF is a meta-language. A meta-language
is a language used to describe another
language.
In BNF, abstractions are used to represent
classes of syntactic structures-they act like
syntactic variables (also called non-terminal
symbols)

A BNF description,or grammar, is a


collection of rules consists terminals
and non terminals
e.g.
<while_stmt> -> while <logic_expr>
do <stmt>
This is a rule; it describes the
structure of a while statement
<if_stmt> if ( <logic_expr> )
<stmt>

A rule has a left-hand side (LHS) and a


right-hand side (RHS), and consists of
terminal and nonterminal symbols
A grammar is a finite nonempty set of rules
An abstraction (or nonterminal symbol) can
have more than one RHS
<stmt> -> <single_stmt>
| begin <stmt_list> end
Syntactic lists
described in BNF using recursion
<ident_list> -> ident| ident, <ident_list>

Grammars and Derivations


A grammar is a generative device for
defining languages.
The sentences of the language are
generated through a sequence of
applications of the rules, beginning with
a special nonterminal of the grammar
called the start symbol.
A derivation is a repeated application of
rules, starting with the start symbol and
ending with a sentence (all terminal
symbols).

In a grammar for a complete programming


language, the start symbol represents a
complete program and is often named
<program>.

Derivation of a program

This derivation, begins with the start symbol, in this case


<program>. The symbol => is read derives.
Each successive string in the sequence is derived from
the previous string by replacing one of the nonterminals
with one of that nonterminals definitions.
Each of the strings in the derivation, including
<program>, is called a sentential form.
Derivations that use this order of replacement are called
leftmost derivations.
The derivation continues until the sentential form
contains no nonterminals.
A sentence is a sentential form that has only terminal
symbols
A leftmost derivation is one in which the leftmost
nonterminal in each sentential form is the one that is
expanded
A derivation may be neither leftmost nor rightmost

A parse tree is a hierarchical


representation of a derivation

Ambiguity
A grammar that generates a
sentential form for which there are
two or more distinct parse trees is
said to be ambiguous

Operator Precedence
How to remove the ambiguity?
By enforcing Operator precedence
Operator with Higher precedence
should be placed lower in the the tree

Separate rules for different


operators

Unambiguous grammar produces a unique parse tree


But a single parse tree can have different derivations

Leftmost derivation

<assign> => <id> = <expr>


=> A = <expr>
=> A = <expr> + <term>
=> A = <term> + <term>
=> A = <factor> + <term>
=> A = <id> + <term>
=> A = B + <term>
=> A = B + <term> * <factor>
=> A = B + <factor> * <factor>
=> A = B + <id> * <factor>
=> A = B + C * <factor>
=> A = B + C * <id>
=> A = B + C * A

Rightmost derivation

<assign> => <id> = <expr>


=> <id> = <expr> + <term>
=> <id> = <expr> + <term> * <factor>
=> <id> = <expr> + <term> * <id>
=> <id> = <expr> + <term> * A
=> <id> = <expr> + <factor> * A
=> <id> = <expr> + <id> * A
=> <id> = <expr> + C * A
=> <id> = <term> + C * A
=> <id> = <factor> + C * A
=> <id> = <id> + C * A
=> <id> = B + C * A
=> A = B + C * A

Both of these derivations, are represented by the same


parse tree.

Associativity of Operators
Left Associativity
Left recursive grammar rule
e.g, a+b+c = (a+b)+c
e.g., A = B + C + A

Right Associativity: Right recursive


grammar rule
e.g., a+b+c = a+(b+c)

What languages do
SmallTalk: no operator precedence
APL: right-associativity with no operator precedence
- Programmers typically add parentheses when things are
difficult to read
if (a < x < b)

equiv. if ((a < x) < b) [returns 1 if b >= 1]

For languages like C, C++ and Java, the number of


precedence levels and operators makes the grammar huge
- this makes the parse very slow
- a possibility is to not "encode" any
precedence/associativity in the grammar, but just have a
table that says what these things are (Java, C)

The syntax graph and EBNF


descriptions of the Ada if statement

Extended BNF
Three extensions are commonly
included in the various versions of
EBNF.
The first of these denotes an optional
part of an RHS, which is delimited by
brackets. For example, a C if-else
statement can be described as
<if_stmt> if (<expression>)
<statement>
[else
<statement>]

The second extension is the use of


braces in an RHS to indicate that the
enclosed part can be
<ident_list>

<identifier>
{,
<identifier>}
This is a replacement of the recursion
by a form of implied iteration; the
part enclosed within braces can be
iterated any number of times
repeated indefinitely or left out

The third common extension deals with


multiple-choice options.
When a single element must be chosen
from a group, the options are placed in
parentheses and separated by the OR
operator, |.
For example,
<term> <term> (* | / | %) <factor>
In BNF, a description of this <term> would
require the following three rules:
<term> <term> * <factor>
| <term> / <factor>
| <term> % <factor>

Attribute Grammar
CFGs cannot describe all of the syntax of
programming languages
An attribute grammar is a device used to
describe more of the structure of a
programming
language
than
can
be
described with a context-free grammar.
An attribute grammar is an extension to a
context-free grammar
Primary value of attribute grammars (AGs)
Static semantics specification
Compiler design (static semantics checking)

Syntax in Lisp?
- No expressions in LISP, but function
applications!
- Everything is parenthesized.
- No associativity, No operator precedence.
- Compare with the 25 pages of the Java manual
- This's also why writing a LISP interpreter is not
very difficult
- write a pure Java interpreter would be
extremely difficult
- writing a byte code interpreter is not that hard

Static semantics Introduction


There are some characteristics of the structure of
programming languages that are difficult to
describe with BNF, and some that are impossible.
In Java, for example, a floating-point value cannot
be assigned to an integer type variable, although
the opposite is legal.
If all of the typing rules of Java were specified in
BNF, the grammar would become too large to be
useful, because the size of the grammar
determines the size of the syntax analyzer.

The static semantics of a language is only


indirectly related to the meaning of programs
during execution; rather, it has to do with the
legal forms of programs (syntax rather than
semantics).
Static semantics is so named because the
analysis required to check these specifications
can be done at compile time.
Because of the problems of describing static
semantics with BNF, a variety of more powerful
mechanisms has been devised for that task.
One such mechanism, attribute grammars, was
designed by Knuth (1968a) to describe both the
syntax and the static semantics of programs.

Attribute grammars
are a formal approach both to describing
and checking the correctness of the
static semantics rules of a program.

Dynamic semantics
which is the meaning of expressions,
statements, and program units

Basic concepts
Attributes,
which are associated with
grammar symbols (the terminal and nonterminal symbols), are similar to variables in
the sense that they can have values assigned
to them.
Attribute
computation
functions,
sometimes called semantic functions, are
associated with grammar rules. They are used
to specify how attribute values are computed.
Predicate functions, which state the static
semantic rules of the language, are associated
with grammar rules to check for attribute
consistency.

Attribute Grammars Defined


An attribute grammar is a context-free
grammar G = (S, N, T, P) with the
following additions:
1. For each grammar symbol x there is a
set A(x)of attribute values
The set A(X) consists of two disjoint sets S(X)
and I(X), called synthesized and inherited
attributes, respectively.
Synthesized attributes are used to pass
semantic information up a parse tree
inherited
attributes
pass
semantic
information down and across a tree.

2. Each rule has a set of functions that define


certain attributes of the non-terminals in the rule
. For a rule X0 X1 ... Xn
Semantic functions of the form S(X0) = f(A(X1), ...
A(Xn))
Compute Synthesized attributes (from child nodes)

Semantic functions of the form I(Xj) = f(A(X0), ... ,


A(Xn)), for I <= j <= n
Compute Inherited attributes (from parent or siblings)

3. Predicate functions
. Boolean expressions on the attribute set
{A(X0), ... , A(Xn)}

Intrinsic attributes
Intrinsic attributes are synthesized
attributes of leaf nodes whose values
are determined outside the parse
tree.

Examples of attribute
grammars
The string attribute of <proc_name>, denoted by <proc_name>.string,
is the actual string of characters that were found immediately following
the reserved word procedure by the compiler.
Syntax rule:
<proc_def> procedure <proc_name>[1] <proc_body> end
<proc_name>[2];

Predicate:
the predicate rule states that the name string attribute of the
<proc_name> nonterminal in the subprogram header must match the
name string attribute of the <proc_name> nonterminal following the
end of the subprogram
<proc_name>[1]string == <proc_name>[2].string

The attributes for the nonterminals in the


example attribute grammar are described
in the following paragraphs:

actual_typeA synthesized attribute


associated with the nonterminals <var>
and <expr>. It is used to store the actual
type, int or real, of a variable or expression.
expected_typeAn inherited attribute
associated with the nonterminal <expr>. It
is used to store the type, either int or real,
that is expected for the expression, as
determined by the type of the variable on
the left side of the assignment statement.

Examples of attribute
grammars

- only two types: int and real


int + int is an int
int + real is a real
real + real is a real

A parse tree of the sentence A = A +


B

Computing Attribute Values


called decorating the parse tree
If all attributes were inherited, this
could proceed in a completely topdown order, from the root to the
leaves.
If all the attributes were synthesized,
it could proceed in a completely
bottom-up order, from the leaves to
the root

The following is an evaluation of the attributes,


in an order in which it is possible to compute
them:

Evaluation
Checking the static semantic rules of a language is an
essential part of all compilers.
Even if a compiler writer has never heard of an attribute
grammar, it is necessary to use fundamental ideas to
design the checks of static semantics rules of the compiler.
One of the main difficulties in using an attribute grammar
to describe all of the syntax and static semantics of a real
contemporary programming language is the size and
complexity of the attribute grammar.
The large number of attributes and semantic rules required
for a complete programming language make such
grammars difficult to write and read.
the attribute values on a large parse tree are costly to
evaluate.

Semantics
There is no single widely acceptable notation or formalism for
describing semantics
Semantics description tool is useful for:
Better understanding the statements of a language
Developing more effective compiler
Program correctness proofs

Three approaches
Operational
Axiomatic
Denotational

Operational Semantics
Describe the meaning of a program by executing its statements on a machine,
either simulated or actual.
The change in the state of the machine (memory, registers, etc.) defines the
meaning of the statement

natural operational semantics


At the highest level, the interest is in the
final result of the execution of a
complete program.

structural operational semantics


At
the
lowest
level,
operational
semantics can be used to determine the
precise meaning of a program through
an examination of the complete
sequence of state changes that occur
when the program is executed.

Basic Process
language is clarity
to design an appropriate intermediate language
Every construct of the intermediate language must
have an obvious and unambiguous meaning.

If the natural operational semantics is used, a


virtual machine (an interpreter) must be
constructed for the intermediate language.
The virtual machine can be used to execute either
single statements, code segments, or whole programs.
The semantics description can be used without a
virtual machine if the meaning of a single statement is
all that is required.

If structural operational semantics is used, the


intermediate code can be visually inspected.

semantics of the C for construct


Example

Evaluation
Good if used informally (language manuals, etc.)
Extremely complex if used formally (e.g., Vienna
Definition Language - VDL), it was used for
describing semantics of PL/I (PL/I stands for
"Programming Language 1". PL/I was an
antecedent of the C programming language,
which essentially replaced it as an all-purpose
serious programming language).
Useful for language users and implementors
Based on algorithms, rather than mathematics

Denotational semantics
It is an approach of formalizing the
meanings ofprogramming languages
by
constructing
mathematical
objects
(calleddenotations)
that
describe
the
meanings
of
expressions from the languages
The method is named denotational
because the mathematical objects
denote the meaning of their
corresponding syntactic entities.

Building Denotational specification for


a language:
Define a Number for each grammar symbol
Define a function which converts:
Grammar symbol -> corresponding numbers

Denotational vs operational
semantics

The mapping functions of a denotational


semantics programming language
specification, like all functions in
mathematics, have a domain and a range.
The domain is the collection of values that are
legitimate parameters to the function it is
called the syntactic domain
the range is the collection of objects to which
the parameters are mapped it is called the
semantic domain.

Example

We want to associate a semantic to each BNF


production
We define a function M_bin, that maps the
syntactic objects in the grammar rules to the
objects in N, the set of non negative integers.

The state of a program


Operational semantics:
The change in the state of the machine Memory, registers, etc.
The state change is defined by coded algorithms

Denotational semantics:
The change in the values of the program's variables
The state change is defined by mathematical functions

Let the state s of a program be represented as a set of


ordered pairs, as follows:
s = {<i1, v1>, <i2, v2>, . . . , <in, vn>}
Each i is the name of a variable, and the associated vs are
the current values of those variables.
Let VARMAP be a function of two parameters: a variable
name and the program state.
VARMAP: a function that returns the current value of a
variable, given the variable name and a state
VARMAP(ij, s) = vj

Semantic functions maps state to state (or number)

Expressions
Semantic function: maps expressions to
integer value (or error)

Assignment Statements
Semantic function: maps state to state

Logical Pretest Loops


Semantic function: maps state to state
Existing functions
Mb: maps boolean expression to boolean
values (true, false, undef)
Msl: maps statement lists to states

Evaluation of denotational
semantics:
When a complete system has been defined
for a given language, it can be used to
determine the meaning of complete
programs in that language. (i.e) Provides a
rigorous way to analyze programs
Can be used to prove the correctness of
programs
Can be an aid to language design and
compiler generation
But too complex for language users

Axiomatic Semantics
Axiomatic semanticsis an approach
based onmathematical logic to proving
thecorrectness of computer programs
Goal: program correctness proof
(i.e) Rather than directly specifying the
meaning of a program, axiomatic semantics
specifies what can be proven about the
program.

In axiomatic semantics, there is no model of the


state of a machine or program or model of state
changes that take place when the program is
executed.
The meaning of a program is based on
relationships among program variables and
constants, which are the same for every
execution of the program.
Axiomatic semantics has two distinct
applications:
program verification
program semantics specification.

Approach
Specify constraints for each statement
by logical expressions called assertions (or
predicate)

Pre-condition:
an assertion before a statement
Describes the constraints on the variables at that
point

Post-condition:
an assertion following a statement
Describes the new constraints on those variables

Weakest precondition:
The least restrictive precondition that will
guarantee the postcondition.
Pre-post form Notation: {P} S {Q}
S: program statement
P: constraints on variables BEFORE statement
execution called a "precondition"
Q: constraints on variables AFTER statement
execution called a "postcondition"
Example: a = b + 1 {a > 1}
One possible precondition: {b > 10}
Weakest precondition: {b > 0}

An inference rule is a method of inferring the truth of one


assertion on the basis of the values of other assertions. The
general form of an inference rule is as follows:

This rule states that if S1, S2, . . . , and Sn are true, then
the truth of S can be inferred.
The top part of an inference rule is called its antecedent;
the bottom part is called its consequent.
An axiom is a logical statement that is assumed to be true.
Therefore, an axiom is an inference rule without an
antecedent.

Process of Program proof

Assignment Statements

Let "x=E" be an assignment statement, and


Q be its postcondition
To compute its weakest precondition, P
Axiom: P = Qx->E or {Qx->E} x = E {Q}
P is computed as Q in which all occurrences of x
have been replaced by E

- Example:
a = b / 2 - 1 {a < 10}
the weakest precondition is:
P={b/2-1
< 10} which reduces to {b < 22}.

The usual notation for specifying the


axiomatic semantics of a given statement
form is
{P}S{Q}
where P is the precondition, Q is the
postcondition, and S is the statement form.
In the case of the assignment statement,
the notation is

e.g., Prove the correctness of {x > 3} x = x - 3


{x>0}
Answer: {x 3 > 0} {x > 3}, thus TRUE
e.g., How about {x > 5} x = x - 3 { x > 0 } ?
Since {x > 5} {x > 3}, it is still TRUE

the rule of consequence says that a postcondition


can always be weakened and a precondition can
always be strengthened.

Sequences
The weakest precondition for a sequence of
statements cannot be described by an axiom,
because the precondition depends on the particular
kinds of statements in the sequence.
So the precondition can only be described with an
inference rule.
Let S1 and S2 be adjacent program statements. If S1
and S2 have the following pre- and post-conditions

The inference rule states that to get


the sequence precondition, the
precondition of the second statement
is computed.
This new assertion is then used as
the postcondition of the first
statement, which can then be used
to compute the precondition of the
first statement, which is also the
precondition of the whole sequence.

Selection

Logical Pretest Loops


Computing the weakest precondition for a while loop is
inherently more difficult than for a sequence, because the
number of iterations cannot always be predetermined.
When the number of iterations is known, the loop can be
unrolled and treated as a sequence.
The principal step in induction is finding an inductive
hypothesis.
The corresponding step in the axiomatic semantics of a
while loop is finding an assertion called a loop invariant
(loop invariant is a condition that is necessarily true
immediately before and immediately after each iteration
of a loop), which is crucial to finding the weakest
precondition.

loop invariant Characteristics


P => I --the loop invariant must be true
initially
{I} B {I} --evaluation of the Boolean must
not change the validity of I
{I and B} S {I} --I is not changed by
executing the body of the loop
(I and (not B)) => Q --if I is true and B is
false, is implied
The loop terminates

To find I, the loop postcondition Q is used to


compute preconditions for several different
numbers of iterations of the loop body
In general weakest precondition is given by,
wp(statement, postcondition) = precondition
A wp function is often called a predicate
transformer, because it takes a predicate, or
assertion, as a parameter and returns another
predicate.

Example
while y <> x do y = y + 1 end {y = x}
Note:
equal sign => In assertions, it means mathematical
equality;
outside assertions, it means the assignment operator.

For zero iterations


{y = x}
For one iteration, it is
wp(y = y + 1, {y = x}) = {y + 1 = x}, or {y
= x - 1}
For two iterations, it is
wp(y = y + 1, {y = x - 1})={y + 1 = x - 1},
or {y = x - 2}

We must ensure that our choice satisfies


the four criteria for I for our example loop.
1. P = I, P => I.
2. It must be true that {I and B} S {I}
From previous example,
{y <= x and y <> x} y = y + 1 {y <= x}
Applying the assignment axiom to
y = y + 1 {y <= x}
we get {y + 1 <= x}, which is equivalent to
{y < x}, which is implied by
{y <= x and y <> x}.
Statement 2 is proven.

3. It must be true that {I and (not B)} =>


Q
From previous example,
{(y <= x) and not (y <> x)} => {y = x}
{(y <= x) and (y = x)} => {y = x}
{y = x} => {y = x}
So, this is obviously true.

4. loop termination
From previous example, the question is
whether the loop
{y <= x} while y <> x do y = y + 1 end
{y = x} terminates.

If loop termination can be shown, the


axiomatic description of the loop is
called total correctness.
If the other conditions can be met
but termination is not guaranteed, it
is called partial correctness.

Program Proofs

Evaluation
Developing axioms or inference rules for
all of the statements in a language is
difficult
It is a good tool for correctness proofs,
and an excellent framework for reasoning
about programs, but it is not as useful for
language users and compiler writers
Its usefulness in describing the meaning
of a programming language is limited for
language users or compiler writers

Evaluation of axiomatic semantics:


It is hard to develop axioms or
inference rules
for all of the statements in a
language
Nice tool for correctness proofs
But not for language users and
compiler writers

Summary

Вам также может понравиться