Lab 1(c): Syntax Definition

Project
September 08, 2020

In this lab, you define the concrete and abstract syntax of ChocoPy in Spoofax. From this definition, you generate an Eclipse editor, which provides syntax checking, error recovery, and syntax highlighting.

In this lab, focus on translating the grammar in Figure 3 of the ChocoPy Reference Manual to a context-free syntax definition with constructors. In the next lab, you extend this definition with disambiguation rules for associativity, priority, and layout constraints.

Objectives

In the chocopy.syntax.test project, develop a syntax definition for ChocoPy in SDF3 and generate an Eclipse editor from it. The definition should include:

  1. The start symbol Program
  2. A lexical syntax definition for all ChocoPy lexemes
  3. A context-free syntax definition ChocoPy complying with the grammar in Firgure 3 of the reference manual
  4. Constructor names for context-free syntax productions

Syntax Definition

You should define your syntax in SDF3. You can start by opening the file syntax/chocopy.sdf3 in your chocopy.syntax project. You can also split your syntax definition over several modules in syntax/ and import modules into module chocopy.

module chocopy
imports lex
imports expressions
imports statements
// etc.
context-free start symbols Program

The module already imports a module Common.sdf3, which you can find in any initial Spoofax editor project. This module provides syntax definitions for common lexical sorts such as identifiers, integers, strings, whitespace, and comments.

For more information on how to write SDF3 syntax definitions, also check the documentation on how to define a language in Spoofax.

Context-free Syntax

Start with the context-free syntax of the language. Use the context-free grammar in the ChocoPy Language Reference Manual as a reference.

We recommend to use template productions for your context-free syntax definition, since they help when generating artifacts other than just the parser. When you use template productions, you need to make sure that templates are correctly tokenised. Otherwise, the parser would reject layout in certain positions, for example between [ and ] in an array type. Check the SDF3 documentation for details. In case you want to use < or > as symbols inside a template, you can use alternate template brackets [...].

Lexical Syntax

Continue with the lexical syntax definition including identifiers, integer, and simple layout. The Common module in syntax/Common.sdf3 already provides definitions, but these do not necessarily comply with the lexical syntax of ChocoPy. You can either fix these definitions in Common, or define and import your own module and use Common only for inspiration.

First, define lexical syntax rules:

lexical syntax
  ID     = ...
  INT    = ...
  LAYOUT = ...

Second, define follow restrictions to ensure longest match:

lexical restrictions
  ID -/- ...
  INT -/- ...
context-free restrictions
 LAYOUT? -/- ...

Finally, use rejection rules to rule out reserved words.

lexical syntax
  ID = ... {reject}

Testing Your Syntax Definition

You now can check your tests. It is important to get your grammar working at this stage. Do not move on if you have issues here, since there is a high chance that these issues influence your other tests as well. If you experience weird behaviour on your tests, this is most likely caused by an erroneous definition of LAYOUT.

Finally, you should add lexical syntax rules for comments to your syntax definition. Start with single line comments. Do not forget to define follow restrictions.

Editor Services

When developing a language in Spoofax, the syntax definition written in SDF3 does not only produce a parser but other editor services as well.

Pretty Printing

Spoofax generates pretty-printing rules from your syntax definition. You can find these rules in src-gen/pp/<name>-pp.str.

In order to test the pretty-print builder, you need to build your project. Your ChocoPy editor provides a menu entry named Format that uses these generated rules to pretty-print a ChocoPy file. Create or open a .cpy test file with a valid program, go to the Spoofax -> Syntax menu and choose Format. This will apply Format to the current file and show the result in a new editor.

If your start symbols are not defined in the main SDF3 module, you might need to import the generated src-gen/pp/*-pp.str files into trans/pp.str.

Typically, the pretty-printed code lacks proper indentation and line breaks. You can fix this by improving your templates in the syntax definition. The pretty-printer follows the indentation and line breaks from the syntax definition.

You should improve your syntax definition in order to get readable code with a consistent indentation. You might read on indent styles for some inspiration.

Make sure that your altered syntax definition is still correct and can be used to parse ChocoPy programs.

Syntactic Code Completion

Together with a pretty-printer, Spoofax also automatically derives syntactic code completion from the syntax definition. This feature allows for example, for new users to discover the language’s syntax while editing the program. To know more details about syntactic code completion in Spoofax, check the documentation.