//////////////////////////////////////////////////////////////////////////////
// Copyright (c) 2021, 2026 Contributors to the Eclipse Foundation
//
// See the NOTICE file(s) distributed with this work for additional
// information regarding copyright ownership.
//
// This program and the accompanying materials are made available
// under the terms of the MIT License which is available at
// https://opensource.org/licenses/MIT
//
// SPDX-License-Identifier: MIT
//////////////////////////////////////////////////////////////////////////////

include::_part_attributes.asciidoc[]

:leveloffset: -2

[[rail-lexical-syntax]]
== Lexical syntax

=== Whitespace

Spaces, tabs, and new line characters are supported as whitespace.
Whitespace is ignored (except in the literals described below), but can be used to separate tokens as well as for layout purposes.
The use of tab characters is allowed, but should be avoided if possible, as layout will be different for text editors with different tab settings.
You may generally format the input as you see fit, and start on a new line when desired.

Examples:

[source, raildiagram]
----
# Normal layout.
DiagramName : A B C
            ;

# Alternative layout.
DiagramName
  : A
    B C
  ;
----

=== Comments

Comments start with a hash symbol (`+#+`) and end at the end of the line.
Comments are ignored.

Examples:

[source, raildiagram]
----
DiagramName : A B C ; # Comment
----

=== Terminals

The following terminals are defined:

`Identifier`::
An identifier.
Defined by the regular expression: `+[\-A-Za-z0-9_]pass:[+]+`.
They thus consist of dashes (`-`), letters, digits and underscores (`+_+`).
+
Examples:
+
[source, raildiagram]
----
Expression
ABC_DEF
ABC-DEF
----

`BracketedString`::
A bracketed string.
Defined by the regular expression:
`\[([^\\\]\n]|\\[\\\]])*\]`.
They are thus enclosed in square brackets (`[` and `]`).
They must be on a single line and must thus not include new line characters (`\n`, Unicode U+0A).
To include a closing bracket (`+]+`), it must be escaped as `+\]+`.
Since a backslash (`\`) serves as escape character, to include a backslash it must be escaped as `\\`.
+
Examples:
+
[source, raildiagram]
----
[literal]
[line-break]
[abc\]def]
[abc\\def]
----

`SingleQuotedString`::
A single-quoted string literal.
Defined by the regular expression:
`+'([^\\'\n]|\\[\\'])*\'+`.
These literals are enclosed in single quotes (`+'+`).
They must be on a single line and must thus not include new line characters (`\n`, Unicode U+0A).
To include a single quote (`+'+`) in a string literal, it must be escaped as `+\'+`.
Since a backslash (`\`) serves as escape character, to include a backslash in a string literal it must be escaped as `\\`.
+
Examples:
+
[source, raildiagram]
----
'hello world'
'abc\'def'
'abc\\def'
----

`DoubleQuotedString`::
A double-quoted string literal.
Defined by the regular expression:
`+\"([^\\\"\n]|\\[\\\"])*\"+`.
These literals are enclosed in double quotes (`+"+`).
They must be on a single line and must thus not include new line characters (`\n`, Unicode U+0A).
To include a double quote (`+"+`) in a string literal, it must be escaped as `+\"+`.
Since a backslash (`\`) serves as escape character, to include a backslash in a string literal it must be escaped as `\\`.
+
Examples:
+
[source, raildiagram]
----
"hello world"
"abc\"def"
"abc\\def"
----
