The basics of the tunnel parsing algorithm are covered in an article in: 2019 book in the series of Springer Lecture Notes in Computer Science (LNCS), Volume 11950, 2021.

A DOI and a PDF link will be placed here, after are available.

At 2021 Jun 30, 12:12 GMT in news channel Tunnel Parsing

The result of a parsing process is often a Syntax Tree (ST). This is a tree, that represents the syntactic information extracted from the input during the parsing process.

Tunnel Grammar Studio generates Parsing Machines (PM) from Augmented Backus-Naur form grammars to a source code. The PM implements the visitor pattern for the ST construction. One way to use the PM is to inherit the visitor interface and at runtime to construct a custom ST. The second way is to generate an abstract or a concrete syntax tre… Read moree architect - an object that will construct automatically the ST as instances of object oriented classes. After the ST is complete, the client code can use it and free it on demand. The automatically generated ST are supporting iterative and recursive iterations, to string conversions and destructors.

The client may use the trees directly or write a code (glue code), that connects the automatically generated ST structures with the actual client program. This level of abstraction brings a lot of benefits in the development process, because changes to the language grammar may require only changes to the glue code, not to the whole client program.

For more information: Tunnel Grammar Studio

At 2019 Nov 17, 16:48 GMT in news channel Versions

When writing a grammar, it is difficult to imagine (especially in the big grammars) all the connections and internal states that will be generated by the parser generator.
Tunnel Grammar Studio (TGS) is a parser generator which provides a visualization of your grammars as automata. For debugging purposes, in real-time, with the help of the graphical unit interface, the developer can step by step, forward and backward, observe the creation of the syntax tree (generic, abstract or concrete) for the supplied in… Read moreput. Additionally to the syntax tree, the developer can see the stages of the input bytes decoded as a Unicode characters, their formation in tokens by the lexer grammar and the current parsing stack. This helps the developer to better understand the language defined by the grammars and to see quickly, without a compilation and any external tools, how the actual parsing is going to be done for this particular input.
In the TGS Online Laboratory, you can view the automata for your grammars and generate syntax trees for a your input string, in real-time, directly in your browser.

For more information: Tunnel Grammar Studio

At 2019 Nov 17, 16:47 GMT in news channel Versions

Usually the parser generators provide a wide range of libraries with the generated parsers to ensure the compatibility with the different compilers and runtime environments. Tunnel Grammar Studio is a parser generator which generates stand-alone parsers. At the end of the parsing machine compilation process there is only a C++ source code which can be compiled by a C++ compiler that supports at least the C++98 standard. The generated to a source code parsing machines are using the standard C++ library for v… Read moreery basic operations only, to remain with a high runtime performance and low memory requirements regardless of the particular standard library implementation.
To avoid the dependency hell, the generated parsing machines can be easily included into the clients program without the need of any static or dynamic link libraries or any third party software in their run time. No matter which compiler is used, the parsing machines generated by Tunnel Grammar Studio do not depend on them.

For more information: Tunnel Grammar Studio

At 2019 Nov 17, 16:45 GMT in news channel Versions

Unicode support should be a default feature in any parser generator, but many of them are using their custom non-standard syntax that requires care to support Unicode. Some are allowing at most 16 bit Unicode code points in the grammar syntax, that forces the developer to write per byte Unicode character recognition.
Tunnel Grammar Studio is using the Augmented BNF (ABNF) Internet Standard 68 syntax, defined in RFC 5234 and updated by RFC 7405 for case-sensitive string support. It allows the developer direct… Read morely to write grammars with a full Unicode support.
There are many different character decoders available to be included into the generated parser: UTF-8, UTF-16/32 (little and big endian) as well as a Universal decoder that will first check the input stream for UTF-8/16/32 and then fall back automatically to ISO 8859-1 (Latin-1) or to ASCII depending from the compiled decoders.
All of this is available out of the box, without the need, the user to write any additional code, but the grammar and to choose the required input decoders. This flexibility ensures that the generated parser can be deployed universally and internationally, no matter what input format the end-user will use.

For more information: Tunnel Grammar Studio

At 2019 Nov 17, 16:44 GMT in news channel Versions