How do I extract a specific pattern from a TokenId to create a terminal?

SyntaxEditor for WPF Forum

Posted 10 months ago by JP Garza
Version: 19.1.0685
Avatar
Hello,

We have a TokenId called Punctuation which has 4 patterns (. , : ;). If we wanted to create a grammar Terminal called @semicolon, how could I specify to use the ';' pattern only from the Punctuation TokenId?

var @semiColon = new Terminal(TokenId.Punctuation, "Semicolon") { ErrorAlias = "';'" };
 
 

Comments (5)

Posted 10 months ago by Actipro Software Support - Cleveland, OH, USA
Avatar

Hello,

Since the grammar-based parser works off of token IDs, you would have to assign the semi-colon its own unique token ID and then "classify" it the same as the other punctuation for the purposes of syntax highlighting.


Actipro Software Support

Posted 10 months ago by JP Garza
Avatar

Okay. Is it possible to go from a EBNF file to a grammar classs file? Would you recommend any way to do this if we already have our custom syntax EBNF rules?

Posted 10 months ago by Actipro Software Support - Cleveland, OH, USA
Avatar

There are so many forms of EBNF notation, so there's nothing to do an automated conversion.  You would need to hand write your EBNF rules in our C#-based EBNF-like syntax based on the documentation and samples.


Actipro Software Support

Posted 10 months ago by JP Garza
Avatar

Okay. Could you maybe provide an example of how the C# Operator token was converted into terminals? From the CSharp.langproj I can see you guys did this (this also applies to othe TokenIds like NativeType or ReservedWord):

https://i.imgur.com/fcz41lU.png

How was the terminals created for the different operators?

Answer - Posted 10 months ago by Actipro Software Support - Cleveland, OH, USA
Avatar

Hello,

Oh that CSharp.langdef was just a basic definition for a lexer to be used with syntax highlighting.  In that scenario, it doesn't matter if multiple keywords/operators use the same token ID.

However in our .NET Languages Add-on where we have a grammar-based parser, we use a programmatic lexer that defines a separate token ID for each keyword, operator, etc.  That's the only way a parser can identify each keyword/operator.


Actipro Software Support

The latest build of this product (v21.1.1) was released 1 month ago, which was after the last post in this thread.

Add Comment

Please log in to a validated account to post comments.