How do I extract a specific pattern from a TokenId to create a terminal?

SyntaxEditor for WPF Forum

Posted 6 months ago by JP Garza
Version: 19.1.0685
Avatar
Hello,

We have a TokenId called Punctuation which has 4 patterns (. , : ;). If we wanted to create a grammar Terminal called @semicolon, how could I specify to use the ';' pattern only from the Punctuation TokenId?

var @semiColon = new Terminal(TokenId.Punctuation, "Semicolon") { ErrorAlias = "';'" };
 
 

Comments (5)

Posted 6 months ago by Actipro Software Support - Cleveland, OH, USA
Avatar

Hello,

Since the grammar-based parser works off of token IDs, you would have to assign the semi-colon its own unique token ID and then "classify" it the same as the other punctuation for the purposes of syntax highlighting.


Actipro Software Support

Posted 6 months ago by JP Garza
Avatar

Okay. Is it possible to go from a EBNF file to a grammar classs file? Would you recommend any way to do this if we already have our custom syntax EBNF rules?

Posted 6 months ago by Actipro Software Support - Cleveland, OH, USA
Avatar

There are so many forms of EBNF notation, so there's nothing to do an automated conversion.  You would need to hand write your EBNF rules in our C#-based EBNF-like syntax based on the documentation and samples.


Actipro Software Support

Posted 6 months ago by JP Garza
Avatar

Okay. Could you maybe provide an example of how the C# Operator token was converted into terminals? From the CSharp.langproj I can see you guys did this (this also applies to othe TokenIds like NativeType or ReservedWord):

https://i.imgur.com/fcz41lU.png

How was the terminals created for the different operators?

Answer - Posted 6 months ago by Actipro Software Support - Cleveland, OH, USA
Avatar

Hello,

Oh that CSharp.langdef was just a basic definition for a lexer to be used with syntax highlighting.  In that scenario, it doesn't matter if multiple keywords/operators use the same token ID.

However in our .NET Languages Add-on where we have a grammar-based parser, we use a programmatic lexer that defines a separate token ID for each keyword, operator, etc.  That's the only way a parser can identify each keyword/operator.


Actipro Software Support

The latest build of this product (v2020.1.1) was released 15 days ago, which was after the last post in this thread.

Add Comment

Please log in to a validated account to post comments.