How do I get started reading Tokens and Parse data?

SyntaxEditor for WPF Forum

Posted 9 years ago by Bob Puckett
Version: 15.1.0623

I have read the docs and have not found an answer yet.

I need to write some code that translates BASIC and Pascal source to Python.

I could write my own parser for those languages, but my current strategy is to load the BASIC code into a SyntaxEditor control, apply the free VBScript and Pascal langauges to the editor, let the language get the tokens and parse the code, and then let me get the token stream and AST to use as input to my python code generator.

I have the editor created and the language highlighted in the editor for both BASIC and PASCAL just fine.

Now, how do I get to the tokens and parse data?

Here is where I have started, but don't really see how to get the TokenSet or other streams that I will need.

var parseData = mySyntaxEditor.Document.ParseData;
IParser parser = mySyntaxEditor.Document.Language.GetParser();
var lexer = mySyntaxEditor.Document.Language.GetLexer();

Where do I start?  

Thanks very much.

Comments (1)

Answer - Posted 9 years ago by Actipro Software Support - Cleveland, OH, USA

Hi Bob,

To get tokens, you would get a snapshot reader like:

var reader = mySyntaxEditor.Document.CurrentSnapshot.GetReader(0);

Then use the methods on that ot iterate through the tokens.  The "Text/Parsing Framework / Core Text Featuers / Scanning Text Using a Reader" documentation topic talks about this.

The free language definitions for BASIC and Pascall that come with SyntaxEditor do tokenizing but they don't have an advanced parser.  So there isn't any parse data available for them.

Actipro Software Support

The latest build of this product (v24.1.2) was released 3 months ago, which was after the last post in this thread.

Add Comment

Please log in to a validated account to post comments.