SyntaxLanguageParsingExtension: ITextSnapshotReader recognize no tokens - SyntaxEditor for Silverlight Forum

Posted 11 years ago by Christel

Version: 13.2.0161

Hi,

I use the SyntaxLanguageParsingExtension with Parse(ISyntaxLanguage, Text)

Before, the SyntaxLanguage is created, and when creating the SyntaxLanguage, it is initialized by a self-defined langdef-File for the syntax.

When debugging, the method

public override ActiproSoftware.Text.Parsing.IParseData Parse(ActiproSoftware.Text.Parsing.IParseRequest request)

is reached.

ITextSnapshotReader was created successfully, but

IToken token = reader.ReadToken();

returns no token:

TOKEN == null.

The reader now does not recognize tokens, it reads sign by sign.

I use a dynamic lexer and did only use the langdef-File for initializing the SyntaxLanguage. No special TokenReader is implemented.

But parsing with an editor does work perfectly.

What did I forgot or made wrong?

Thanks!

Christel

Comments (3)

Posted 11 years ago by Actipro Software Support - Cleveland, OH, USA

Hi Christel,

It's hard to say without debugging but the two main things that a reader needs to be able to support token reading is an ILexer service as well as a token tagger provider service, generally something like this for languages with dynamic lexers:

language.RegisterService(new TokenTaggerProvider<TokenTagger>());

If you are missing either of those, then no tokens will be found.

Actipro Software Support

Posted 11 years ago by Christel

...

And why does the token reading work when parsing the text in the editor itself

although the language is initialized by the langdef-File as well without a TokenTagger?

Why does it work with editor but not with the ParsingExtension?

Should I better use the the generated classes from the LanguageDesigner then the generated langdef file to initalize the language for parsing?

Thanks

Christel

Posted 11 years ago by Actipro Software Support - Cleveland, OH, USA

Thanks for the sample sent to our support address the other day Christel. For anyone reading this, the problem ended up being that the custom IParser was trying to use a snapshot reader instead of interacting with the ILexer directly. Due to how the SyntaxLanguageParsingExtension.Parse method is structured for efficiency, snapshot readers can't be used in the parser there. Be sure to use the ILexer directly to get tokens instead.

Actipro Software Support

The latest build of this product (v18.1 build 0233) was released 6 years ago, which was after the last post in this thread.