How to get the context in the GetNextToken method ?

Hi Peter,

You have access to a ITextBufferReader, which is a low-level plain text reader.  You won't have access to previous tokens since the lexers are creating the tokens as the GetNextToken method is called.  For pre-processor directives, you might want to have them transition to a new lexical state.  That way when you get to your '<' you'll know you're in a pre-processor directive.  Then exit the state on the '\n' character.

Alternatively, you can use the ITextBufferReader to scan backwards through the raw text characters and do your determination with that.  Just make sure you return its Offset to where it should be following the token creation.

Hi,

Thank you for that.

I defined two lexical states, and it works fine, but another question:

Because the preprocessor directives can be used in everywhere(just like comment), so I skipped them in TokenReader class, but also skipped the parser, if I want to parse them in grammar class, how to do ? Two SyntaxLanguage classes and two Grammar classes with CreateChangeBatch calling ?

Thanks again!

Hi Peter,

That can be a tricky scenario.  You may want to make a custom IParser that does two rounds of parsing.  The first would just look for pre-processor directives and store info for those.  The second would be more like your normal one now.  Then merge the IParseData results together into a new IParseData and return that from IParser.Parse.