Yes. YOu want to backtrack to at least the previously valid token (prior to the one that contains the current offset), otherwise, you get f u n c t i o n as different identfiers as you are typing.
I think this is done in the example, if not, it's done in this code:
public override TextRange PerformLexicalParse(Document document, TextRange parseTextRange, ILexicalParseTarget parseTarget)
{
// Update the parse text range
int startDocumentLineIndex;
int endDocumentLineIndex = document.Lines.IndexOf(parseTextRange.EndOffset);
if (document.Lines[endDocumentLineIndex].Contains(parseTextRange.StartOffset))
startDocumentLineIndex = Math.Max(0, endDocumentLineIndex - this.LexicalParserLineLookBehind);
else
startDocumentLineIndex = Math.Max(0, document.Lines.IndexOf(parseTextRange.StartOffset) - this.LexicalParserLineLookBehind);
int parseStartOffset = document.Lines[startDocumentLineIndex].StartOffset;
int parseThroughOffset = document.Lines[Math.Min(document.Lines.Count - 1, endDocumentLineIndex + this.LexicalParserLineLookAhead)].EndOffset;
// Move back to the start of the token
int lexicalStateID = 0;
if (parseStartOffset > 0)
{
IToken initialToken = document.Tokens.GetTokenAtOffset(parseStartOffset);
lexicalStateID = initialToken.LexicalStateID;
if (initialToken.StartOffset < parseStartOffset)
parseStartOffset = initialToken.StartOffset;
}
// Initialize the modified start/end offsets
int modifiedStartOffset = parseTextRange.StartOffset;
int modifiedEndOffset = parseTextRange.EndOffset;
// NOTE: LineIndex parameter here is wrong (0) but we don't care for highlighting parsing
ITextBufferReader reader = new StringBuilderTextBufferReader(document.GetCoreTextBuffer(), 0, parseStartOffset);
IToken token;
// Notify the parse target that parsing is starting
parseTarget.OnPreParse(parseStartOffset);
// Loop and generate all the tokens... this is optimized for incremental parsing
while (!reader.IsAtEnd)
{
// Get the next token
token = lexicalParser.GetNextToken(reader, lexicalStateID);
lexicalStateID = token.LexicalStateID;
// Update the parse target
if (parseTarget.OnTokenParsed(token, reader.Offset - token.StartOffset))
{
if (token.StartOffset < modifiedStartOffset)
modifiedStartOffset = token.StartOffset;
if (reader.Offset > modifiedEndOffset)
modifiedEndOffset = reader.Offset;
}
else
{
// Quit the loop if nothing was changed
if (reader.Offset >= parseThroughOffset)
break;
}
}
// Notify the parse target that parsing is complete
parseTarget.OnPostParse();
return new TextRange(modifiedStartOffset, modifiedEndOffset);
}
This code is from Actipro's example code that they sent me.
As for the NonMergableTokenBase, it's in ActiproSoftware.SyntaxEditor namespace after around build 240 or so, I think. They added it pretty recently (last several months or so I think).
Kelly Leahy
Software Architect
Milliman, USA
