ITextSnapshotReader.PeekTokenReverse

Hi,

I just found out that there is a difference in the snapshot reader between before and after calling the PeekTokenReverse method on the ITextSnapshotReader.

How to reproduce it:

snapshot text: "a" (only one character)

reader offset: 1

call reader.PeekTokenReverse()

Before calling reader.PeekTokenReverse() is called, reader.IsAtTokenStart is true and reader.Token is {[MergableToken: (1, 1)]}. But after calling it, reader.IsAtTokenStart is false and reader.Token is null.

I'm seeing this in CSharpContextFactory.UpdateTargetExpression. Inside that method, reader.PeekTokenReverse() is called when the LexicalStateId of the token is DocumentEnd or LineTerminator.

Thank you.