Right, in your example, the state change affected everything else. So PostParse is really where we want to grab the data.
How about this? Instead of just providing string values for deleted text, would it be possible to persist the lexical and semantic data along with the text? So that one could iterate through the tokens of the recently deleted (or even added) text? The only real question here is how to handle the partial deletion of a token like in your example.
Even though just one backslash is being removed, you would not want to do another lexical parse of the newly deleted data.
OK, final thought, persist the lexical/semantic data after deletion so that lexical parsing can take place on the rest of the text but we can still dig through the deleted text and really know what was deleted. Second, include a boolean in with these tokens to determine if the token in question is a "partial token". If so, then we know to go back and look at the rest of the token if need be. Or maybe, just include another string value for the entire token. In other words, in your example if the second slash was removed, then the deleted token key is "CommentStartToken" but the 'FullTokenText' value or some such is "//".
Does any of this make sense? Thanks.