"\n" is non whitespace token?

SyntaxEditor for Windows Forms Forum

Posted 16 years ago by Alex_3333
Avatar
I set code lower to the main form of the sample application with default sample document:

TokenStream tokenstream = this.editor.Document.GetTokenStream( 0 );
Token token = null;
while ( !( token = tokenstream.Read() ).IsEndOfDocument )
if ( token.Text == "\n" && !token.IsWhitespace )
MessageBox.Show( "Error" );

and see "Error" three times.

Comments (3)

Posted 16 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar
You have to set the IsWhitespace flag in your language definitions for each pattern group. The lexical scanner doesn't assign that flag dynamically. See this example in the C# language definition:

<RegexPatternGroup Token="LineTerminatorToken" PatternValue="{LineTerminatorMacro}" IsWhitespace="True" />


Actipro Software Support

Posted 16 years ago by Alex_3333
Avatar
Sorry but I am slow-witted <IMG SRC="smile.gif" border="0"> I tell about SyntaxEditor' sample application and its sample C# language definition. As I understand mentioned pattern group sescribes line terminator tokens at the document. Why 3 of 56 "\n" tokens are non whitespace one?
Posted 16 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar
I just took a look and see what you mean. Some of the pattern groups that end scopes with a line terminator do not have the IsWhitespace="true" flag set. I added those in for the next release.


Actipro Software Support

The latest build of this product (v2020.1 build 0402) was released 24 days ago, which was after the last post in this thread.

Add Comment

Please log in to a validated account to post comments.