# Dynamic Lexer Bug With New Lines

SyntaxEditor for WPF Forum

Posted 9 years ago by Emil
Version: 12.2.0570
Platform: .NET 4.0
Environment: Windows 7 (64-bit)

I am trying out your product and I encountered a bug in the dynamic lexer v2012.2.570.

Either the lexing is not being done properly or the syntax highlighting is not being updated for multi-line patterns.

Here is how to reproduce the bug:

1. Open the following language with the language editor:

<?xml version="1.0" encoding="utf-8"?>
<LanguageProject LanguageKey="MyLanguage" LanguageDescription="My Language" OutputLanguageDefinition="False" OutputLanguageKey="CSharp" OutputNamespaceName="MyCompanyName.Text.Languages.MyLanguage" xmlns="http://schemas.actiprosoftware.com/langproj/1.0">
<!-- Classification types -->
<LanguageProject.ClassificationTypes>
<ClassificationType Key="Black" DefaultStyle="#FF000000" />
<ClassificationType Key="Red" DefaultStyle="#FFFF0000" />
</LanguageProject.ClassificationTypes>
<!-- Lexer -->
<LanguageProject.Lexer>
<DynamicLexer>
<!-- Default state -->
<State Key="Default" DefaultClassificationTypeKey="Black">
<RegexPatternGroup ClassificationTypeKey="Red" Pattern="x[^x]*x" />
</State>
</DynamicLexer>
</LanguageProject.Lexer>
</LanguageProject>


2. Click on "Build", then click on "Live Test".

3. In the Live Test document, paste the following (with the new lines):

a
x
b
x
c

4. You should see that both x's and the b are all red.
5. Place the caret to the left of the last x and hit the "DEL" key to delete the last x. Notice that the c is not red, but it should be. Similar problems happen when deleting the first x.

Please let me know if I'm doing something wrong or if this is indeed a bug.

Thanks!

Posted 9 years ago by Emil

Was anyone else able to reproduce it?

I can reproduce the same bug in the C# editor demo as well:

2. Type the following text (make sure there are no extra spaces or tabs):

aaa
/*
bbb
*/
ccc

3. The aaa and ccc is  black and the bbb is green.

4. Position the caret to the left of the closing comment "*/".

5. Press SHIFT+END to select the line with the closing comment.

6. Press DEL to delete the "*/".

7. Notice that the ccc is black but it should be green.

8. Now go to the line with the "ccc" and add another "c" to it. It turns green all of a sudden.

Thanks,
Emil

Posted 9 years ago

Hi Emil,

Yes we can repro it and are still researching the problem.  It has something to do with the optimizations for refreshing classification tags, and is pretty tricky code.

Actipro Software Support

Answer - Posted 9 years ago

Hi Emil,

We fixed the problem for the next maintenance release.

One other note for you too.  The way your red pattern is defined won't work well right now since it's requiring an end 'x'.  Say you have a couple x chars many lines apart and you delete the last.  After re-adding an 'x' at that same spot, the lexer won't pick up that it's valid again since the lexer is optimized to only go back a couple lines when typing new characters (unless an existing token puts it back farther).  So to fix that issue, change your pattern to this (with ? at the end):

x[^x]*x?

With that update and ours for the next maintenance release, it will be fully working.

Actipro Software Support

Posted 9 years ago by Emil

That's great that you fixed it so fast. I'm considering buying the SyntaxEditor to use in a LaTeX editor application.

In LaTeX, it is very common to do something like this:

$$multi-line math goes here multi-line math goes here it should be able to parse control sequences like this: \pi there might even be symbols like this: \$$

Will it be possible to write a dynamic lexer which has child elements withing the multiline $$...$$ region like in the above example? Does your workaround work for such cases?

Thanks!

Posted 9 years ago

Hi Emil,

In the scenario want to handle, you'd define things a bit differently.  Instead of having a single token for the $$...$$ range (which would NOT allow you to syntax highlight other things like control sequences inside it), you'd want to make a lexical state that starts with $$and ends with$$.  Many of our sample lexer definitions show this sort of concepts.  For instance the CSharp.langdef file has a String state in it that can handle escape sequences and optionally highlight them differently.  So check that out.

The bug you had found shouldn't come into play with that sort of lexer setup either.

Actipro Software Support

The latest build of this product (v22.1.0) was released 1 month ago, which was after the last post in this thread.