Starting and ending a lexical state with the same token

SyntaxEditor for WPF Forum

Posted 5 years ago by Matt Gerrans
Version: 14.2.0610

Our DSL is token delimited, so for example a word in curlies is one type of variable and one in angle brackets is another.  This is working fine matching pairs of delimiters like {}, <>, [] and (), but we also have some cases where the start and end delimiter are the same (eg. start ~variable~ or ^variable^).  Using the Language Designer, I have a different explicit pattern for the Start Pattern Group and End Pattern Group in the former case.   In the latter case, I have the same pattern, but that doesn't seem to do the trick.   When I look at those with the same start/end delimiter in the Live Test, they don't transition out of their lexical state like those with different delimiters do.   What is the right way to get them to behave similarly?   Do they need to have a state transition?

Comments (4)

Posted 5 years ago by Matt Gerrans

Okay, I figured out the problem and you guys are due for a bit of scolding.   Apparently, the default lexical state must be named "Default" and nothing else.    What I had done is name all my states things like ConceptLexicalState, CriteriaLexicalState and so on, including changing "Default" to "DefaultLexicalState" for consistency.    It was some time later (and after making other changes, like adding some new states and classifications) that I noticed that all the syntax highlighting had stopped working.  That's why it took a while to make the connection.  It didn't seem natural that any particular lexical state would be required to have a particular name, so it took me a lot of tweaking before I tried reverting all the names.

I think either it changing the name of the default lexical state's name should work correctly, or at the very least it needs a special case to disallow changing it in the Language Designer.   Of course, you can't prevent someone doing it directly in the langproj file with a text editor, but you could make it an error when they try to regenerate.   Best would be to just allow renaming it; I suspect it is just a bug where "Default" is hard-coded somehwere?   Anyway, someone has been a naughty programmer and should be feeling suitably ashamed!

Posted 5 years ago by Actipro Software Support - Cleveland, OH, USA

Hi Matt,

If you are just using these for simple variable names, instead of using lexical states, wouldn't it be easier to just make a regex pattern in the default state such as:


Back on the naming issue though, we haven't heard of anyone running into that before so sorry it caused some trouble for you.  After looking at the SyntaxLanguageDefinitionSerializer code, as it reads in all the lexical states, it just adds them to the lexical states collection of the DynamicLexer being created.  If there isn't one with a "Default" key then nothing will update the default lexical state that is in place (in the lexer's DefaultLexicalState property) meaning that it will effectively be empty of patterns.  That's probably what you ran into.  We will add an error in the Language Designer on build that warns you when there is no lexical state with a key of "Default".

Actipro Software Support

Posted 4 years ago by Vikrant Bhosale



I am seeing the exact same issue

I got response on that ticket but it doesnt seem to work when I tried it.

Posted 4 years ago by Actipro Software Support - Cleveland, OH, USA

Hi Vikrant,

If you are having trouble getting working what we described in the private ticket then please reply back directly to that with more info so we can help you further.

Actipro Software Support

The latest build of this product (v2019.1 build 0683) was released 1 month ago, which was after the last post in this thread.

Add Comment

Please log in to a validated account to post comments.