modifying language during lexical parse?

SyntaxEditor for Windows Forms Forum

Posted 13 years ago by Benjamin Lewis
Avatar
Is it possible to hook into the lexical parser, and modify the language during the parsing sequence? I'd like to recognize variables declared at the beginning of a block, and add them as explicit patterns.

For example, for a pascal-ish language, I'd like to do something like the following

var foo : int; // recognize "foo" as a variable via regex match, add to language as
// explicit pattern
begin
foo := 3; // highlight using syntax rules for variables
end.


Thanks,
--
Benjamin Lewis

Comments (7)

Posted 13 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar
You probably can't do that in v3.1 however in v4.0 that we're working on a new feature is programmatic lexical parsers. So that means you can write a lexical parser in code and probably do this sort of thing.


Actipro Software Support

Posted 13 years ago by Benjamin Lewis
Avatar
I've got it to mostly work now -- when the document is modified, I loop through all the tokens, and any VariableTokens (which were initially found via regex matching their declarations) get added as explicit patterns. Then I re-parse the document with the updated language.

It's admittedly a bit klunky, but it works -- except that for some reason, the first time the document is loaded from my data source, the Document.Reparse() appears to have no effect.

--
Benjamin Lewis
Posted 13 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar
Make sure you set SyntaxLanguage.IsUpdating to true before making any changes to the language and then reset it to false afterwards. Doing that will also reparse your document so no need to call Document.Reparse.

I have a feeling that you were missing the IsUpdating calls.


Actipro Software Support

Posted 13 years ago by Benjamin Lewis
Avatar
Nope; the calls are there. It works fine everywhere but at the initial document load. I'll try removing the call to Reparse, though.

Thanks,
--
Benjamin Lewis
Posted 13 years ago by Benjamin Lewis
Avatar
It turns out the problem was unrelated -- my documents are being loaded from data grids, and I wasn't aware that when I select one row, the next N rows get loaded as well for efficiency. Since I'm reusing my language across documents, my modifications were being made properly but then getting cleared by subsequent loads.

You're correct, my call to Reparse() was superfluous.

I don't suppose there's some way to reuse a static base language across my documents, but to have local modifications on a per-document basis?

Thanks,
--
Benjamin Lewis
Posted 13 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar
Unfortunately no since the language definition dictates how the lexical parser operates. So if you are using it in all your documents then any changes made will apply to all those documents. In this case if you are making document-specific changes to the language then you'll have to make a new language instance for each document.


Actipro Software Support

Posted 13 years ago by Benjamin Lewis
Avatar
The language updating and reparsing happens quickly enough that it's not an issue, so now I'm just calling my language updating routine when the document becomes visible/focused. Tastes great, less (memory) filling!

I suppose speed could become an issue for longer documents, but the code in these documents is for defining database stored procedures, and tends to be quite short.

Thanks again; this is starting to look great!
--
Benjamin Lewis
The latest build of this product (v2018.1 build 0341) was released 3 months ago, which was after the last post in this thread.

Add Comment

Please log in to a validated account to post comments.