Is an XML language definition file absolutely necessary?

SyntaxEditor for Windows Forms Forum

Posted 7 years ago by David Stonier-Gibson
Version: 12.1.0302

Hi, I am evaluating SyntaxEditor for a new IDE I need to make for our embedded controller products, and it is definitely the front runner. Our SPLat language is much closer to an assembler than a structured  language. I am writing the IDE in VB2010 Express.

I am a bit hesitant about using XML files for my language definition because:-

  • I can see no easy way of generating the XML for hundreds of instructions
  • I am not at all familiar with XML, so it represents a rather daunting challenge
  • I want my language definition tables to include all kinds of "private" stuff like instruction signatures and other special tags.
  • I am planning a scheme where there is a base language plus product-specific extension to support special purpose and varying hardware configurations. That means I need to merge two or more instruction sets at the time the running IDE is connected to a particular hardware setup.

All in all, it seems to me I would be better off with my own proprietary file format, or possibly even CSV, then generate the language variant at run time and then feed it to the SyntaxEditor lexer.

In the Actipro documentation here seem to be references to the abillity to programmatically send dynamic language defintions to the parser. For example in SyntaxEditor Dynamic Languages Guide:Overview it says"

"The language can be defined either by using a dynamic language XML definition file or by 
constructing the language programmatically."

 If I am mis-interpreting the docco, and it's is not possible to bypass XML, I have some questions:

  1. Will SyntaxEditor choke if it sees tags in the  Dynamic Language XML Definition it does not recognize? (i.e. can I embedd my own custom stuff in there?)
  2. Can it be sent several XML files with cumulative effect?

Plan C would be to read in my own proprietary files, mash them up, generate a temporary XML file to the Dynamic Language XML Definition Specification and then read it into SyntaxEditor. Is that what I'll have to do?

Note: I am aware of the option to write my own lexer. That is not what I want to do.

[Modified 7 years ago]

Comments (4)

Posted 7 years ago by Tobias Lingemann - Software Devolpment Engineer, Vector Informatik GmbH

You probably want to write your own SyntaxLanguage.
I would recommend writing your own lexer too since its much faster. But you even can use XML file to define the lexer and specify the languge with the SyntaxLanguage Tag.
Using the XML file for the lexer is a good start, because its very easy, but in long-term its best to write your own classes. Not only is it much faster, you are also more flexible.

There is no way to load languages from a different format, but you can do everything you want to, if you implement it in code.
Look at the SimpleLanguage Add-On to get the basic idea how to implement your own language and lexer.

Maybe you should read the "Language Defintion Guide" and "Dynamic Languages Guide" in the docu. Especially the first chapters give you a good hint, how the system from lexer to parser and language works.

[Modified 7 years ago]

Best regards, Tobias Lingemann.

Posted 7 years ago by David Stonier-Gibson

Thank you. I shall look at the possibility of making my own lexer, though it sounds rather scary. I am still struggling to get my head around .NET after coming from VB6.

Posted 7 years ago by Tobias Lingemann - Software Devolpment Engineer, Vector Informatik GmbH

Well, you don't make a code-editor within a week.
That's a complex task, especially if you wan't a good product.

To be honest, I don't think it's a one-man-job either, especially if you struggle with .net.
I have never really used VB, but I find it absolutely unreadable if you are a C# developer. And if I can't really read it, I can't really work with it...

You will need some time to learn all corners of the SyntaxEditor. I started evaluating more than a year ago and by now our first product using it has been released. The second one will come out soon.

Best regards, Tobias Lingemann.

Answer - Posted 7 years ago by Actipro Software Support - Cleveland, OH, USA

Hi David,

Your two options with lexing is to use a dynamic language or a more basic language where you write the lexical parser code by hand.  It sounds like you are trying to stick with the dynamic language lexer.

The good news is that you can easily make your base dynamic language XML definition that defines all your common pieces of the lexical grammar.  It's really not hard to clone one of our sample XML definitions and modify it for your needs.  Then leave some empty stub lexical pattern groups in there for where you'll dynamically put in your instruction set keywords.

At run-time in code you can programmatically modify the dynamic language.  If you look at the DynamicallyCreateForm code in our SdiEditor demo, you can see an instance of that.  Just wrap the language updates with IsUpdating = true and IsUpdating = false calls.  But in between is where you'd dynamically pull in your instruction set keywords into the pattern groups you defined in the XML definition.

As for your questions, the XML reader probably won't like unknown tags in it.  You could however use XML comments anywhere you want.

Only one dynamic language XML definition can be loaded at a time unless you are doing a merged language scenario.  The HTML language sample does that where it transitions to other languages like CSS and back.  You need to have well-defined delimiters to know when to transition and come back.

The only other comment I have is that if you are loading hundreds of keywords for your instruction set, you'll have to see how the performance ends up being.  Dynamic language lexers are better at handling regex-based patterns and not hundreds of keywords.  So if you notice performance issues, you may have to switch to making a programmatic lexical parser.  Our Simple language shows an example of that.

Actipro Software Support

The latest build of this product (v2018.1 build 0341) was released 3 months ago, which was after the last post in this thread.

Add Comment

Please log in to a validated account to post comments.