integrating dynamic languages

SyntaxEditor for Windows Forms Forum

Posted 13 years ago by Krzysztof

I have written a language in xml. Now I want to write a semantic parser for it.

I know this has been a subject to many posts, but none satisfy my needs - the answers are usualy like: 'you usualy do that by writting language in code, but if you instst, you may see how we did that in out web addon'. But you don't attach code of that addon to Syntaxeditor's samples, so I can't see how it's done. You also never put any code fragments.

I thougt MergableSyntaxLanguage is all about that, but it turned out that it's rather about jumping from 1 language to another, not integrating diffrent functionalities of a single language.

I tried to use SyntaxLanguageTypeName, but your example with it is about outlining, which I don't need at all. I tried to attach a language with syntax parsing that way, but it just broke the lexer (there was no highlighting any more, no syntax parsing and no info about what went wrong).

Or mayby there is an even better thing - Thats why I thought that it is maybe possible the other way - to create a language with semantic parsing and than attach a xml file that does the lexing and highlighting. That way I would be able to debug it with VS...

So my request is: could you show me how (if it's possible at all of course) to get a lexer from xml-defined language, or add it to a language in code?

Comments (7)

Posted 13 years ago by Actipro Software Support - Cleveland, OH, USA
On a quick side note before I answer your question, we've been working on redesigning a next generation platform for SyntaxEditor that is in our WPF and Silverlight versions. Over there, we have a much nicer design where lexers and parsers and other language related features are all just services that you can easily attach/detach to the core SyntaxLanguage. We've done a lot of work to make the overall usage and implementation of languages much easier than in WinForms.

The WinForms SyntaxEditor (until we eventually get those newer designs ported over) is a bit more convoluted. What you'd do to load up an XML-based lexer and add a parser is this:

1) First create your dynamic language XML definition file.

2) Create a code-behind class for the language similar to our examples that show outlining. Use the SyntaxLanguageTypeName attribute in your XML file to tie it to the class (be careful of namespace and assembly names being correct). If you don't need outlining you can just inherit DynamicSyntaxLanguage for your class.

3) In your code-behind class, add this:

public override void PerformSemanticParse(Document document, TextRange parseTextRange, SemanticParseFlags flags) {
SemanticParserServiceRequest request = new SemanticParserServiceRequest(SemanticParserServiceRequest.MediumPriority,
document, parseTextRange, flags, this, document);

That will make the parser get called in a worker thread as long as you've started the semantic parser service via SemanticParserService.Start.

4) Have your code-behind class implement the ISemanticParserServiceProcessor interface with a method like this:

void ISemanticParserServiceProcessor.Process(SemanticParserServiceRequest request) {
request.SemanticParseData = MergableLexicalParserManager.PerformSemanticParse(this, request.TextBufferReader, request.Filename) as ISemanticParseData;

5) Finally, implement this method and put your parser call in here and return the parse result:

protected override object PerformSemanticParse(MergableLexicalParserManager manager) {
MergableRecursiveDescentLexicalParser lexicalParser = new MergableRecursiveDescentLexicalParser(this, manager);

return yourParser.Parse(lexicalParser)

That should get you working. As mentioned above, our next-gen design in the WPF/Silverlight versions is much easier than this though since we recognized this is a bit tricky to grasp. We hope to eventually port the newer, easier designs back to WinForms.

Actipro Software Support

Posted 13 years ago by Krzysztof
Thanks a lot! I'll try this out right away.

As for SE for WPF/Silverlight - I've started with it and I admit it's a way nicer, but it turned out (after I had my grammar nice and running) that my client needs it working with .net 2.0, so I had to start from the begining again.
Posted 13 years ago by Krzysztof
I've got a problem loading TokenID class:

An error occurred while loading the DynamicSyntaxLanguage 'MT':
The token ID type 'Bus.Grammar.MtTokenID, Bus.Grammar' could not be loaded.
The problem occurred near line 1, position 2
in D:\Documents and Settings\kp\Pulpit\Edytor\Edytor\bin\Release\Grammar\Lexer.xml.

The beginning of my Lexer.xml looks like that:
<SyntaxLanguage Key="MT" LanguageDefinitionVersion="4.0" Secure="True"
                TokenIDTypeName="Bus.Grammar.MtTokenID, Bus.Grammar"
My solution looks like this:
- Windows forms project (with SyntaxEditor control - here I load Lexer.xml and add a reference to the class library project below)
- class library project - here I have the MtTokenId class (that was created when I designed a little bit of a grammar and used your grammar designer to create code)

How do I connect these two?
Posted 13 years ago by Actipro Software Support - Cleveland, OH, USA
That is saying look in an assembly "Bus.Grammar.dll" for a type named "MtTokenID" in the "Bus.Grammar" namespace. Are you sure your assembly name is correct and also that the namespace is correct? You can use the Object Browser in Visual Studio to be sure. Also the type name might be case sensitive so be careful it is exactly what the Object Browser says.

If you do that right, then it should load.

Actipro Software Support

Posted 13 years ago by Krzysztof
Here I load the xml:
namespace Edytor
using Bus;

    public partial class Form1 : Form
        public Form1()
            string path = System.IO.Path.GetFullPath("Grammar/Lexer.xml");
            syntaxEditor1.Document.LoadLanguageFromXml(path, 0); //here it throes the exception
This is my library project:
namespace Bus.Grammar
    #region Token IDs
    /// <summary>
    /// Contains the token IDs for the <c>MT</c> language.
    /// </summary>
    public class MtTokenID
[Modified at 03/12/2011 08:13 AM]

As You can see, the namespaces in MtTokenID and TokenIDTypeName match.

[Modified at 03/12/2011 08:17 AM]
Posted 13 years ago by Actipro Software Support - Cleveland, OH, USA
Hi Krzysztof,

Yes that looks ok. Are you positive that it is in an assembly named "Bus.Grammar.dll"? Also check that there is a public parameterless constructor on the token ID class.

If that doesn't help, please make a new simple sample project showing the issue and email it to us so we can have a look. Please don't include any .exe files in the ZIP and rename the .zip file extension so it doesn't get spam blocked.

Actipro Software Support

Posted 13 years ago by Krzysztof
an assembly named "Bus.Grammar.dll"

Hey! Now we're talking!

I thought it was just a namespace - like in SyntaxLanguageTypeName - not a separate dll, so I attached my class to my project everywhere I could and wondered what I did wrong...

Now when I look at it in help files, it does mention
Make sure the attribute includes the full namespace, Type name, and assembly specification
, but that are 3 parameters (1 - namespace, 2 - Type name, 3 - assembly), so it's kind'a misleading (and hidden in text anyway. Look at msdn - they outline the parameters, return value and such, so that people knew how to use them right away).

[Modified at 03/14/2011 01:00 PM]
The latest build of this product (v24.1.0) was released 4 months ago, which was after the last post in this thread.

Add Comment

Please log in to a validated account to post comments.