integrating dynamic languages

On a quick side note before I answer your question, we've been working on redesigning a next generation platform for SyntaxEditor that is in our WPF and Silverlight versions. Over there, we have a much nicer design where lexers and parsers and other language related features are all just services that you can easily attach/detach to the core SyntaxLanguage. We've done a lot of work to make the overall usage and implementation of languages much easier than in WinForms.

The WinForms SyntaxEditor (until we eventually get those newer designs ported over) is a bit more convoluted. What you'd do to load up an XML-based lexer and add a parser is this:

1) First create your dynamic language XML definition file.

2) Create a code-behind class for the language similar to our examples that show outlining. Use the SyntaxLanguageTypeName attribute in your XML file to tie it to the class (be careful of namespace and assembly names being correct). If you don't need outlining you can just inherit DynamicSyntaxLanguage for your class.

3) In your code-behind class, add this:

public override void PerformSemanticParse(Document document, TextRange parseTextRange, SemanticParseFlags flags) {
SemanticParserServiceRequest request = new SemanticParserServiceRequest(SemanticParserServiceRequest.MediumPriority,
document, parseTextRange, flags, this, document);
SemanticParserService.Parse(request);
}

That will make the parser get called in a worker thread as long as you've started the semantic parser service via SemanticParserService.Start.

4) Have your code-behind class implement the ISemanticParserServiceProcessor interface with a method like this:

void ISemanticParserServiceProcessor.Process(SemanticParserServiceRequest request) {
request.SemanticParseData = MergableLexicalParserManager.PerformSemanticParse(this, request.TextBufferReader, request.Filename) as ISemanticParseData;
}

5) Finally, implement this method and put your parser call in here and return the parse result:

protected override object PerformSemanticParse(MergableLexicalParserManager manager) {
MergableRecursiveDescentLexicalParser lexicalParser = new MergableRecursiveDescentLexicalParser(this, manager);
lexicalParser.InitializeTokens();

return yourParser.Parse(lexicalParser)
}

That should get you working. As mentioned above, our next-gen design in the WPF/Silverlight versions is much easier than this though since we recognized this is a bit tricky to grasp. We hope to eventually port the newer, easier designs back to WinForms.

Thanks a lot! I'll try this out right away.

As for SE for WPF/Silverlight - I've started with it and I admit it's a way nicer, but it turned out (after I had my grammar nice and running) that my client needs it working with .net 2.0, so I had to start from the begining again.

I've got a problem loading TokenID class:

An error occurred while loading the DynamicSyntaxLanguage 'MT':
The token ID type 'Bus.Grammar.MtTokenID, Bus.Grammar' could not be loaded.
The problem occurred near line 1, position 2
in D:\Documents and Settings\kp\Pulpit\Edytor\Edytor\bin\Release\Grammar\Lexer.xml.

The beginning of my Lexer.xml looks like that: My solution looks like this:
- Windows forms project (with SyntaxEditor control - here I load Lexer.xml and add a reference to the class library project below)
- class library project - here I have the MtTokenId class (that was created when I designed a little bit of a grammar and used your grammar designer to create code)

How do I connect these two?

That is saying look in an assembly "Bus.Grammar.dll" for a type named "MtTokenID" in the "Bus.Grammar" namespace. Are you sure your assembly name is correct and also that the namespace is correct? You can use the Object Browser in Visual Studio to be sure. Also the type name might be case sensitive so be careful it is exactly what the Object Browser says.

If you do that right, then it should load.

Here I load the xml: This is my library project: [Modified at 03/12/2011 08:13 AM]

As You can see, the namespaces in MtTokenID and TokenIDTypeName match.

[Modified at 03/12/2011 08:17 AM]

Hi Krzysztof,

Yes that looks ok. Are you positive that it is in an assembly named "Bus.Grammar.dll"? Also check that there is a public parameterless constructor on the token ID class.

If that doesn't help, please make a new simple sample project showing the issue and email it to us so we can have a look. Please don't include any .exe files in the ZIP and rename the .zip file extension so it doesn't get spam blocked.

Quote:
an assembly named "Bus.Grammar.dll"

Hey! Now we're talking!

I thought it was just a namespace - like in SyntaxLanguageTypeName - not a separate dll, so I attached my class to my project everywhere I could and wondered what I did wrong...

Now when I look at it in help files, it does mention

Quote:
Make sure the attribute includes the full namespace, Type name, and assembly specification

, but that are 3 parameters (1 - namespace, 2 - Type name, 3 - assembly), so it's kind'a misleading (and hidden in text anyway. Look at msdn - they outline the parameters, return value and such, so that people knew how to use them right away).

[Modified at 03/14/2011 01:00 PM]