v4.0 : Documentation correction & startup questions - SyntaxEditor for Windows Forms Forum

Posted 18 years ago by Jared Phelps

Hello-
I've been evaluating v4.0, and so far I'm really liking it. This is definitely a must-purchase for me.

I am struggling a little bit with knowing how best to integrate my current code with the actipro object model. I manually wrote both a lexer and a semantic parser for my language, and obviously I'd like to change as little as possible. I already integrated my lexer with the control, but what would be your suggested approach to integrate my semantic parser? I could pattern it after the Simple Language add-on generated by the parser generator, write my own compilationunit & astnode classes, implement all the interfaces and use the visitor pattern, but that seems like a lot of work and I'm not sure what I would gain by doing so. Is there any functionality I get 'for free' (ie intelliprompt features) by using your object model instead of just plugging in calls to my existing code where applicable?

I am having trouble understanding the difference between Direct lexical state transitions and scope lexical state transitions. The example given in the documentation shows that the former would be used for ASP-style <% tags, while the latter would be used for a <script> tag. I think this means that for a direct transition, no matter what lexical state you are in in the 'current' language, the appearance of the transition token means we're going to another language. But for the scope transitions, you have to finish out the lexical state you're in before switching. So you could do this with direct but not scope:
<blah href="<%=geturl()%>">url</blah>

And you could do this with either:
<body>
<%
'code here
%>
</body>

Am I on the right track here?

As a side note, I noticed that in Documentation->Language Definition Guide->Semantic Parsing->Overview, there is this sentance: "Semantic parsing code is placed in the PerformSemanticParsing method of the Document class." This should be PerformSemanticParsing method of the SyntaxLanguageClass, shoudn't it?

In general, I'm very impressed with the documentation. It's main flaw is in the class library...while it sufficiently documents the 'what', it doesn't have any information on where a particular entity is used, why it is used, when you should use it, and how you should use it. Most of my information on individual classes has come from looking at the code in the Simple language add-on.

Thanks and keep up the good work!
Jared

[Modified at 09/07/2006 02:34 PM]

Comments (3)

Posted 18 years ago by Actipro Software Support - Cleveland, OH, USA

Hi Jared,

The AST object model is nice because it enables you to easily find context of an offset and use that for when you populate IntelliPrompt. But perhaps in your case, you are better off just keeping what you have if it gives you what you need. You can override SyntaxLanguage.PerformSemanticParse and just call whatever your semantic parser is. In fact you could even set up a call to the semantic parser service in that method and then in the callback do your semantic parsing code. That way it's done out of the main thread. The results should be passed back to Document.SemanticParseData.

Direct lexical state transitions occur immediately after a start scope pattern is recognized and transition back when the end scope pattern is recognized. Scope lexical state transitions are more like normal states however once their end scope is found (thus terminating the state), they transition into the other language. The return transition is defined by the pattern in the StateTransition tag block.

So direct is best with things like <% %> and scope is best with things like <style> </style>. Hope that clarifies it.

Thanks for the doc error notes. It's fixed for the upcoming release.

Actipro Software Support

Posted 18 years ago by Jared Phelps

Thanks for the response. I'll use my code in the short term and hopefully move to the AST object model as I have time. My current object model also allows me to find context from an offset, so I should be OK.

I've noticed a couple of things that seem strange with my state transitions, using the simple language and the example in DynamicallyCreateForm. First, I've noticed that if I use direct lexical state transitions, I can't just transition states at arbitrary times and still have my language's lexer being called:

<xmltag name='<% test(1); %>'>

I think this should transition into my language (Simple, in this case) and call its lexer and semantic parser, but it doesn't. Any ideas?

Similarly, I've noticed that when combining languages at run-time like DynamicallyCreate does, the semantic parser for the "second" language never gets called, although the lexer does. For example:

<xml>
<%
function test()
{
var x;
return 3;
}
%>
</xml>

I put breakpoints in the semantic parser part of simple that never get hit here. You can see it's never being called by checking out the intelliprompt options. Am I missing something? I assume we're able to do semantic parsing in a merged language...

Thanks!
Jared

Posted 18 years ago by Actipro Software Support - Cleveland, OH, USA

The lexer is "greedy" so once a pattern starts to be matched it will match characters a long as it can. If one of your <% lexemes shows up in the middle, it's not looking for those. So you need to insert breaks in the middle of the pattern so that it knows to look for something else. You can use our MainForm in the sample project to help debug. Since as you move the caret to different offsets, it shows you what token and lexical state you are over. It puts an * when you are on a token start character.

Transitioning of semantic parsers is still something we need to finalize before the 4.0 final. However I believe it somewhat works now. Note that only the root language's PerformSemanticParse will be called. However, if your languages inherit MergableSyntaxLanguage, then the mergable language coordinator code will call an overload of PerformSemanticParse defined on MergableSyntaxLanguage. This overload takes a MergableLexicalParserManager. As language transitions are found it should pop in and out of calls to the PerformSemanticParse(MergableLexicalParserManager) overload for each language. Then if you passed back IAstNode objects from each of those, it will combine the result into one large tree. Again this is still work in progress but I believe it was working the last time we tried. It will end up being a really slick feature when its done since each language will be able to handle its own semantic parsing and the results are automatically combined for you.

Actipro Software Support

The latest build of this product (v24.1.1) was released 5 days ago, which was after the last post in this thread.

Add Comment

Please log in to a validated account to post comments.