GoToPreviousMatchingToken for Dynamic Languages

SyntaxEditor for Windows Forms Forum

Posted 12 years ago by Chris Morris
Version: 4.0.0245
Avatar
I don't seem to be able to get the GoToPreviousMatchingToken in the TextStream class to work for a dynamic language.

I have declared the following syntax in the language xml file (which seems to work as outlining is disaplyed in the editor around the brackets)
<ExplicitPatternGroup Key="OpenParenthesisPatternGroup" TokenKey="OpenParenthesisToken" PatternValue="(" EndBracket="CloseParenthesisPatternGroup" />
<ExplicitPatternGroup Key="CloseParenthesisPatternGroup" TokenKey="CloseParenthesisToken" PatternValue=")" StartBracket="OpenParenthesisPatternGroup" />


However when running my program and entering some text like Invalidate(a,b) in the Editor the following source code always returns false for the GoToPreviousMatchingToken:

TextStream stream = syntaxEditor.Document.GetTextStream(syntaxEditor.Caret.Offset);
IToken token = stream.ReadTokenReverse(); // token selected is the )
bool foundMatchingToken = stream.GoToPreviousMatchingToken(token);


The matching open bracket is not found.

Any suggestions would be appreciated?

Chris.

Comments (8)

Posted 12 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar
Hi Chris,

You are correct, it will not work for dynamic languages. That method uses the MatchingTokenID property of the token you pass to determine which token ID to look for. This is designed for advanced language implementations that create custom tokens. In those cases, you would override that method and do a switch on the current token's ID to determine what ID value to return as the matching token ID.


Actipro Software Support

Posted 12 years ago by Chris Morris
Avatar
Thanks for the quick response. I suspected as much.

Sometimes the Actipro Documentation is unclear on which methods require advance language implementation.
Posted 12 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar
I'll add a note about that in the GoToPreviousMatchingToken remarks. Also, there is an overload that lets you specify the end token ID. So if you assign token IDs in your dynamic language XML definition, then you can reference the appropriate one when using that second overload.


Actipro Software Support

Posted 12 years ago by Chris Morris
Avatar
Thanks! Yes by specifying a TokenId in the dynamic language xml file and using the overloaded GoToPreviousMatchingToken with a Token StartID works!

Here's the working code example if anyone else is interested:

XML File:
<ExplicitPatternGroup Key="OpenParenthesisPatternGroup" TokenKey="OpenParenthesisToken" PatternValue="(" EndBracket="CloseParenthesisPatternGroup" TokenID="123"/>
<ExplicitPatternGroup Key="CloseParenthesisPatternGroup" TokenKey="CloseParenthesisToken" PatternValue=")" StartBracket="OpenParenthesisPatternGroup" TokenID="456"/>

Source Code:

TextStream stream = syntaxEditor.Document.GetTextStream(syntaxEditor.Caret.Offset);
IToken token = stream.ReadTokenReverse(); // ) token
bool found = stream.GoToPreviousMatchingToken(token,123);
token = stream.Token; // ( token !!
Posted 12 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar
Then just to make your code more readable, you can use an enum or class with constants on it. And the root SyntaxLanguage tag has an attribute where you can specify that class.


Actipro Software Support

Posted 12 years ago by Matthew Smith - Developer, One Plus One Solutions Pty Limited
Avatar
Quote:
Then just to make your code more readable, you can use an enum or class with constants on it. And the root SyntaxLanguage tag has an attribute where you can specify that class.


Could I clarify what you suggesting here.

Add this to the dynamic language definition:

<SyntaxLanguage Key="VB.NET" LanguageDefinitionVersion="4.0" Secure="True" 
    SyntaxLanguageTypeName="TestApplication.VBDotNetDynamicSyntaxLanguage, TestApplication"
    TokenIDTypeName="TestApplication.TokenID, TestApplication"
    xmlns="http://ActiproSoftware/SyntaxEditor/4.0/LanguageDefinition">
Create the TokenID class (along the lines of this):

Namespace TestApplication
    Public Class TokenID
        Public Const [InvalidToken] As Integer = 0
        Public Const [DocumentEndToken] As Integer = 1
        Public Const [WhitespaceToken] As Integer = 2
        Public Const [LineTerminatorToken] As Integer = 3
        Public Const [ReservedWordToken] As Integer = 5
        Public Const [VariableWordToken] As Integer = 7
        Public Const [MaxTokenIDToken] As Integer = 8
    End Class
End Namespace
The above token name matching the TokenKey fields as follows (example):

<!-- Whitespace -->
<RegexPatternGroup TokenKey="WhitespaceToken" PatternValue="{WhitespaceMacro}+" IsWhitespace="True" />
<!-- Line Terminators -->
<RegexPatternGroup TokenKey="LineTerminatorToken" PatternValue="{LineTerminatorMacro}" IsWhitespace="True" />
The main reason I wish to switch is to ensure my parenthesis brackets match (as above previous message) for Intellipromt, but also so I can use the TokenID field rather than the TokenKey string (? would this not be quicker??)

[Modified at 03/27/2007 01:28 AM]

Regards, Matt

Posted 12 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar
Yes I believe that looks correct. Actually I don't think you need the "Token" on the end of each const name either. It will find the const either way and is less typing to write it without that.

Yes numeric comparisons are always going to run faster than string comparisons.


Actipro Software Support

Posted 12 years ago by Matthew Smith - Developer, One Plus One Solutions Pty Limited
Avatar
Yeh, did notice the non-requirement for 'Token' on the end!

Thanks for the confirmation on the speed of using IDs instead of Keys - thought I'd read that previously.

Also I've started to get into the more advanced stuff recently, this control is great - was able to add intellisence in the space of a day or so! Awesome stuff!

Regards, Matt

The latest build of this product (v2018.1 build 0341) was released 3 months ago, which was after the last post in this thread.

Add Comment

Please log in to a validated account to post comments.