Bracket MatchingBracketToken

SyntaxEditor for Windows Forms Forum

Posted 15 years ago by tobias weltner
Avatar
I am experimenting with MatchingBracketToken but can't really get it to work.
What I need to do is while parsing backwards, whenever I encounter a closing bracket I want to move on to the corresponding opening bracket, thus ignoring the arguments inbetween.
It seems as if BracketHighlighting assumes its positioning information from caret position instead of some provided argument. If that's right, would I then really need to reposition the caret? Seems odd...
Of course I could manually keep a stack of opening and closing brackets but since there seems to be some form of bracket synch included, I'd love to be able to use it.

Comments (15)

Posted 15 years ago by tobias weltner
Avatar
I just double-checked, and it does indeed seem to be related to the current caret position.

To illustrate my problem, it is very hard to determine the starting bracket manually. You can't just move to an openingbracket-token because you might miss other closing brackets inbetween, so although this can be done manually, it really is a pain. Would love to see a way to do it with built-in methods... ;-)
Posted 15 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar
The matching bracket code should find the matching bracket at the same level as the one the caret is next to and will skip over child brackets within the bracket scope. Isn't that what you want?

If that's not what you want, try using a TextStream or TokenStream instead. Those are very powerful mechanisms for moving around and parsing the Document. You can look at each's methods to determine which is right to use for your situation. They can be retrieved via Document.GetTextStream() and Document.GetTokenStream().


Actipro Software Support

Posted 15 years ago by tobias weltner
Avatar
Your code works perfectly and does exactly what I want IF and only IF I set the caret to the position I find a closing bracket. Since this may occur multiple times, I would need to set caret position all the time.

I need to parse backwards regardless of current caret position, though. Let's say someone triggers a member list, then I need to parse backwards to find the object the user wants information for. In a .NET world this is easy due to reflection, but in the unmanaged world, I need to construct the complete latebound call. For this, I need to exclude the arguments which in themselves can be quite complex hierarchies.

Can you get what I mean? I'd love to be able to use your functionality because it works great. However in a perfect world, MatchingBracketToken should accept any token and deliver the corresponding token. ;-)
Posted 15 years ago by Boyd - Automated Software Testing Consultant / Developer, Patterson Consulting, LLC
Avatar
I actually went the route of creating my own parsing routine using TokenStream to locate a matching bracket that was independent of the definitions in Syntax Editor. The routine was fairly simple to create, and it executes very quickly.

My situation involved a VBScript language definition where I wanted to be able to match "Public Function" with "End Function". I use BracketHighlighting so I didn't want to define these Tokens in the language definition (since it would cause them to highlight).

If you plan on doing extensive parsing of your tokens, be sure to setup and use TokenId's instead of string comparisons on the TokenKey.
Posted 15 years ago by tobias weltner
Avatar
Interesting! Would you mind sharing your routine? If that's not possible, how do you make use of ids instead of keys? Does "Find" accept id's as well?
Posted 15 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar
You either want to iterate back using a TokenStream or do the trick that Boyd came up with that makes things a little easier.

When examining tokens, there is a Token.Key property that helps identify what it is. That is string based. In SyntaxEditor 3.0, we introduced a Token.ID property that is a number. Numeric comparisons are faster than string ones so that's what we recommend using for parsing. Our samples are only using the string Keys right now.

For more info on IDs, look in the docs at the "Parsing Guide / Lexical State and Token IDs" topic.


Actipro Software Support

Posted 15 years ago by Boyd - Automated Software Testing Consultant / Developer, Patterson Consulting, LLC
Avatar
Happy to share. There's probably a more efficient way to do this, but I tend to stick with whatever I get working :)
/// <summary>
/// Returns the TokenKey that corresponds to <paramref name="key"/> and
/// sets the value of <paramref name="isStartToken"/> to indicate if
/// <paramref name="key"/> is the first key in the pair.
/// </summary>
public string GetMatchingTokenKey(string key, ref bool isStartToken)
{
    // Simply 'switch' between the various keys and return it's opposite
    isStart = false;
    switch (key)
    {
        case "OpenParenthesisToken":
            isStart = true;
            return "CloseParenthesisToken";
        case "CloseParenthesisToken":
            return "OpenParenthesisToken";
        case "StringStartToken":
            isStart = true;
            return "StringEndToken";
        case "StringEndToken":
            return "StringStartToken";
    }
    return null;
}

/// <summary>
/// Returns the Token that corresponds to <paramref name="initialToken"/>.
/// </summary>
public Token FindMatchingToken(ActiproSoftware.SyntaxEditor.Document document, Token initialToken)
{
    bool readForward = true; // direction of read
    string openKey = null; // TokenKey of token that starts a pair
    string closeKey = null; // TokenKey of token that ends a pair 
    string keyToFind = null; // TokenKey of token we're looking for (either openKey or closeKey)
    int groupLevel = 0; // counter to determine nested level
    
    // Determine search criteria
    keyToFind = GetMatchingTokenId(initialToken.Key, ref readForward);
    if (keyToFind == null)
        return null; // Token does not support matching pairs
    if (readForward)
    {
        openKey = initialToken.Key;
        closeKey = keyToFind;
    }
    else
    {
        openKey = keyToFind;
        closeKey = initialToken.Key;
    }

    // Open a Token Stream to look for the desired token
    TokenStream stream = document.GetTokenStream(document.Tokens.IndexOf(initialToken.StartOffset));
    Token token = initialToken;

    while (token != null)
    {
        // Adjust group level if current token key is either the Open or Close key
        if (token.Key == openKey)
            groupLevel++;
        else if (token.Key == closeKey)
            groupLevel--;

        // Check if we found the Token we want
        if (token.Key == keyToFind && groupLevel == 0)
            return token;

        // Check for end of stream
        if ((stream.Position == 0 && readForward == false) || (strea.IsPastEnd && readForward))
            break; // end of stream reached

        // Advance to next token
        if (readForward)
        {
            token = stream.Read();
            if (token.StartOffset == initialToken.StartOffset)
                token = stream.Read(); // first forward read on a new stream returns the same token we started with
        }
        else
        {
            token = stream.ReadReverse();
        }
    }
    return null; // no match found
}
That basically covers it. Lots of room for improvement, but it works. I adjusted the routine to use Token.Key instead of Token.ID (it's easy enough to switch to Token.ID once you get that setup).

[Modified at 05/25/2005 11:52 AM]

[Modified at 05/25/2005 12:47 PM]
Posted 15 years ago by tobias weltner
Avatar
Thanks a lot!
I'm going to dig my way into it.
Just out of curiosity: is there a particular reason why first Read and first ReadRevers may or may not move to the next token?
Posted 15 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar
Read and ReadReverse always move the stream's position. The Peek variations do not.


Actipro Software Support

Posted 15 years ago by Boyd - Automated Software Testing Consultant / Developer, Patterson Consulting, LLC
Avatar
I ran into certain situations where the first read on the stream would return the same token. It's been a while since I saw this, but I think it had to do with reading in reverse. I added the check just to make sure.


[Modified at 05/25/2005 12:43 PM]
Posted 15 years ago by Boyd - Automated Software Testing Consultant / Developer, Patterson Consulting, LLC
Avatar
OK... I just isolated why I did that. Place the following code in the Editor.SelectionChanged event handler of the sample project:
// Get Current Token
Token currentToken = editor.Document.Tokens.GetTokenAtOffset(editor.SelectedView.Selection.FirstOffset);
this.WriteLine("Index of Current Token = '" + editor.Document.Tokens.IndexOf(currentToken.StartOffset) + "'");
// Create two streams that start with this token
TokenStream forwardStream = editor.Document.GetTokenStream(editor.Document.Tokens.IndexOf(currentToken.StartOffset));
TokenStream reverseStream = editor.Document.GetTokenStream(editor.Document.Tokens.IndexOf(currentToken.StartOffset));
// Return the index of the token returned from a read operation in both directions on the stream
try
{
    this.WriteLine("Index of Stream.Read Token = '" + editor.Document.Tokens.IndexOf(forwardStream.Read().StartOffset) + "'");
}
catch
{
    this.WriteLine("Unable to perform Stream.Read at this location");
}
try
{
    this.WriteLine("Index of Stream.ReadReverse Token = '" + editor.Document.Tokens.IndexOf(reverseStream.ReadReverse().StartOffset) + "'");
}
catch
{
    this.WriteLine("Unable to perform Stream.ReadReverse at this location");
}
This code, when executed, will result in output like the following:
Index of Current Token = '57'
Index of Stream.Read Token = '57'
Index of Stream.ReadReverse Token = '56'
In this situation, you can see that the 'Read' on the stream returns the same token that I used to open my stream. I have already processed the current token, so I need to make sure it moves to the next token in the stream. Since 'Read' and 'ReadReverse' were not consistent on what was returned, I just put in that little check to make sure I had a new token.
Posted 15 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar
It's pretty simple... The Position is whatever token index you are currently at. So if you start the TokenStream at index 0, the Position will be 0, and the first time you call Read, you will get the first Token in the collection and after the call, the position will be at 1. If while at position 1, you call ReadReverse, it jumps back to the first Token and returns that.
public Token Read() {
    Token token = tokens[position];
    this.Seek(1);
    return token;
}

public Token ReadReverse() {
    this.Seek(-1);
    return tokens[position];
}


Actipro Software Support

Posted 15 years ago by Boyd - Automated Software Testing Consultant / Developer, Patterson Consulting, LLC
Avatar
Thanks for the clarification! My post was simply to clarify why I had that check in my code. The stream reading functionality works exactly as expected. I updated my posted code to be more specific about the reason for the extra read.

[Modified at 05/25/2005 12:47 PM]
Posted 15 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar
I was writing my post just as you were doing yours but you beat me by a split second! :)


Actipro Software Support

Posted 14 years ago by Peter Chapman
Avatar
This is a great answer, very useful.

Many thanks.
The latest build of this product (v2020.1 build 0400) was released 24 days ago, which was after the last post in this thread.

Add Comment

Please log in to a validated account to post comments.