Stopping 'Unexpected token' in error messages

Posted 16 years ago by Timothy Coltman

I am attempting to create a "proper" parser for a VB-esque language, using CR/LFs to delineate statements. Each statement consists of a keyword, which may or may not be followed by an argument.

For example:

ASSIGN %FRED% = 12
RATIONALIZE

I am trying to get the syntax editor to highlight malformed statements, such as:

RATIONALIZE 12

Here, it should underline the 12 and say "End of line expected". However, when I generate the VB code from the grammar designer, and type "RATIONALIZE 12" into my syntax editor, I get this:

"Unexpected token.
End of line expected".

If I put a breakpoint on my ReportSyntaxError method, and look at the stack trace, the "Unexpected token" message seems to be coming from the "Match" method of the class that derives from "RecursiveDescentSemanticParser". I can't look inside this, so am at a loss as to how to proceed.

The relevant XML looks like this:


<!-- FPL program. -->
    <NonTerminal Key="FplProgram">
        <Production>
            <![CDATA[
            
                <%
                
                    _braceLevel = 0
                    _fplProgram = New FplProgram()
                    _fplProgram.StartOffset = Me.LookAheadToken.StartOffset
                    
                    _errorReported = False
                    
                    While (Not Me.IsAtEnd)
                        If (IsNonTerminal("Statement")) Then
                            _errorReported = False
                %>
                            { "Statement <- ->" }
                            
                <%
                        Else
                            ' Advance to the next token.
                            If (Not _errorReported) Then
                                MyClass.ReportSyntaxError("Statement expected.")
                                _errorReported = True
                            End If
                            MyClass.AdvanceToNext(FplTokens.NewLine)
                            MyClass.AdvanceToNext()
                        End If
                            
                    End While
                    
                    _fplProgram.EndOffset = Me.LookAheadToken.EndOffset
                %>
            ]]>
        </Production>
    </NonTerminal>

<!-- Statement. -->
    <NonTerminal Key="Statement">
        <Production>
            <![CDATA[
            
                "StatementNoOp" | "StatementRationalize"
            
            ]]>
        </Production>
    </NonTerminal>

<!-- Statement: no op. -->
    <NonTerminal Key="StatementNoOp">
        <Production>
            <![CDATA[
            
                'NewLine'
            
            ]]>
        </Production>
    </NonTerminal>

<!-- Statement: RATIONALIZE. -->
    <NonTerminal Key="StatementRationalize">
        <Production>
            <![CDATA[
                
                'KeywordRationalize'
                'NewLine <- MyClass.ReportSyntaxError("End of line expected.")
                _errorReported=true
                Return False
                ->'
                
            ]]>
        </Production>
    </NonTerminal>

Please help, I'm completely stuck!

Comments (1)

Posted 16 years ago by Actipro Software Support - Cleveland, OH, USA

Timothy,

Right, since it is looking for a NewLine but that isn't found, that is the error message you get. Now you can tidy it up a bit if your SyntaxLanguage overrides the GetTokenString method. Check out this method in our SimpleSyntaxLanguage sample:

public override string GetTokenString(int tokenID) { ... }

In there we have a case statement with stuff like:

case SimpleTokenID.LineTerminator:
    return "Line terminator";

If you do that for your tokens, your error message will instead be "Line terminator expected."

Actipro Software Support

The latest build of this product (v25.1.0) was released 20 days ago, which was after the last post in this thread.

Comments (1)

Add Comment