Stopping 'Unexpected token' in error messages

Posted 14 years ago by Timothy Coltman

I am attempting to create a "proper" parser for a VB-esque language, using CR/LFs to delineate statements. Each statement consists of a keyword, which may or may not be followed by an argument.

For example:

ASSIGN %FRED% = 12
RATIONALIZE

I am trying to get the syntax editor to highlight malformed statements, such as:

RATIONALIZE 12

Here, it should underline the 12 and say "End of line expected". However, when I generate the VB code from the grammar designer, and type "RATIONALIZE 12" into my syntax editor, I get this:

"Unexpected token.
End of line expected".

If I put a breakpoint on my ReportSyntaxError method, and look at the stack trace, the "Unexpected token" message seems to be coming from the "Match" method of the class that derives from "RecursiveDescentSemanticParser". I can't look inside this, so am at a loss as to how to proceed.

The relevant XML looks like this:


<!-- FPL program. -->
    <NonTerminal Key="FplProgram">
        <Production>
            <![CDATA[
            
                <%
                
                    _braceLevel = 0
                    _fplProgram = New FplProgram()
                    _fplProgram.StartOffset = Me.LookAheadToken.StartOffset
                    
                    _errorReported = False
                    
                    While (Not Me.IsAtEnd)
                        If (IsNonTerminal("Statement")) Then
                            _errorReported = False
                %>
                            { "Statement <- ->" }
                            
                <%
                        Else
                            ' Advance to the next token.
                            If (Not _errorReported) Then
                                MyClass.ReportSyntaxError("Statement expected.")
                                _errorReported = True
                            End If
                            MyClass.AdvanceToNext(FplTokens.NewLine)
                            MyClass.AdvanceToNext()
                        End If
                            
                    End While
                    
                    _fplProgram.EndOffset = Me.LookAheadToken.EndOffset
                %>
            ]]>
        </Production>
    </NonTerminal>

<!-- Statement. -->
    <NonTerminal Key="Statement">
        <Production>
            <![CDATA[
            
                "StatementNoOp" | "StatementRationalize"
            
            ]]>
        </Production>
    </NonTerminal>

<!-- Statement: no op. -->
    <NonTerminal Key="StatementNoOp">
        <Production>
            <![CDATA[
            
                'NewLine'
            
            ]]>
        </Production>
    </NonTerminal>

<!-- Statement: RATIONALIZE. -->
    <NonTerminal Key="StatementRationalize">
        <Production>
            <![CDATA[
                
                'KeywordRationalize'
                'NewLine <- MyClass.ReportSyntaxError("End of line expected.")
                _errorReported=true
                Return False
                ->'
                
            ]]>
        </Production>
    </NonTerminal>

Please help, I'm completely stuck!

Comments (1)

Posted 14 years ago by Actipro Software Support - Cleveland, OH, USA

Timothy,

Right, since it is looking for a NewLine but that isn't found, that is the error message you get. Now you can tidy it up a bit if your SyntaxLanguage overrides the GetTokenString method. Check out this method in our SimpleSyntaxLanguage sample:

public override string GetTokenString(int tokenID) { ... }

In there we have a case statement with stuff like:

case SimpleTokenID.LineTerminator:
    return "Line terminator";

If you do that for your tokens, your error message will instead be "Line terminator expected."

Actipro Software Support

The latest build of this product (v24.1.0) was released 2 months ago, which was after the last post in this thread.

Comments (1)

Add Comment