Keeping Part of a Failed Match Without Consuming Tokens

SyntaxEditor for WPF Forum

Posted 8 months ago by Will Gauthier
Version: 21.1.3
Avatar

My Lua grammar currently cannot handle function definitions whose parameter list ends in the var arg operator, like this one:

function foo(x, y, ...) print("hello") end

The relevant terminals are parameterList and identifierList:

parameterList.production = identifierList + (@comma + @varArgOperator).Optional()
identifierList.production = @identifier + (@comma + @identifier).ZeroOrMore()

When parsing inside the identifierList production hits the var arg operator, it terminates unsuccessfully because it's not an identifier.

I tried using this CanMatchCallback for identifierList:

private bool CanMatchIdentifierList(IParserState state)
{
  var tokenReader = state.TokenReader;
  tokenReader.Push();
  try
  {
    if (tokenReader.LookAheadToken.Id != LuaTokenId.Identifier)
    {
     return false;
    }
    tokenReader.Advance();
    while (tokenReader.LookAheadToken.Id == LuaTokenId.Comma)
    {
      tokenReader.Advance();
      if (tokenReader.LookAheadToken.Id != LuaTokenId.Identifier)
      {
        return false;
      }
      else
      {
        tokenReader.Advance();
      }
    }
  }
  finally
  {
  tokenReader.Pop();
  }
  return true;
}

However, that results in no identifierList match at all, not a partial one up to the comma before the var arg operator.

I also tried using an OnError callback that returns the Ignore result. When I advance the token reader inside of it, parsing is successful, but the var arg operator is passed over entirely. If I only return the Ignore result, I still get no identifierList match.

How do I correctly handle this scenario so that the parameterList match goes up until the last comma before the var arg operator when it's present?

Comments (2)

Answer - Posted 8 months ago by Actipro Software Support - Cleveland, OH, USA
Avatar

Hi Will,

The way you want to handle this is split your identifierList production up into two.  Then add a CanMatchCallback on the second one like:

identifierList.Production = @identifier + identifierListMoreIdentifiers.ZeroOrMore();

identifierListMoreIdentifiers.CanMatchCallback = (state => state.TokenReader.AreNext(LuaTokenId.Comma, LuaTokenId.Identifier));

identifierListMoreIdentifiers.Production = @comma + @identifier;

I believe that will handle the scenario and allow it to work.


Actipro Software Support

Posted 8 months ago by Will Gauthier
Avatar

Ingenious, that worked perfectly! Thank you.

The latest build of this product (v22.1.3) was released 20 days ago, which was after the last post in this thread.

Add Comment

Please log in to a validated account to post comments.