Keeping Part of a Failed Match Without Consuming Tokens

SyntaxEditor for WPF Forum

Posted 2 years ago by Will Gauthier
Version: 21.1.3
Avatar

My Lua grammar currently cannot handle function definitions whose parameter list ends in the var arg operator, like this one:

function foo(x, y, ...) print("hello") end

The relevant terminals are parameterList and identifierList:

parameterList.production = identifierList + (@comma + @varArgOperator).Optional()
identifierList.production = @identifier + (@comma + @identifier).ZeroOrMore()

When parsing inside the identifierList production hits the var arg operator, it terminates unsuccessfully because it's not an identifier.

I tried using this CanMatchCallback for identifierList:

private bool CanMatchIdentifierList(IParserState state)
{
  var tokenReader = state.TokenReader;
  tokenReader.Push();
  try
  {
    if (tokenReader.LookAheadToken.Id != LuaTokenId.Identifier)
    {
     return false;
    }
    tokenReader.Advance();
    while (tokenReader.LookAheadToken.Id == LuaTokenId.Comma)
    {
      tokenReader.Advance();
      if (tokenReader.LookAheadToken.Id != LuaTokenId.Identifier)
      {
        return false;
      }
      else
      {
        tokenReader.Advance();
      }
    }
  }
  finally
  {
  tokenReader.Pop();
  }
  return true;
}

However, that results in no identifierList match at all, not a partial one up to the comma before the var arg operator.

I also tried using an OnError callback that returns the Ignore result. When I advance the token reader inside of it, parsing is successful, but the var arg operator is passed over entirely. If I only return the Ignore result, I still get no identifierList match.

How do I correctly handle this scenario so that the parameterList match goes up until the last comma before the var arg operator when it's present?

Comments (2)

Answer - Posted 2 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar

Hi Will,

The way you want to handle this is split your identifierList production up into two.  Then add a CanMatchCallback on the second one like:

identifierList.Production = @identifier + identifierListMoreIdentifiers.ZeroOrMore();

identifierListMoreIdentifiers.CanMatchCallback = (state => state.TokenReader.AreNext(LuaTokenId.Comma, LuaTokenId.Identifier));

identifierListMoreIdentifiers.Production = @comma + @identifier;

I believe that will handle the scenario and allow it to work.


Actipro Software Support

Posted 2 years ago by Will Gauthier
Avatar

Ingenious, that worked perfectly! Thank you.

The latest build of this product (v24.1.2) was released 2 days ago, which was after the last post in this thread.

Add Comment

Please log in to a validated account to post comments.