Catching the Empty String as a Token

SyntaxEditor for WPF Forum

Posted 8 months ago by Will Gauthier
Version: 21.1.3

I'm having trouble getting my Lua grammar to recognize assignment of the empty string to a variable as syntactically valid, e.g.:

resultString = ""

It successfully enters and exits the DoubleQuoteString lexical scope, but there's no token to match between the start pattern group explicit pattern " and end pattern group explicit pattern ".

I've tried using regex pattern groups in the DoubleQuoteString state to catch the empty string, but neither ^$ nor ^.{0}$ nor [^"]* work.

I also tried adding an explicit empty string pattern match to the default lexical scope as "" with the usual look-ahead of {NonWord}|\z, but instead of matching the two characters and consuming them as an empty string token, it just enters the DoubleQuoteString state.

How can I match the empty string as a token so that I can recognize its inclusion as valid inside an assignment statement?

Comments (1)

Posted 8 months ago by Actipro Software Support - Cleveland, OH, USA

Hi Will,

A lexer must consume at least one character to produce a valid token, so that isn't possible.  Thus you will end up with a token stream like [StringStartDelimiter][StringEndDelimiter].

That is fine though for a parser.  You could have a parser non-terminal production look for StringStartDelimiter, followed by zero to many StringText, and then StringEndDelimiter terminals.

Actipro Software Support

The latest build of this product (v22.1.3) was released 23 days ago, which was after the last post in this thread.

Add Comment

Please log in to a validated account to post comments.