Catching the Empty String as a Token

SyntaxEditor for WPF Forum

Posted 2 years ago by Will Gauthier
Version: 21.1.3

I'm having trouble getting my Lua grammar to recognize assignment of the empty string to a variable as syntactically valid, e.g.:

resultString = ""

It successfully enters and exits the DoubleQuoteString lexical scope, but there's no token to match between the start pattern group explicit pattern " and end pattern group explicit pattern ".

I've tried using regex pattern groups in the DoubleQuoteString state to catch the empty string, but neither ^$ nor ^.{0}$ nor [^"]* work.

I also tried adding an explicit empty string pattern match to the default lexical scope as "" with the usual look-ahead of {NonWord}|\z, but instead of matching the two characters and consuming them as an empty string token, it just enters the DoubleQuoteString state.

How can I match the empty string as a token so that I can recognize its inclusion as valid inside an assignment statement?

Comments (1)

Posted 2 years ago by Actipro Software Support - Cleveland, OH, USA

Hi Will,

A lexer must consume at least one character to produce a valid token, so that isn't possible.  Thus you will end up with a token stream like [StringStartDelimiter][StringEndDelimiter].

That is fine though for a parser.  You could have a parser non-terminal production look for StringStartDelimiter, followed by zero to many StringText, and then StringEndDelimiter terminals.

Actipro Software Support

The latest build of this product (v24.1.2) was released 2 months ago, which was after the last post in this thread.

Add Comment

Please log in to a validated account to post comments.