Parsing document comments

Posted 8 years ago by Yury Asheshov

Version: 17.1.0650

Hi,

I'm trying to implement a grammar for Franca IDL (https://github.com/franca/franca).

There is a special comment to document the interfaces and types <** ... **>:

// minimal interface definition
package testcases
<** @description : Test interface with some meta information.
@description : Somebody **>
interface MyInterface {
	version { major 1 minor 0 }
 }

As you can see Franca IDL has several pre-defined tags, for example @description. A tag starts with '@' and ends width ':'. Also spaces between tag name and colon are allowed. How to configre lexer to parse it? The best way is to parse it into a collection of tags and text for each tag. And of course I would like to have the tags with different style, for example bold. My problem is extra whitespaces before tag and between tag name and colon...

Thanks in advance...

[Modified 8 years ago]

Comments (4)

Posted 8 years ago by Actipro Software Support - Cleveland, OH, USA

Hi Yury,

There are a couple ways to do it.

Option 1) Handle this all in the lexer using a regex and make one token that spans the '@' through to the ':'. Then that just returns a tag token, which the parser consumes.

Or this next option, which is probably what I'd do:

Option 2) Have your lexer consume anything starting with '@' and followed by alpha chars to be a tag name token. Have one or more whitespace chars be a whitespace token. Have a ':' be its own colon token.

If you are using our LL(*) Parser Framework, there is a token reader that provides tokens to the parser. That can be customized to exclude things like comments and whitespace. If you exclude the whitespace from ever being passed to the parser, in the parser grammar, a tag non-terminal production would be a tag name token followed by a colon token.

Actipro Software Support

Posted 8 years ago by Yury Asheshov

Ok, I've already tried second option, but "Live Test" was OK only if the tag was at the begining of the line. I assume that it should work fine only with my own control with my customized token reader. Is it possible to implement live view with my DLL? Because debugger already can load customer DLL...

PS: Hope LeBron win this year one more title, the Finals last year were great! ;)

Posted 8 years ago by Actipro Software Support - Cleveland, OH, USA

Hi Yury,

The Live Test tab only helps debug the lexer. With it you can move the caret around and the status bar tells you what token is under the caret and if it's a start token. If you are seeing problems with the tokens being constructed, the Live Test is where you want to debug that sort of thing.

Once you are sure all the tokens are being lexed properly based on your definitions, if you are using our LL(*) Parser Framework, you can then load your parser grammar into the LL Parser Debugger and step through the parsing process. Assuming you have set up a custom token reader in your parser, it should be using that.

And thanks for the Finals wish, we'd love to see another title here this year! If they have a rematch against the Warriors, it will certainly be a tough battle.

Actipro Software Support

Answer - Posted 8 years ago by Yury Asheshov

Ok I got it and it provides very nice syntax highlighting and can be parsed pretty easy. Thank you for help…
PS: yes, KD and McGee are strong addition and furthermore CLE has no rim protector, but it is still a long way to go…

The latest build of this product (v24.1.4) was released 29 days ago, which was after the last post in this thread.

Comments (4)

Add Comment