Posted 14 years ago by tobias weltner
Avatar
I'd like to suggest another thing regarding language definitions.
Currently, tokens can be identified in two different ways: key and id.
It would be great if tokens could be also assigned custom attributes, maybe a limited number of "category1" to category10" or something like this.

This would make it possible to logically group tokens into specific semantic groups that would ease semantic parsing considerably.

Currently, I work around this by using id as bit mask which works but isn't really beautiful because it makes the identification process slow.

It would already be a great help if there was another property just like id that could be used as bit mask for additional grouping info.

Comments (5)

Posted 14 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar
There is a SemanticParseData property on Tokens. You can use that to store any custom data that you'd like.


Actipro Software Support

Posted 14 years ago by tobias weltner
Avatar
This is not what I want.
I don't want to store information in individual tokens but rather add general grouping info in the language definition.

For example, a commentEnd and a LineFeed and a DocumentEnd all share the same meaning: they terminate a line, so in this simple example, I'd like to group them all with one category id.

There may be other token groups that for example allow to call help on them, so I'd like to add those to a group as well. Did I explain the idea sufficiently?

There may be more than one such logical grouping so I'd need a couple of categories or at least a bitmask.

This is not however about individual tokens or semanticparsedata. It's something about the quality/richness of the language definition.
Posted 14 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar
Oh so you mean you want something on a lexical pattern group level, right?


Actipro Software Support

Posted 14 years ago by tobias weltner
Avatar
Yes exactly, it would be something to add to the XML definition!
Posted 14 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar
For lexical pattern groups, there is a Tag property that you can use to store whatever data you want. You'd have to assign it at run-time but that will work for you.


Actipro Software Support

The latest build of this product (v2018.1 build 0341) was released 6 months ago, which was after the last post in this thread.

Add Comment

Please log in to a validated account to post comments.