Single token cannot end scope for two state levels. - SyntaxEditor for Windows Forms Forum

Posted 18 years ago by Bob Rundle - Director, Dynamic Workflow, JOA Oil & Gas BV

I have a language that does not have explicit terminations for blocks. Blocks are introduced by keywords and there are two levels of blocks. Here is a simple example...

RUNSPEC

OIL

WATER

GAS

DISGAS

GRID

NEWTRAN

END

Needless to say this is highly simplified. The "RUNSPEC" keyword starts one top level section and the "GRID" keyword starts another.

The problem is that I don't know that the RUNSPEC section ends until I see the GRID keyword. This keyword in turn starts another section. The GRID keyword also indicates the end of the previous statement (which can have a lot of different formats...only the one line version is shown here).

So I have a language definition that includes this...(again, highly simplified)

<SyntaxLanguage Key="Eclipse" LanguageDefinitionVersion="3.0" Secure="False" xmlns="http://ActiproSoftware/SyntaxEditor/3.0/LanguageDefinition">
<States>
<State Key="DefaultState">
<ChildStates>
<ChildState Key="DeckSectionState"/>
</ChildStates>
</State>
<State Key="DeckSectionState">
<Scopes>
<Scope>
<RegexPatternGroup Type="StartScope" TokenKey="DeckSectionStartToken" PatternValue="^RUNSPEC|^GRID|^EDIT|^PROPS|^REGIONS|^SOLUTION|^SUMMARY|^SCHEDULE|^OPTIMIZE|^END" Style="ReservedWordStyle"/>
<RegexPatternGroup Type="EndScope" TokenKey="DeckSectionEndToken" PatternValue="{LineTerminatorMacro}" LookAhead="^RUNSPEC|^GRID|^EDIT|^PROPS|^REGIONS|^SOLUTION|^SUMMARY|^SCHEDULE|^OPTIMIZE|^END" IsWhitespace="True"/>
</Scope>
</Scopes>
<ChildStates>
<ChildState Key="StatementState"/>
</ChildStates>
</State>
<State Key="StatementState">
<Scopes>
<Scope>
<RegexPatternGroup Type="StartScope" TokenKey="StatementBlockStartToken" PatternValue="^[A-Z][A-Z0-9]*" Style="ReservedWordStyle"/>
<RegexPatternGroup Type="EndScope" TokenKey="StatementBlockEndToken" PatternValue="{LineTerminatorMacro}" LookAhead="^[A-Z][A-Z0-9]*" IsWhitespace="True"/>
</Scope>
</Scopes>
</State>
</States>
</SyntaxLanguage>

I find that this *almost works*. I find that I can fire "StatementBlockEndToken". However if the keyword that ended the statement is "GRID", then I want a "DeckSectionEndToken" also. This will not happen because the pattern that matched StatementBlockEndToken ate the line terminator and therefore the pattern in the parent state for StatementBlockEndToken will never match.

I have been fooling around with various formulations to make this work. I am convinced that this is no way to make this work in the SyntaxEditor.

Am I wrong? Any help is appreciated.

Regards,
Bob Rundle

Comments (2)

Posted 18 years ago by Actipro Software Support - Cleveland, OH, USA

Hi Bob,

Yes the problem as you see is that both states are looking for a LineTerminator.
However the lexical parser looks at the innermost parent state first and sees that
it finds a match and therefore quits.

I did some tweaking and think I got it working...


<SyntaxLanguage Key="Eclipse" LanguageDefinitionVersion="3.0" Secure="False" xmlns="http://ActiproSoftware/SyntaxEditor/3.0/LanguageDefinition">
    <Styles>
        <Style Key="ReservedWordStyle" ForeColor="Blue" BackColor="Default"/>
        <Style Key="StatementStyle" ForeColor="Maroon" BackColor="Default"/>
    </Styles>
    <States>
        <State Key="DefaultState">
            <ChildStates>
                <ChildState Key="DeckSectionState"/>
            </ChildStates>
        </State>
        <State Key="DeckSectionState">
            <Scopes>
                <Scope>
                    <RegexPatternGroup Type="StartScope" TokenKey="DeckSectionStartToken" PatternValue="^ (RUNSPEC|GRID|EDIT|PROPS|REGIONS|SOLUTION|SUMMARY|SCHEDULE|OPTIMIZE|END)" Style="ReservedWordStyle"/>
                    <RegexPatternGroup Type="EndScope" TokenKey="DeckSectionEndToken" PatternValue="{LineTerminatorMacro}" LookAhead="^ (RUNSPEC|GRID|EDIT|PROPS|REGIONS|SOLUTION|SUMMARY|SCHEDULE|OPTIMIZE|END)" IsWhitespace="True"/>
                </Scope>
            </Scopes>
            <ChildStates>
                <ChildState Key="StatementState"/>
            </ChildStates>
        </State>
        <State Key="StatementState">
            <Scopes>
                <Scope>
                    <RegexPatternGroup Type="StartScope" TokenKey="StatementBlockStartToken" PatternValue="^[A-Z][A-Z0-9]*" Style="StatementStyle"/>
                    <RegexPatternGroup Type="EndScope" TokenKey="StatementBlockEndToken" PatternValue="{LineTerminatorMacro}(?! ^ (RUNSPEC|GRID|EDIT|PROPS|REGIONS|SOLUTION|SUMMARY|SCHEDULE|OPTIMIZE|END))" LookAhead="^[A-Z][A-Z0-9]*" IsWhitespace="True"/>
                </Scope>
            </Scopes>
        </State>
    </States>
</SyntaxLanguage>

Some other good news is that we're days away from a beta of v4.0 in which we've
done a ton of work on languages to make them more abstracted. In v4.0 you can
define your own lexical parser in code if you wish instead of using the XML
definition format. This might be a situation where a programmatic lexical parser
would be better because that way you could do your own lookahead and lookbehind as
needed to determine what is what. Of course defining a programmatic lexical
parser is more complicated than using an XML definition but it does offer more
flexibility in certain regards. However if what I have listed above works for you
then by all means use that.

Actipro Software Support

Posted 18 years ago by Bob Rundle - Director, Dynamic Workflow, JOA Oil & Gas BV

Yes, it seems to work. I have implemented outlining and both the deck sections and the statements seem to outline properly

There is only one thing I don't like. The parser does not always return StatementBlock{Start,End}Token pairs, because when a deck section keyword is seen a DeckSectionStartToken is returned.

I suppose this is a limitation of the parser since it appears that every match must eat a token. What I was trying to do is have an null PatternValue and use the LookAhead and LookBehind to fire the rule. This does not seem to work however.

Anyway, I can work around this issue. Thanks for the help.

Bob Rundle

The latest build of this product (v24.1.0) was released 2 months ago, which was after the last post in this thread.

Add Comment

Please log in to a validated account to post comments.