AstNode StartOffset Incorrect

SyntaxEditor for WPF Forum

The latest build of this product (v26.1.0) was released 6 days ago, which was before this thread was created.
Posted 5 days ago by Rick - Developer, Visual Software Systems LLC
Version: 26.1.0
Platform: .NET 4.8
Environment: Windows 11 (64-bit)
Avatar

I've run into a problem with the StartOffset and EndOffset values for nodes in  a nested structure that I cannot seem to figure out.  I'm writing the Intellisense logic for my SQL-like language that allows a sub-query in the WHERE clause.  When I'm traversing the AST to set Context, the process fails because the StartOffset values are incorrect.  Here's an example:

Given this query:

SELECT Name FROM Table WHERE TypeId IN (SELECT id FROM FieldTypes)

Produces this AST: 

Document[
	Query[
		Select[
			AstField[
				AstRelationship
			]
		] 0-11
		AstFrom[
			AstObject
		] 12-33
		Where[
			SubSelectExpr[
				AstField[
					AstRelationship
				] 40-52
				Operator[
					"in"
				] 40-83	s/b 53-55
				SubQuery[
					Select[
						AstField[
							AstRelationship
						]
					] 57-66
					AstFrom[
						AstObject 72-82
					] 56-83	s/b 67-82
				] 40-83	s/b 57-83
			] 40-83
		] 34-83
		""
	]
]

As you can see, the offset values of the AstNodes in the SubSelectExpr node are incorrect, except for the first node.  I have a screenshot from the VS debugger showing the incorrect values.  I can send that to you if you want.  

The behavior was the same in 25.1.3.   I also want to note that 26.1 doesn't provide the TextRange values of AstNodes like prior version did with

parseData.Ast.ToTreeString

You can see this behavior in the Sample App.  Would be nice to get that back as it's very useful.

Regards, Rick Brown

Comments (3)

Posted 2 days ago by Actipro Software Support - Cleveland, OH, USA
Avatar

Hi Rick,

Thanks for the example. The LL Parser assigns terminal AST nodes the exact range of their tokens. New AST nodes created by a production's tree constructor normally receive the full range matched by that production, while a node returned through AstFrom retains the range it received when it was originally constructed.

That distinction may explain the pattern in your AST. If the Operator and SubQuery nodes are being created directly within the SubSelectExpr tree constructor, they will each inherit the full SubSelectExpr range. An AST node returned through AstFrom can retain its narrower original range. Constructing Operator and SubQuery in their own non-terminal productions, and then bringing those completed nodes into SubSelectExpr with AstFrom, should give them tighter ranges.

We agree that requiring a separate production solely to obtain a more precise range is not ideal. We have recorded this behavior for review as we modernize the LL Parser, with the goal of providing clearer control over where a constructed node's range comes from.

Regarding ToTreeString, we could not find a change between versions 25.1.3 and 26.1.0 that removed offsets. ToString includes the node's offset range unconditionally. It is possible that you are remembering ToString behavior as it's meant a little more for debugging.


Actipro Software Support

Posted 2 days ago by Rick - Developer, Visual Software Systems LLC
Avatar

Thanks.  I will make the changes you suggest to the ast construction and let you know the results. Regarding the ToTreeString observation, in 25.1.3 I see this in Sample 4d:

CompilationUnit[
FunctionDeclaration[
BlockStatement[
ReturnStatement[
BinaryOperatorExpression[
SimpleName (477-478)
SimpleName (481-482)
] (477-482)
] (470-483)
] (467-486)
] (448-486)
FunctionDeclaration[
BlockStatement[
ReturnStatement[
ParenthesizedExpression[
BinaryOperatorExpression[
SimpleName (521-522)
LiteralExpression (525-526)
] (521-526)
] (520-527)
] (513-528)
] (510-530)
] (488-530)
FunctionDeclaration[
BlockStatement[
VariableDeclarationStatement (612-623)
AssignmentStatement[
FunctionAccessExpression[
SimpleName (644-645)
] (634-646)
] (625-647)
ReturnStatement[
BinaryOperatorExpression[
SimpleName (656-662)
SimpleName (665-666)
] (656-666)
] (649-667)
] (568-669)
] (532-669)
] (448-669)

And this in 26.1 Sample 4d:

CompilationUnit[
FunctionDeclaration[
BlockStatement[
ReturnStatement[
BinaryOperatorExpression[
SimpleName
SimpleName
]
]
]
]
FunctionDeclaration[
BlockStatement[
ReturnStatement[
ParenthesizedExpression[
BinaryOperatorExpression[
SimpleName
LiteralExpression
]
]
]
]
]
FunctionDeclaration[
BlockStatement[
VariableDeclarationStatement
AssignmentStatement[
FunctionAccessExpression[
SimpleName
]
]
ReturnStatement[
BinaryOperatorExpression[
SimpleName
SimpleName
]
]
]
]
]

The code behind is the same in both versions of the sample project. Also, I am not able to view the source code for Sample 4d in the sample browser because it throws a licensing exception (I'm not licensed for the language add-ons).  In prior versions, I was still able to view sample code.

Thanks,

Rick

Answer - Posted 2 days ago by Actipro Software Support - Cleveland, OH, USA
Avatar

Hi Rick,

Thanks for the followup. It turns out that our AstNodeBase implementation did include offsets based on a certain pre-processor directive being present, which was only ever intended to be on for us internally. However it mistakenly made its way into production code in .NET Framework assemblies only, and was fixed as part of v26.1. That's why you saw the difference.

We understand that the offsets are helpful for you though and will restore them for the next build.


Actipro Software Support

Add Comment

Please log in to a validated account to post comments.