Performance of OutlinerParser and StructureMatcher for large file

SyntaxEditor for WPF Forum

Posted 4 months ago by Vadim Tsedrik
Version: 19.1.0683
Avatar

Hello, some of our customers are using large files for the specific syntax language.

We used some services based on your examples such as IndentProvider, OutlinerParser, QuickInfoProvider, StructureMatcher. Everything is working fine if file is not big enough. But if file size exceed 5000 lines of code those services become very slow, and UI freeze up to 10 seconds when editing.

https://share.getcloudapp.com/Apuk9BOA

I send test project with syntax language and test file to support team.

Thanks.

[Modified 4 months ago]

Comments (5)

Posted 4 months ago by Actipro Software Support - Cleveland, OH, USA
Avatar

Hello,

Thank you for the sample.  I noticed that you didn't set up an ambient parse request dispatcher, so that meant that all parsing was occurring in the main UI thread, which is bad. 

We want parsing (especially when we load large documents) to be offloaded to asynchronously occur in a worker thread.  Adding this code to your App class solves that and typing has no delay then:

protected override void OnStartup(StartupEventArgs e) {
    AmbientParseRequestDispatcherProvider.Dispatcher = new ThreadedParseRequestDispatcher();

    base.OnStartup(e);
}


Actipro Software Support

Posted 4 months ago by Vadim Tsedrik
Avatar

Hello, thank you for quick response i added

ThreadedParseRequestDispatcher

as you described but it seems that it doesn't help a lot with our solution.

When i press "Enter" at the end of file it takes too much time to calculate indent amount in UI thread.

StructureMatcher.Match takes: 1ms
OutliningParser.Parse takes: 3088ms
GetIndentAmount takes: 3082ms
UI Freeze = 2909.3279 ms

https://share.getcloudapp.com/qGuNrgjm

Posted 4 months ago by Actipro Software Support - Cleveland, OH, USA
Avatar

Hello,

It's tough to say what's happening without debugging it.  I suspect things could be optimized or done in other ways though since GetIndentAmount shouldn't take that long.  It needs to be a fast call, looking at nearby tokens or possibly an AST that was already asynchronously parsed.  Our premium language add-ons all work all right with large files and have extensive feature sets, so we know performance can be ok with large files if syntax language and service code is configured in an optimized way.

If you'd like us to take a look so that we can offer suggestions, please create a new simple sample project with your syntax language code and send that to our support address, mentioning this thread.  In your email, tell us how to reproduce the issue.  Be sure to remove the bin/obj folders from the ZIP so it doesn't get spam blocked.  Thanks!


Actipro Software Support

Posted 4 months ago by Vadim Tsedrik
Avatar

Hello, i sent you sample project again,

And i will appreciate any help from you.

Thank you.

Posted 4 months ago by Actipro Software Support - Cleveland, OH, USA
Avatar

Hello,

Thank you for the sample project.  It appears that the logic in the ACSPLIndentProvider class is what is causing the slowdown because in the scenario where you press Enter at the end of your sample document, it's effectively iterating backward through all of the document tokens and that amount of intense scanning in a large document can take some time.  Thus it will lock up the editor while that scans since it's in the UI thread.

What we generally do to keep performance high is a combination of some token scanning and AST examination.  You could scan backward through tokens but only to a start of a statement, etc.  Then translate that snapshot offset back to the snapshot of the current parse data.  We use that translated snapshot and recurse into the AST to see at what level of indentation should occur.  That way, it's making effective use of the AST in the parse data that currently is available and is much faster than scanning backwards through thousands of tokens.


Actipro Software Support

The latest build of this product (v2020.1.1) was released 15 days ago, which was after the last post in this thread.

Add Comment

Please log in to a validated account to post comments.