Outlining performance

SyntaxEditor for WPF Forum

The latest build of this product (v22.1.4) was released 2 months ago, which was before this thread was created.
Posted 1 month ago by Tobias Lingemann - Software Devolpment Engineer, Vector Informatik GmbH
Version: 22.1.1
Platform: .NET 4.8
Environment: Windows 10 (64-bit)
Avatar

Hi,

the code files of our customers are sometimes really big, escpecially if they are generated. In some rare cases this leads to multiple thousands of outlining nodes in one file. With the old WinForms version we made some modifications to get a good performance. With the WPF release we noticed some performance issues for these files.

Since the updates are performed in the parser thread, this isn't immediately visible. But after some time the parser threads are busy with updating the outlining data and the main thread blocks until the parser thread is free again.

The key issue here is the use of RemoveAt():

95,81 %   OnParseDataChanged  •  292.723 ms  •  ActiproSoftware.Text.Implementation.CodeDocument.OnParseDataChanged(ParseDataPropertyChangedEventArgs)
  95,81 %   [Native code]  •  292.723 ms
    95,80 %   OnEvent  •  292.713 ms  •  ActiproSoftware.Text.Utility.WeakEventListener`2.OnEvent(Object, TEventArgs)
      95,80 %   r7P  •  292.713 ms  •  ActiproSoftware.Internal.ut+<>c.r7P(ut, Object, EventArgs)
        95,80 %   EmL  •  292.713 ms  •  ActiproSoftware.Internal.ut.EmL(Object, EventArgs)
          95,80 %   PmR  •  292.713 ms  •  ActiproSoftware.Internal.ut.PmR(Nullable, TextSnapshotRange, Boolean)
            95,77 %   UpdateAutomaticOutlining  •  292.601 ms  •  ActiproSoftware.Internal.ut.UpdateAutomaticOutlining(IOutliningSource, TextSnapshotRange)
              47,86 %   ixi  •  146.233 ms  •  ActiproSoftware.Internal.bY.ixi(IOutliningNodeDefinition, Int32)
                47,84 %   fgO  •  146.157 ms  •  ActiproSoftware.Internal.bY.fgO
                  46,50 %   RemoveAt  •  142.074 ms  •  System.Collections.Generic.List`1.RemoveAt(Int32)
                  0,47 %   HxN  •  1.438 ms  •  ActiproSoftware.Internal.w4.HxN
                  0,21 %   Add  •  643 ms  •  System.Collections.Generic.List`1.Add(T)
                  0,18 %   Vxs  •  550 ms  •  ActiproSoftware.Internal.w4.Vxs
                  0,12 %   8 functions hidden  •  370 ms total
                0,02 %   3 functions hidden  •  76 ms total
              47,62 %   EgT  •  145.493 ms  •  ActiproSoftware.Internal.bY.EgT(Int32)
                47,62 %   qxc  •  145.483 ms  •  ActiproSoftware.Internal.bY.qxc(w4, Int32, Int32, Int32)
                  46,45 %   RemoveAt  •  141.933 ms  •  System.Collections.Generic.List`1.RemoveAt(Int32)
                  0,42 %   HxN  •  1.277 ms  •  ActiproSoftware.Internal.w4.HxN
                  0,18 %   Insert  •  543 ms  •  System.Collections.Generic.List`1.Insert(Int32, T)
                  0,15 %   7 functions hidden  •  452 ms total
                0,00 %   VxQ  •  10 ms  •  ActiproSoftware.Internal.bY.VxQ
              0,27 %   8 functions hidden  •  817 ms total
            0,04 %   qm9  •  112 ms  •  ActiproSoftware.Internal.ut.qm9(ref Nullable)
    0,00 %   UpdateTags  •  10 ms  •  Vector.ITE.Languages.Common.Language.ErrorTagging.ActiproErrorTagCache.UpdateTags(ICodeDocument, IParseErrorProvider)

#stacktrace

Here is the callstack of the main thread, which waits for the parser thread to finish:

90,01 %   OnTextInput  •  292.330 ms  •  ActiproSoftware.Windows.Controls.SyntaxEditor.Primitives.EditorView.OnTextInput(TextCompositionEventArgs)
  90,01 %   Eth  •  292.330 ms  •  ActiproSoftware.Windows.Controls.SyntaxEditor.Primitives.EditorView.Eth(TextCompositionEventArgs)
    90,01 %   PerformTyping  •  292.330 ms  •  ActiproSoftware.Windows.Controls.SyntaxEditor.Primitives.EditorView.PerformTyping(String)
      90,01 %   ExecuteEditAction  •  292.330 ms  •  ActiproSoftware.Windows.Controls.SyntaxEditor.Primitives.EditorView.ExecuteEditAction(IEditAction)
        90,01 %   Execute  •  292.330 ms  •  ActiproSoftware.Windows.Controls.SyntaxEditor.EditActions.TypingAction.Execute(IEditorView)
          90,00 %   ReplaceSelectedText  •  292.319 ms  •  ActiproSoftware.Windows.Controls.SyntaxEditor.Primitives.EditorView.ReplaceSelectedText(ITextChangeType, String, IEditorViewTextChangeOptions)
            90,00 %   Apply  •  292.319 ms  •  ActiproSoftware.Text.Implementation.TextChange.Apply
              90,00 %   ApplyTextChange  •  292.319 ms  •  ActiproSoftware.Text.Implementation.TextDocumentBase.ApplyTextChange(TextChange, Action)
                90,00 %   OnTextChanged  •  292.319 ms  •  ActiproSoftware.Text.Implementation.CodeDocument.OnTextChanged(TextSnapshotChangedEventArgs)
                  90,00 %   OnTextChanged  •  292.319 ms  •  ActiproSoftware.Text.Implementation.TextDocumentBase.OnTextChanged(TextSnapshotChangedEventArgs)
                    89,98 %   [Native code]  •  292.244 ms
                      89,98 %   OnEvent  •  292.244 ms  •  ActiproSoftware.Text.Utility.WeakEventListener`2.OnEvent(Object, TEventArgs)
                        89,98 %   j7W  •  292.244 ms  •  ActiproSoftware.Internal.ut+<>c.j7W(ut, Object, TextSnapshotChangedEventArgs)
                          89,98 %   wmc  •  292.244 ms  •  ActiproSoftware.Internal.ut.wmc(Object, TextSnapshotChangedEventArgs)
                            89,96 %   [Native code]  •  292.171 ms
                              89,96 %   Wait  •  292.171 ms  •  System.Windows.Threading.DispatcherSynchronizationContext.Wait(IntPtr[], Boolean, Int32)
                                0,06 %   2 functions hidden  •  181 ms total
                            0,02 %   2 functions hidden  •  52 ms total

#stacktrace

Maybe there is a way to improve the performance and to avoid the blocking in the main thread.


Best regards, Tobias Lingemann.

Comments (6)

Posted 1 month ago by Actipro Software Support - Cleveland, OH, USA
Avatar

Hi Tobias,

I tried opening a large C# document with over 100,000 lines and did performance profiling with it.  The way SyntaxEditor works is that it will let the parser determine initial outlining data via the parse data results and when the parse data updates, it will run automatic outlining.  The first time, it builds the complete tree.  As you edit the text from that point on, we try to do immediate updates to the tree.  For instance, typing a character in a method will increase the method's outlining node's end offset by one, along with increasing the offset of all nodes after it by one.  The modification kicks off another parse and when that asynchrously finishes later, it will try and translate offsets to the current document snapshot and do an incremental comparison of its results with the current document outlining tree.  In general, things should line up ok for most edits since we are doing the immediate changes above as you type.  When you start/end a node via editing though, that's where things could get off and more complex incremental updates might need to take place.  This incremental update period is where the main thread can get blocked.  Usually it's negligible time but if there are a lot of large updates required for a large document, then it can be noticeable.

When I did performance profiling, I didn't really see calls to RemoveAt in my stack.  Your stack is obfuscated, but I believe it's probably when adding a newly typed outlining node start, which has to take all the nodes that were after it at the same level and make them children of the new node.  That involves removing them from the original parent.

In this scenario, how many nodes do you typically have a certain level?  And what exact kind of edit operation sequences lead to the delay?  It would be helpful if you can tell us a detailed sequence of what triggers it. 

One thing that can help if it is due to starting to type a new node is what we do in our C# add-on.  When we type "{" there, it auto-inserts a "}".  That way there is less immediate pivoting around of nodes that follow that edit offset.

If you need to send us a file or communicate privately, please write our support address and reference this thread.


Actipro Software Support

Posted 1 month ago by Tobias Lingemann - Software Devolpment Engineer, Vector Informatik GmbH
Avatar

Hi,

I have generated some documents that have between 5.000 and 25.000 empty functions that look like this:

void MyFunc_00001()
{
}

Each function is a node and there is no namespace, so each node is a root node.

Usually I just write a comment inside one of the first functions and after a few key strokes, the UI starts to block. In case of the file with 5.000 functions, this takes about 25 seconds. However the time seems to grow exponentially. With 10.000 functions I already measured about 100 seconds.

Interesstingly I wasn't able to reproduce the problem with the SDI demo. With 25.000 nodes the editor is a bit stressed, but still useable. So I guess I have to figure out, where the difference comes from.


Best regards, Tobias Lingemann.

Posted 1 month ago by Actipro Software Support - Cleveland, OH, USA
Avatar

Hi Tobias,

We made a similar function to create a document with 25,000 class nodes in our C# add-on and saw the same as what you saw in the SDI demo.  It will get a little slugglish but the delays are still useable.

If you can't sort out what's causing it, please make a new simple sample project that shows the issue and send that to our support address so we can debug the scenario with your configuration.  Reference this thread in your email and be sure to exclude the bin/obj folders from the .zip you send so it doesn't get spam blocked.  Thanks!


Actipro Software Support

Posted 17 days ago by Tobias Lingemann - Software Devolpment Engineer, Vector Informatik GmbH
Avatar

It seems the issue only occurs if multiple threads are involved.

I have noticed that everything is fine, if I wait a second after each key is pressed. Each update comes from the same parser thread. If I start typing more, a second parser thread is created and the delay happens. I am not sure what happens here. Since OutlineManager has a lock, I don't see how this should affect the performance.

To test this, I have only allowed a single parser thread and the issue still occured. But when I disable all parser threads and use the main thread, the issue is gone.

To conclude, the problem is not that each update takes so long. The problem only occures, when a update is triggered, while another update is still running. When I look at the callstack and your source-code, it seems this causes an update for child nodes (childrenNeedAdjustment). Which is curious, because there shouldn't be any child nodes at all. I will continue my investigations, maybe this info already helps you.


Best regards, Tobias Lingemann.

Posted 17 days ago by Actipro Software Support - Cleveland, OH, USA
Avatar

Hi Tobias,

We appreciate you sharing the additional context.  

Regarding the childrenNeedAdjustment code, that should generally only be set to true if a node was "open" before (no end offset defined) and is now being closed, or it's closed before one of its child node's start offset.  If you are seeing a bottleneck here, perhaps the outlining tree structure you are producing needs more examination to ensure it's being created how you want (proper number of hierarchy levels and end offsets set to make "closed" nodes).  Also if you are working in enormous documents, you may wish to generally limit the count of total outlining nodes generated.

Beyond that, if you'd like us to have more direct input, you can provide us a new simple sample project that shows it occurring at our support address, referencing this thread.  Please remember to remove any bin/obj folders before zipping and sending us anything so it doesn't get blocked.  Thanks and have a great weekend!


Actipro Software Support

Answer - Posted 14 days ago by Tobias Lingemann - Software Devolpment Engineer, Vector Informatik GmbH
Avatar

I finally figured it out. The outlining source was working with the current snapshot, instead of the older snapshot the parse data was created with. This caused a missmatch between the parse data and the slightly newer snapshot.

Everything is fine now. Thanks for the help.


Best regards, Tobias Lingemann.

Add Comment

Please log in to a validated account to post comments.