Slow Find & Replace with Large Documents

Posted 16 years ago by David Green

I'm hoping someone can provide ideas about what we might do to speed up a Find & Replace operation that is taking a very long time. Just assume I'm doing everything wrong when you respond and try and provide me some direction.

Here's the scenario:

* Test .Net 2.0 app in C# hosting the SyntaxEditor control (latest version)
* Open a SQL file that contains about 23,000 lines of code
* Each line has an INSERT with a table name
* I open the Find & Replace dialog and tell it to look for the curent table and replace it with a new table name (no options are checked on the dialog - no mark with bookmarks, no match whole word, no match case, no search hiddent text, and no search in selection)
* Click Replace All
* 100% CPU for 10 minutes and then I kill the app (I'm on a Pentium M 2.13GHz laptop)
* Document is attached to the control - not disconnected

So, I know I must be doing something wrong. My text editor can do this in .2 seconds so 10 minutes+ seems ridiculous and clearly points the finger at me. And by me, I mean my developer.

How would you implement a fast Find & Replace when you knew that 23,000 replaces would have to happen for the operation to complete?

All answers welcome and you won't make me feel foolish since it's one of my developers who will be yelled at when I finally figure it out for him.

Thanks.

--
David

Comments (7)

Posted 16 years ago by David Green

Bump

Posted 16 years ago by Actipro Software Support - Cleveland, OH, USA

Hi David, sorry still looking into this more before we reply.

Actipro Software Support

Posted 16 years ago by Actipro Software Support - Cleveland, OH, USA

Hi David,

The problem appears to be that we have a some updates that occur (like scrollbar max updates etc) when atomic changes occur, even when lexical parsing is disabled. So when making thousands of updates fast, it is running some of these things that many times. This is something we are looking into for the future though since obviously it isn't good to have a delay in this scenario.

What we need to do is cache more info about updates and then wait for a layout call to be made and do it once instead of for each atomic change. We're attempting to prototype out this concept in the WPF version of SyntaxEditor that we're working on. Then hopefully we can port it back in a future major version of SyntaxEditor for WinForms.

Actipro Software Support

Posted 16 years ago by David Green

That's really unfortunate for a few reasons:
* The SyntaxEditor control is at version 4 already; a mature version by any standard
* Find and Replace functionality is basic operation for a control such as this and should work on all file sizes. Currently. it's not even usable with large files and a large number of replacements.
* WPF is still a bit of a gimmic, used by no one I know for production applications. Whereas, WinForms are used everywhere. Your SyntaxEditor control is used many people. Just look at the number of posts on your Forums. You should really be concentrating on fixing the WinForms control first before moving to WPF.

I'd like to talk to someone about this in more detail - about how we use the control currently and what might help remedy this situation.

Can you forward a contact name and number over to me?

Thanks

--
David

Posted 16 years ago by Actipro Software Support - Cleveland, OH, USA

Sorry I should have responded in a bit more detail. What I was describing as a resolution is more of a long-term resolution. But we will be looking into a different way to work around this problem for the short term.

Actipro Software Support

Posted 16 years ago by David Green

Just wondering if actipro had an interim solution for this. We are getting ready to release another version of our application internally and would really benefit from improved performance in this area.

Thanks.

--
David

Posted 16 years ago by Actipro Software Support - Cleveland, OH, USA

Hi David,

We did look into it more a little while ago so let me share the results with you. After doing some profiling, we found that actually our first glance was incorrect. It isn't as much of a layout issue as we thought, it is mostly due to the updating of the Document.Tokens collection.

Basically the Document currently stores all the tokens in it when lexical parsing is enabled. Each token has a start offset and length. when changes are made to a document, all the later tokens need to either increase or decrease their start offsets based on the length delta of the text modification. This is a huge bottleneck in large documents, or in this case where we are doing hundreds of replaces.

One thing we found that helps is to change part of the code in the FindReplaceForm to this:

FindReplaceResultSet resultSet;
try {
    syntaxEditor.Document.LexicalParsingEnabled = false;
    resultSet = syntaxEditor.SelectedView.FindReplace.ReplaceAll(options);
}
catch (Exception ex) {
    MessageBox.Show(this, AssemblyInfo.Instance.Resources.GetString("FindReplaceForm_Error_Message") + "\r\n" + ex.Message, Application.ProductName, MessageBoxButtons.OK, MessageBoxIcon.Error);
    return;
}
finally {
    syntaxEditor.Document.LexicalParsingEnabled = true;
}

If you change that portion in replaceAllButton_Click in the FindReplaceForm in the sample project, you should notice a 3-4x speed increase for large documents. It doesn't completely fix the problem but it's probably about the best major change we can do at this point until we revamp the document model.

As a side note, we have already implemented some of the core of the next gen document model for our WPF prototype and that allows for bulk replaces to be processed as a single modification transaction. We added that feature specifically to address things like this scenario. Additionally the next gen model doesn't track tokens like in SE4 and that particular bottleneck is completely removed. We think these things will really help, however again it may be a while before we get that finished and moved over to the WinForms version too.

We'll continue to look into seeing if we can make some more tweaks here and there in the WinForms version to help out.

Actipro Software Support

The latest build of this product (v24.1.1) was released 4 months ago, which was after the last post in this thread.

Comments (7)

Add Comment