Token object using up too much memory

SyntaxEditor for Windows Forms Forum

Posted 16 years ago by (Anonymous)
Version: 3.1.0222
Avatar
I am using Syntax Editor to display rolling data. This means that I append data at the bottom and remove data at the top. Data scrolls dynamically for a period of hours.

When I profile memory with Ants, I notice that I am accumulating Token objects, more than 100,000. It appears that Token objects get created but not garbage collected.

I am not replacing or giving the control any Token objects of my own. It uses whatever it uses by default. We are using this like a simple text control without any parsing.

Is there some call I need to make to free up the Token objects or is this related to the way I append data and delete data?

I do my deletes like this:


RawData.Document.UndoableDelete(DocumentModificationType.Delete, 0, textToCrop);

And my appends like this:

RawData.Document.UndoableInsert(DocumentModificationType.Typing, RawData.Document.Length, 0, allDevices.ToString());

Mike.Jones@woofumdust.com

[Modified at 01/11/2008 05:38 PM]

Comments (5)

Posted 16 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar
Hi Mike,

I'm profiling our v3.1 test application and am basically doing what you are doing. I type some text at the end of the document and delete some from the start. As I do this, I see the live instances of the Token class change appropriately. If I delete out almost all of the test, the number of live token instances drops to just around 3.

This is in the latest build here, which really shouldn't be any different than the build 222 you have.


Actipro Software Support

Posted 16 years ago by (Anonymous)
Avatar
Thanks for the reply. Note that I only add and delete text programmaticly, but I assume it makes no difference.

I noticed that after a period of time the number of Token object started to stabilize (Watching computer memory usage). The way the application works is I add text at the bottom until we reach 100 pages or so and then start taking text off the top as I add it to the bottom.

Perhaps I did not wait long enough to see if the number of tokens stabilize when using Ants.

The text has numbers and words. Do I get one token per word?

Eventually, I want to really parse this data and highlight certain items. If I do that will it reduce the number of tokens because it can find what it wants, or is it like a lexer that will have to generate tokens for all words so I can parse the ones I want?

I have not used the parsing aspect of the control, so that is why the above question. I use it mainly because of its performance, not its features, however I want to use the features in the future.

Mike
Posted 16 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar
It could have just been that the garbage collection has not yet taken place. You could probably kick off GC.Collect every once and a while if you want to try and force some collection sooner.

The token creation completely depends on which language definition you are using. Each language uses a lexer to tokenize text. If you are using the plain text one then it will break things up into words and numbers. Once you get into highlighting you can parse text however you wish using patterns to tokenize the text.

But actually for now if you aren't even coloring any text, I'd highly recommend turning off lexical parsing (Document.LexicalParsingEnabled) and semantic parsing (Document.SemanticParsingEnabled). Those will remove tokens from being used at all (saving memory and increasing performance) but of course will prevent you from using syntax highlighting down the road.


Actipro Software Support

Posted 16 years ago by (Anonymous)
Avatar
Are the two properties used to stop parsing only available on a newer version that what I have? I don't see them in the designer nor are they available in the code.

I have v3.1.0222
Posted 16 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar
My apologies... You are correct, those were some big features that were added in v4.0.


Actipro Software Support

The latest build of this product (v24.1.0) was released 2 months ago, which was after the last post in this thread.

Add Comment

Please log in to a validated account to post comments.