SemanticParserService on a large number of files - SyntaxEditor for Windows Forms Forum

Posted 16 years ago by Adam Dickinson

Version: 4.0.0280

Platform: .NET 2.0

Environment: Windows XP (32-bit)

Hi,

My application is a project-based multi-document editor. When the user loads a project, I pre-parse all of its code files using the SemanticParserService and the SemanticParserServiceRequest constructor that takes in the text, rather than the Document. I sit and wait on a loading dialog until SemanticParserService.PendingRequestCount is 0. The problem is that some of my parse requests are not returning any results. That is, they are never calling my ISemanticParseDataTarget.NotifySemanticParseComplete() function.

I added my own parse request tracking. Immediately after calling SemanticParserService.Parse, I make sure the SemanticParserService has it pending. Later when SemanticParserService.PendingRequestCount is 0, I see which of my requests are still pending in my tracker. Here's an example of what I'm seeing in a project with 241 files (I have projects which as many as 3300 files that have the same problems). The PendingRequestCount is the number of requests in the queue immediately before that request was made.


Failed to add SemanticParserServiceRequest (Request #0).
Request #111 did not finish parsing (PendingRequestCount=110)
Request #118 did not finish parsing (PendingRequestCount=116)
Request #120 did not finish parsing (PendingRequestCount=117)
Request #126 did not finish parsing (PendingRequestCount=122)
Request #128 did not finish parsing (PendingRequestCount=123)
Request #130 did not finish parsing (PendingRequestCount=124)
Request #134 did not finish parsing (PendingRequestCount=127)
Request #136 did not finish parsing (PendingRequestCount=128)
Request #142 did not finish parsing (PendingRequestCount=133)
Request #144 did not finish parsing (PendingRequestCount=134)
Request #146 did not finish parsing (PendingRequestCount=135)
Request #149 did not finish parsing (PendingRequestCount=137)
Request #213 did not finish parsing (PendingRequestCount=187)
Request #214 did not finish parsing (PendingRequestCount=188)
Request #216 did not finish parsing (PendingRequestCount=189)
Request #217 did not finish parsing (PendingRequestCount=189)
Request #218 did not finish parsing (PendingRequestCount=189)
Request #221 did not finish parsing (PendingRequestCount=190)
Request #223 did not finish parsing (PendingRequestCount=190)
Request #233 did not finish parsing (PendingRequestCount=200)
Request #234 did not finish parsing (PendingRequestCount=200)

There's nothing obviously wrong with the files that are lost. After the project has loaded, I can open those files and they get parsed just fine. What is strange is each file that I open has the same problem as in the first line. SemanticParserService.HasPendingRequest() returns false, but the file gets parsed anyways.

What may be weird on my end is that my ISemanticParseDataTarget.NotifySemanticParseComplete() function can fire off additional parse requests. I request for the file's dependencies to be parsed if they haven't been already. I've added a Mutex around the SemanticParserService whereever its called and it has cleaned up a couple of minor things, but the same files are being skipped every time.

I'm going to try re-requesting the skipped files, but I would sure like to know if there's anything I'm doing wrong. Please help.

Thanks in advance.

UPDATE:
I added the re-request and am still getting some unexpected behavior. (I added some helpful text before my re-queues.)


We have 6 request(s) that did not finish:
Request #242 did not finish parsing (PendingRequestCount=1)
Request #255 did not finish parsing (PendingRequestCount=14)
Request #256 did not finish parsing (PendingRequestCount=14)
Request #257 did not finish parsing (PendingRequestCount=14)
Request #258 did not finish parsing (PendingRequestCount=14)
Request #260 did not finish parsing (PendingRequestCount=14)

We have 4 request(s) that did not finish:
Request #262 did not finish parsing (PendingRequestCount=0)
Request #263 did not finish parsing (PendingRequestCount=1)
Request #264 did not finish parsing (PendingRequestCount=1)
Request #265 did not finish parsing (PendingRequestCount=1)

We have 3 request(s) that did not finish:
Request #268 did not finish parsing (PendingRequestCount=0)
Request #269 did not finish parsing (PendingRequestCount=1)
Request #270 did not finish parsing (PendingRequestCount=1)

We have 2 request(s) that did not finish:
Request #272 did not finish parsing (PendingRequestCount=0)
Request #273 did not finish parsing (PendingRequestCount=1)

We have 1 request(s) that did not finish:
Request #275 did not finish parsing (PendingRequestCount=0)

My best guess now is that I need to add some Thread.Sleep()s in and around my requests. Or maybe I need to make sure my requests are made from the main thread so that the SemanticParserService accessors return accurate and up-to-date values? What do you think?

[Modified at 02/04/2009 07:54 PM]

Comments (4)

Posted 16 years ago by Actipro Software Support - Cleveland, OH, USA

Adam,

First let me ask, is your parse target a different object for each parse? Or is it the same for each request?

With the file based loading of SemanticParserServiceRequest, it makes a ParseHashKey like this:

parseHashKey = text.GetHashCode().ToString("x") + ":" + semanticParseDataTarget.Guid;

So there is a chance I would suppose that if the target is the same and your text.GetHashCodes end up returning the same thing for multiple filenames, that it could be thinking a previously queued item is a duplicate request.

Think that could be the problem here? One way to check would be to see if any of the filename GetHashCode results return the same thing.

Actipro Software Support

Posted 16 years ago by Adam Dickinson

In this situation, my parse target is the same for every file. I checked, and the parse hash keys for the skipped files are colliding with other hash keys.

Any idea how I can work around this?

UPDATE
Are you sure that's how your parse hash keys are calculated? From the filename?

The files that are producing the same parse hash keys are very similar in name. An example:

c:\myfile.txt
c:\myfile_new.txt

I've confirmed that the values returned by string.GetHashCode() are different for each file.

[Modified at 02/05/2009 03:14 PM]

UPDATE 2
Ah, it's not the filename's hashcode you get, it's the text's. Some of my files have identical contents. Well that's just plain weird. Any particular reason you elected to go that route?

[Modified at 02/05/2009 03:21 PM]

Posted 16 years ago by Actipro Software Support - Cleveland, OH, USA

Adam,

Right, sorry the filename GetHashCode was a typo. It is the text's.

With the current build one workaround would be use a different target (different Guid on that target at least when creating the request).

We'll add filename into the parse hash key for the next build so that hash key will be made of full file path, text.GetHashCode, and the target GUID. That will make sure they are all unique.

Actipro Software Support

Posted 16 years ago by Adam Dickinson

Thanks for the info.

I came up with another alternative: When I find the duplicate key, instead of trying to parse the key, I can just copy the parse results from the other file and save myself a little bit of work.

The latest build of this product (v24.1.1) was released 27 days ago, which was after the last post in this thread.

Add Comment

Please log in to a validated account to post comments.