Unknown bug with parsing

Hello,

this problem has not been solved yet. It happens sometimes when I open a document and that it has to parse some C# code :

System.Transactions Critical: 0 : <TraceRecord xmlns="http://schemas.microsoft.com/2004/10/E2ETraceEvent/TraceRecord" Severity="Critical"><TraceIdentifier>http://msdn.microsoft.com/TraceCodes/System/ActivityTracing/2004/07/Reliability/Exception/Unhandled</TraceIdentifier><Description>Exception non gérée</Description><AppDomain>sw7.acquisit.studio.vshost.exe</AppDomain><Exception><ExceptionType>System.ArgumentOutOfRangeException, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</ExceptionType><Message>L'index doit être dans les limites de la List.
Nom du paramètre : index</Message><StackTrace>   à System.ThrowHelper.ThrowArgumentOutOfRangeException(ExceptionArgument argument, ExceptionResource resource)
   à System.Collections.Generic.List`1.Insert(Int32 index, T item)
   à System.Collections.ObjectModel.Collection`1.InsertItem(Int32 index, T item)
   à ActiproSoftware.Text.Utility.SimpleObservableCollection`1.InsertItem(Int32 index, T item)
   à ActiproSoftware.Text.Utility.KeyedSynchronizedCollection`1.InsertItem(Int32 index, T item)
   à System.Collections.ObjectModel.Collection`1.Add(T item)
   à #Grc.#usc.Add(String sourceFileKey, CompilationUnit compilationUnit)
   à ActiproSoftware.Text.Languages.DotNet.Implementation.DotNetSyntaxLanguageBase.#NDc(String sourceFileKey, CompilationUnit compilationUnit)
   à ActiproSoftware.Text.Languages.DotNet.Implementation.DotNetSyntaxLanguageBase.OnDocumentParseDataChanged(ICodeDocument document, ParseDataPropertyChangedEventArgs e)
   à ActiproSoftware.Text.Languages.DotNet.Implementation.DotNetSyntaxLanguageBase.#MDc(ICodeDocument document, ParseDataPropertyChangedEventArgs e)
   à ActiproSoftware.Text.Implementation.CodeDocument.OnParseDataChanged(ParseDataPropertyChangedEventArgs e)
   à ActiproSoftware.Text.Implementation.CodeDocument.set_ParseData(IParseData value)
   à ActiproSoftware.Text.Implementation.CodeDocument.NotifyParseComplete(IParseRequest request, IParseData result)
   à ActiproSoftware.Text.Implementation.CodeDocument.#bGc(IParseRequest request, IParseData result)
   à ActiproSoftware.Text.Parsing.Implementation.ThreadedParseRequestDispatcher.#g.#0Yc.#nXc(IParseRequest request)
   à ActiproSoftware.Text.Parsing.Implementation.ThreadedParseRequestDispatcher.#g.#0Yc.#56c()
   à System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   à System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   à System.Threading.ThreadHelper.ThreadStart()</StackTrace><ExceptionString>System.ArgumentOutOfRangeException: L'index doit être dans les limites de la List.
Nom du paramètre : index
   à System.ThrowHelper.ThrowArgumentOutOfRangeException(ExceptionArgument argument, ExceptionResource resource)
   à System.Collections.Generic.List`1.Insert(Int32 index, T item)
   à System.Collections.ObjectModel.Collection`1.InsertItem(Int32 index, T item)
   à ActiproSoftware.Text.Utility.SimpleObservableCollection`1.InsertItem(Int32 index, T item)
   à ActiproSoftware.Text.Utility.KeyedSynchronizedCollection`1.InsertItem(Int32 index, T item)
   à System.Collections.ObjectModel.Collection`1.Add(T item)
   à #Grc.#usc.Add(String sourceFileKey, CompilationUnit compilationUnit)
   à ActiproSoftware.Text.Languages.DotNet.Implementation.DotNetSyntaxLanguageBase.#NDc(String sourceFileKey, CompilationUnit compilationUnit)
   à ActiproSoftware.Text.Languages.DotNet.Implementation.DotNetSyntaxLanguageBase.OnDocumentParseDataChanged(ICodeDocument document, ParseDataPropertyChangedEventArgs e)
   à ActiproSoftware.Text.Languages.DotNet.Implementation.DotNetSyntaxLanguageBase.#MDc(ICodeDocument document, ParseDataPropertyChangedEventArgs e)
   à ActiproSoftware.Text.Implementation.CodeDocument.OnParseDataChanged(ParseDataPropertyChangedEventArgs e)
   à ActiproSoftware.Text.Implementation.CodeDocument.set_ParseData(IParseData value)
   à ActiproSoftware.Text.Implementation.CodeDocument.NotifyParseComplete(IParseRequest request, IParseData result)
   à ActiproSoftware.Text.Implementation.CodeDocument.#bGc(IParseRequest request, IParseData result)
   à ActiproSoftware.Text.Parsing.Implementation.ThreadedParseRequestDispatcher.#g.#0Yc.#nXc(IParseRequest request)
   à ActiproSoftware.Text.Parsing.Implementation.ThreadedParseRequestDispatcher.#g.#0Yc.#56c()
   à System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   à System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   à System.Threading.ThreadHelper.ThreadStart()</ExceptionString></Exception></TraceRecord>
Une exception non gérée du type 'System.ArgumentOutOfRangeException' s'est produite dans mscorlib.dll

Informations supplémentaires : L'index doit être dans les limites de la List.

 This bug crashes my studio. Please try to do something or add a try catch somewhere so that I can continue to work when it occurs.

Based on the stack trace, it looks like it's happening in our SourceFileCollection.Add method.  That method is just calling the core collection's normal Add method, which then calls the Insert method with an index at the end of the collection.  Something in another thread must be removing an item from the collection after the core Add call but before the actual List<T>.Insert executes.

You orignally you said you can make it happen sometimes by automating a bunch of SyntaxEditor creations.  Perhaps you could try that to repro it?

Unfortunately it's not good design to put try...catch blocks in components in cases other than for things like external file/stream access, because doing so would simply mask issues that should be resolved via a fix instead.  If you are unable to find a scenario that allows us to repro this, you can always inherit the language class (CSharpSyntaxLanguage or VBSyntaxLanguage) and in an OnParseDataChanged override, put a try...catch around the base method call.

I'm making a code change through where I'll add a thread lock in the SourceFileCollection.Add method.  Since all our code for updating source code files in a project assembly run through that, that change may fix this problem.  I'm thinking of one multi-threaded scenario where without this lock, the exception you describe could occur.  Thus you might want to add the try...catch mentioned above for now, but then after getting the next maintenance release, take it back out and see if the problem still happens.

Thank you for your reply.

"You orignally you said you can make it happen sometimes by automating a bunch of SyntaxEditor creations".

 Inside my development studio using the .net addon I have a search function in all files of a solution which opens several SyntaxEditors sequentially. I use it sometimes, so I can see the bug doing this because I multiply the occurences of its production. However, I can see it also simply by opening a source file.

Since I think about it, I don't know if it crashes only in the first minute of the studio life or at any time. Let me be more clear : 

 At initialisation time I add all the source files of my solution for parsing using the IProjectAssembly.SourceFiles.QueueFile method, in a dedicated thread. May be I can have the bug only when this thread is still working and I open a specific source file with a SyntaxEditor besides that in the main UI thread. But I cannot be sure.

The thread lock might do the trick but take care of not freeze everything ;)  !

The lock shouldn't slow anything down since it's only locking over the portion between when Add gets the count of items and then calls Insert with that index.  Between there is where I believe the exception might be happening, so locking around that should help fix it since calls to the Remove from elsewhere already have the lock check in place.

I just had the bug by opening several SyntaxEditors after a search operation. In fact it's not at initialisation time. I had it hours after opening my studio, so the initial parsing thread work was over. What I don't understand, is that when I perform a seach operation I don't create another thread and everything occurs in the main UI thread : search and SyntaxEditor opening. Sure I have a ThreadedParseRequestDispatcher created at initialisation time but I don't create other threads.

AmbientParseRequestDispatcherProvider.Dispatcher = new ThreadedParseRequestDispatcher();

I have several projects on my solution, each one having a single  CSharpSyntaxLanguage instance. In my initial thread, I get the IProjectAssembly for each project :

IProjectAssembly projectAssembly = language.GetService<IProjectAssembly>();

then I queue each source file of the project for parsing :

foreach (string filePath in getCompilableFilePathes(projectDocument))
                projectAssembly.SourceFiles.QueueFile(language, filePath);

 That's all. Then if I want to open a SyntaxEditor from the UI, I just do something like :

SyntaxEditor.Document.Language = Language;
SyntaxEditor.Document.FileName = FilePath;

where Language is the CSharpSyntaxLanguage instance of the project which owns the file I want to open. Performing a search operation in my studio is nothing more than opening several SyntaxEditors on the main UI thread, hidden below in the openWindow call :

foreach (SourceFile file in Item.Definition.Document.SolutionDocument.search(Item.Definition.name.Value).OfType<SourceFile>()) {
                StudioFile studioFile = Context.Project.getStudioProject(file.Document)[file];
                FileView view = MainForm.Instance.openWindow<FileView>(MainForm.Instance.getOpenFileViewArgs(studioFile.ImageIndex, file, "search",null));
                view.selectSearch(Item.Definition.name.Value);
            }

So, I think the problem is already in your parsing infrastructure, because I use it on the main UI thread.

Since you've set up an ambient parse request dispatcher (which you should), that will use worker threads to parse documents whenever they are created.  And when those results come in, they are getting merged into the source files from the various threads, which is the area where this happens.

One more test for you... in the line where you create the new ThreadedParseRequestDispatcher, pass 1 to its constructor, meaning don't use more than one worker thread.  See if that helps provide a workaround too until the next build.

in the line where you create the new ThreadedParseRequestDispatcher, pass 1 to its constructor

I forgot this... let's try !