Outlining Sources and Outliners

Outlining sources are objects that an outlining manager can call upon to direct language-specific automatic outlining updates when document text changes. Several helpful base classes are included to make the creation of a syntax language's outlining source a simple task.

An outliner is a language service that returns a language-specific outlining source for a particular ITextSnapshot. Having an outliner service registered on a language is what tells SyntaxEditor that the language supports automatic outlining.

Outlining Source Basics

Outlining sources are represented by the IOutliningSource interface. This interface defines a single GetNodeAction method that is called by the IOutliningManager.

When a text change occurs and the outlining node hierarchy needs to be incrementally updated, the manager iterates through the existing nodes and compares them to the actions indicated by an IOutliningSource at the same offset. For each offset that is examined, the manager calls the GetNodeAction method and passes it by reference the offset to examine. It expects the outlining source to return the OutliningNodeAction to take (None, Start, or End) at that offset. None means that no outlining node should start or end at this offset. Start means that an outlining node should start at this offset. And likewise, End means that an outlining node should end at this offset. If the action is Start or End, then an IOutliningNodeDefinition needs to be returned via an out parameter. The node definition helps the manager to create new nodes and match ends to existing nodes. Also, for an optimization, the offset passed can be moved ahead to indicate the next meaningful offset for the outlining source. When used properly this can result in a lot less calls to GetNodeAction while still returning the same accurate data.

SyntaxEditor comes with two built-in base classes that make it easier to construct an outlining source. One typically executes in the main thread, uses simple token pairs to determine how to create outlining nodes, and is very quick to set up. The other generally executes in a worker thread by a parser and is better suited for potential large documents but requires a bit more code to get working. Both are described in detail below.

Outliners

Outliners, represented by the IOutliner interface, are objects that can create/retrieve an IOutliningSource for use by an outlining manager when it needs to update its outlining node hierarchy. The presence of an outliner service on a language tells the outlining manager that the language supports automatic outlining.

The IOutliner interface has two members. The first member is the UpdateTrigger property, that returns an AutomaticOutliningUpdateTrigger enumeration value indicating what event should trigger an automatic outlining update. If the outlining source is to be used in the main thread, such as with the token-based outlining source, then a TextChanged update trigger should be returned. This tells the outlining manager to update outlining data whenever a text change occurs. If the outlining source is to be generated by a parser, such as with a range-based outlining source, then a ParseDataChanged update trigger should be used. This tells the outlining manager to update outlining data whenever the ICodeDocument.ParseData property is changed.

The second member defined by IOutliner is the GetOutliningSource method, which returns an IOutliningSource to use for a given ITextSnapshot.

If you will be using an outlining source that derives from TokenOutliningSourceBase, you can use the built-in TokenOutliner class to create your outlining source as needed. The TokenOutliner class implements IOutliner and is able to create instances of the outlining source type indicated as a type parameter, assuming the outlining source type accepts a single ITextSnapshot constructor parameter.

Registering with a Syntax Language

As described above, IOutliner objects can be used as language services to provide outlining sources to any outlining manager that requests them.

This code shows how to register an outliner language service, assuming JavascriptOutliner implements IOutliner.

language.RegisterService<IOutliner>(new JavascriptOutliner());

Token-Based Outlining Sources

A token-based outlining source executes in the main UI thread immediately after any text change occurs. It examines tokens from the language's lexer and returns whether a token indicates a start or end of an outlining node, or neither. For instance, a token for an open curly brace would generally indicate an outlining node start. A token for a close curly brace would generally indicate an outlining node end.

Benefits and Drawbacks

The benefits of this sort of outlining source are that it's extremely fast to get up and running with, and the outlining UI is always up-to-date with the document.

The drawbacks are that there is limited customization for when nodes are created, and since it's updating in the main thread, it has the potential to slow down typing speed when used on a very large document. For small to medium size documents, it should perform well though.

Implementation

A token-based outlining source can be created by inheriting the abstract TokenOutliningSourceBase class. The inherited class you create simply needs to override its GetNodeActionForToken method. That method is passed an IToken and expects an OutliningNodeAction returned. If the action is Start or End, then an appropriate IOutliningNodeDefinition must be returned via the "out" parameter. The outlining manager uses this information to determine how to sync up its outlining node hierarchy with your outlining source.

This code shows a sample token-based outlining source for the Javascript language:

public class JavascriptOutliningSource : TokenOutliningSourceBase {

	private static OutliningNodeDefinition curlyBraceDefinition;
	private static OutliningNodeDefinition multiLineCommentDefinition;

	static JavascriptOutliningSource() {
		curlyBraceDefinition = new OutliningNodeDefinition("CurlyBrace");
		curlyBraceDefinition.IsImplementation = true;

		multiLineCommentDefinition = new OutliningNodeDefinition("MultiLineComment");
		multiLineCommentDefinition.DefaultCollapsedContent = "/**/";
		multiLineCommentDefinition.IsImplementation = true;
	}

	public JavascriptOutliningSource(ITextSnapshot snapshot) : base(snapshot) {}

	protected override OutliningNodeAction GetNodeActionForToken(IToken token,
		out IOutliningNodeDefinition definition) {

		switch (token.Key) {
			case "MultiLineCommentStartDelimiter":
				definition = multiLineCommentDefinition;
				return OutliningNodeAction.Start;
			case "MultiLineCommentEndDelimiter":
				definition = multiLineCommentDefinition;
				return OutliningNodeAction.End;
			case "OpenCurlyBrace":
				definition = curlyBraceDefinition;
				return OutliningNodeAction.Start;
			case "CloseCurlyBrace":
				definition = curlyBraceDefinition;
				return OutliningNodeAction.End;
			default:
				definition = null;
				return OutliningNodeAction.None;
		}
	}
}

The outlining source could be used by a language by registering a TokenOutliner service:

language.RegisterService<IOutliner>(
	new TokenOutliner<JavascriptOutliningSource>());

Range-Based Outlining Sources

A range-based outlining source is generated by an IParser in a worker thread. The parser scans and provides outlining data for the entire document in the form of text ranges and corresponding outlining node definitions. This could be done via simple token scanning or by examining an abstract syntax tree (AST) that is constructed by the parser immediately beforehand. Since a majority of the work is done in a separate thread, there is almost no impact to the UI thread.

Benefits and Drawbacks

The benefits of this sort of outlining source are that it doesn't slow down the main UI thread (even for relatively large documents) when typing and allows for complete customization of what text ranges become outlining nodes. For example, you can choose to only make outlining nodes for top-level curly braces, instead of curly braces at all levels.

The drawbacks are that it is slightly more complex than the simpler token-based outlining source mechanism, and there can be a brief delay between when typing and when outlining node UI pops into the margin, due to the multi-threading.

Using Abstract Syntax Trees

Advanced syntax language developers will want to build an IParser to construct an AST of their document, and use that AST data to create the appropriate outlining node ranges for their outlining source. Say your syntax language is for C# and assume you have a parser that makes an AST of the document. Your AST knows which text ranges are namespaces, which are classes, etc. Therefore, that sort of AST node data can be used to construct related outlining node data.

Implementation

A range-based outlining source can be created by inheriting the abstract RangeOutliningSourceBase class. The inherited class you create should configure the outlining node ranges in its constructor. The constructor can be passed an ITextSnapshot and/or AST data if available.

Regardless of which method will be used to determine where outlining node ranges are, the concept remains the same. Some sort of iteration/recursion should take place and when an outlining node range is found, it can be configured by calling the AddNode method. This method takes a TextRange (relative to the ITextSnapshot passed in) and the IOutliningNodeDefinition to use for that range.

Sometimes certain nodes may not have an end. This could happen in VB if a Namespace ... End Namespace block contains a Class declaration however no End Class has yet been typed. In this case the outlining node for the Class is considered "open". It can be added by calling the AddOpenNode method. That method accepts a start offset instead of a TextRange.

Here is some sample code showing how a range-based Javascript outlining source could be made that only creates outlining nodes for multi-line comments and top-level curly braces. This sample does token scanning in the ITextSnapshot. However, if you already have built an AST of the document, that would be more preferable for use.

public class JavascriptOutliningSource : RangeOutliningSourceBase {

	private static OutliningNodeDefinition curlyBraceDefinition;
	private static OutliningNodeDefinition multiLineCommentDefinition;

	static JavascriptOutliningSource() {
		curlyBraceDefinition = new OutliningNodeDefinition("CurlyBrace");
		curlyBraceDefinition.IsImplementation = true;

		multiLineCommentDefinition = new OutliningNodeDefinition("MultiLineComment");
		multiLineCommentDefinition.DefaultCollapsedContent = "/**/";
		multiLineCommentDefinition.IsImplementation = true;
	}

	public JavascriptOutliningSource(ITextSnapshot snapshot) : base(snapshot) {
		int curlyBraceStartOffset = -1;
		int curlyBraceLevel = 0;
		int commentStartOffset = -1;
		IToken token;

		// Get a text snapshot reader so that we can read tokens
		ITextSnapshotReader reader = snapshot.GetReader(0);

		// Read through the entire snapshot
		while (!reader.IsAtSnapshotEnd) {
			// Get the next token
			token = reader.ReadToken();
			if (token != null) {
				switch (token.Key) {
					case "MultiLineCommentStartDelimiter":
						// A multi-line comment is starting... save its start offset
						if (commentStartOffset == -1)
							commentStartOffset = token.StartOffset;
						break;
					case "MultiLineCommentEndDelimiter":
						// A multi-line comment is ending... add its range to the outlining source
						if (commentStartOffset != -1) {
							this.AddNode(new TextRange(commentStartOffset, token.EndOffset), multiLineCommentDefinition);
							commentStartOffset = -1;
						}
						break;
					case "OpenCurlyBrace":
						// An open curly brace... save its start offset if it's a top-level brace
						if (curlyBraceLevel++ == 0) {
							if (curlyBraceStartOffset == -1)
								curlyBraceStartOffset = token.StartOffset;
						}
						break;
					case "CloseCurlyBrace":
						// A close curly brace... add its range to the outlining source if it's a top-level brace
						if (curlyBraceLevel > 0) {
							curlyBraceLevel--;
							if (curlyBraceLevel == 0) {
								this.AddNode(new TextRange(curlyBraceStartOffset, token.EndOffset), curlyBraceDefinition);
								curlyBraceStartOffset = -1;
							}
						}
						break;
				}
			}
			else
				break;
		}

		// If there are any "open" nodes (never found a matching end), add them too
		if (commentStartOffset != -1)
			this.AddOpenNode(commentStartOffset, multiLineCommentDefinition);
		if (curlyBraceStartOffset != -1)
			this.AddOpenNode(curlyBraceStartOffset, curlyBraceDefinition);
	}
}

This code shows a sample outliner for the outlining source above. Note that since much of the outlining work is occurring in a worker thread, the editor control may already be on a new text snapshot by the time the parsing completes. Therefore we do a quick call to TranslateTo to ensure the data in the outlining source that will be used by the outlining manager are up-to-date with the current snapshot in the editor.

public class JavascriptOutliner : IOutliner {

	public IOutliningSource GetOutliningSource(ITextSnapshot snapshot) {
		ICodeDocument document = snapshot.Document as ICodeDocument;
		if (document != null) {
			// Get the outlining source, which should be passed back by the IParser in the parse data
			JavacriptParseData parseData = document.ParseData as JavacriptParseData;
			if ((parseData != null) && (parseData.OutliningSource != null)) {
				// Translate the data to the desired snapshot,
				//   which could be slightly newer than the parsed source
				parseData.OutliningSource.TranslateTo(snapshot);
				return parseData.OutliningSource;
			}
		}
		return null;
	}

	public AutomaticOutliningUpdateTrigger UpdateTrigger {
		get {
			return AutomaticOutliningUpdateTrigger.ParseDataChanged;
		}
	}
}

The outlining source could be used by a language by registering a custom outliner service:

language.RegisterService<IOutliner>(
	new JavascriptOutliner());

Performance Optimizations

Remember that token-based outlining sources are generally used in the main UI thread and thus can block the UI temporarily if editing a large document and a large incremental outlining update is being made. In cases where this becomes noticeable, a range-based outlining source should be used instead. Range-based outlining sources are generally built in a worker thread by a parser, and only cause a minimal UI thread hit when merging the data into the outlining hierarchy.

Code outlining has a lot of dependence on a language's lexer. Therefore, a faster lexer can make a huge difference in outlining performance when compared to a slower lexer. Keep this in mind when looking for ways to speed up outlining performance. For instance, if you started off using a dynamic lexer for your language, you should note that while dynamic lexers are great ways to get started, they are the slowest of the lexer types. You can probably achieve a 2-300% code outlining speed increase by switching your language to a programmatic lexer instead.