In This Article

MergeableLexerBase Class

Represents the base implementation of a lexer (lexical analyzer) for a language that supports merging with other languages.

public abstract class MergeableLexerBase : IMergeableLexer, ILexer, IKeyedObject
Inheritance:
object object
Derived:
DotNetLexerBase JavaScriptLexer JsonLexer PythonLexer DynamicLexer
Implements:
IMergeableLexer ILexer IKeyedObject

Remarks

Instances of this object can be registered with an ISyntaxLanguage using the RegisterService(object, object) method. Once an instance is registered with the language for the ILexer interface type, its features can be used by the language.

Constructors

MergeableLexerBase()

Initializes an instance of the class.

protected MergeableLexerBase()

Properties

DefaultLexicalStateCore

The root ILexicalState for when the language is entered.

protected ILexicalState? DefaultLexicalStateCore { get; set; }

Property Value

ILexicalState

KeyCore

A string that identifies the lexer, generally the related language name.

protected string KeyCore { get; set; }

Property Value

string

LexicalStateIdProviderCore

The ILexicalStateIdProvider that provides access to all the lexical state IDs and keys used by this lexer.

protected ILexicalStateIdProvider? LexicalStateIdProviderCore { get; set; }

Property Value

ILexicalStateIdProvider

TokenIdProviderCore

The ITokenIdProvider that provides access to all the token IDs and keys used by this lexer.

protected ITokenIdProvider? TokenIdProviderCore { get; set; }

Property Value

ITokenIdProvider

Methods

CreateChangeBatch()

Creates an IDisposable object such that while the object is not disposed, the lexer can be modified without any change events being fired, thereby allowing batch updates to occur.

public IDisposable CreateChangeBatch()

Returns

IDisposable:

An IDisposable object that, once disposed, will end the batch.

CreateDocumentEndToken(int, TextPosition, ILexicalState)

Creates an IMergeableToken that represents the end of a document.

public virtual IMergeableToken CreateDocumentEndToken(int startOffset, TextPosition startPosition, ILexicalState lexicalState)
Parameter Type Description
startOffset int

The start offset of the IToken.

startPosition TextPosition

The start TextPosition of the IToken.

lexicalState ILexicalState

The ILexicalState that contains the token.

Returns

IMergeableToken

CreateToken(int, int, TextPosition, TextPosition, MergeableLexerFlags, ILexicalState, IMergeableTokenLexerData?)

Creates an IMergeableToken that represents the range of text with the specified lexer data.

public virtual IMergeableToken CreateToken(int startOffset, int length, TextPosition startPosition, TextPosition endPosition, MergeableLexerFlags lexerFlags, ILexicalState lexicalState, IMergeableTokenLexerData? lexerData)
Parameter Type Description
startOffset int

The start offset of the token.

length int

The length of the token.

startPosition TextPosition

The TextPosition of the first character in the token.

endPosition TextPosition

The TextPosition after the last character in the token.

lexerFlags MergeableLexerFlags

The MergeableLexerFlags for the token.

lexicalState ILexicalState

The ILexicalState that contains the token.

lexerData IMergeableTokenLexerData

The IMergeableTokenLexerData that contains lexer information about the token.

Returns

IMergeableToken

GetAllLexicalStateTransitions()

Returns a collection containing all ILexicalStateTransition objects within the lexer.

public abstract IEnumerable<ILexicalStateTransition>? GetAllLexicalStateTransitions()

Returns

IEnumerable<ILexicalStateTransition>

Remarks

This method allows consumers to see which language transitions can be made within the lexer.

GetDefaultToken(ITextBufferReader, ILexicalState)

Returns a single-character MergeableLexerResult representing the lexer result for the default token in the ILexicalState and seeks forward one position in the ITextBufferReader

public virtual MergeableLexerResult GetDefaultToken(ITextBufferReader reader, ILexicalState lexicalState)
Parameter Type Description
reader ITextBufferReader

An ITextBufferReader that is reading a text source.

lexicalState ILexicalState

The ILexicalState that specifies the current state.

Returns

MergeableLexerResult

GetIncrementalParseRange(TextSnapshotRange)

Returns a TextSnapshotRange that indicates the minimum range to parse when incremental parsing for the specified TextSnapshotRange is requested.

protected virtual TextSnapshotRange GetIncrementalParseRange(TextSnapshotRange snapshotRange)
Parameter Type Description
snapshotRange TextSnapshotRange

The requested TextSnapshotRange to incrementally parse.

Returns

TextSnapshotRange:

The actual TextSnapshotRange to incrementally parse.

Remarks

When performing incremental parsing, a larger snapshot range than the actual modified snapshot range should usually be parsed. For instance, if the end user types text at the end of a word that should make the word a keyword, incremental parsing should start at least at the start of the word or else it won't be recognized as a keyword.

The default implementation of this method starts incremental parsing at the start of the line at which the requested snapshot range begins.

This method is called by the default Parse(TextSnapshotRange, ILexerTarget) implementation to modify the snapshot range to parse.

GetNextToken(ITextBufferReader, ILexicalState)

Performs a lex to return the next MergeableLexerResult from a ITextBufferReader and seeks past it if there is a match.

public abstract MergeableLexerResult GetNextToken(ITextBufferReader reader, ILexicalState lexicalState)
Parameter Type Description
reader ITextBufferReader

An ITextBufferReader that is reading a text source.

lexicalState ILexicalState

The ILexicalState that specifies the current state.

Returns

MergeableLexerResult

OnChanged(EventArgs)

Raises the OnChanged(EventArgs) event.

protected virtual void OnChanged(EventArgs e)
Parameter Type Description
e EventArgs

The event data.

Parse(TextSnapshotRange, ILexerTarget)

Lexically parses the specified TextRange of the ITextSnapshot.

public virtual TextRange Parse(TextSnapshotRange snapshotRange, ILexerTarget parseTarget)
Parameter Type Description
snapshotRange TextSnapshotRange

A TextSnapshotRange indicating the offset range to parse.

parseTarget ILexerTarget

The ILexerTarget that receives the parsing output.

Returns

TextRange:

A TextRange containing the offset range that was modified.

Events

Changed

Occurs after the lexer is changed via a block of code surrounded by CreateChangeBatch().

public event EventHandler? Changed

Event Type

EventHandler

Inherited Members

Extension Methods