PythonTokenReader Class
Represents an object that can provide tokens to a ILLParser in a forward-only direction for the Python language.
public class PythonTokenReader : MergeableTokenReader, ITokenReader- Inheritance:
- object TokenReaderBase MergeableTokenReader object
- Implements:
- ITokenReader
Constructors
PythonTokenReader(ITextBufferReader, IMergeableLexer)
Represents an object that can provide tokens to a ILLParser in a forward-only direction for the Python language.
public PythonTokenReader(ITextBufferReader reader, IMergeableLexer rootLexer)| Parameter | Type | Description |
|---|---|---|
| reader | ITextBufferReader | The ITextBufferReader to use for consuming text. |
| rootLexer | IMergeableLexer | The root IMergeableLexer. |
Properties
TabSize
The tab size, used when watching for indent and dedent tokens.
Methods
GetNextToken()
Returns the next IToken that will be consumed by the token reader.
Pop()
Pops the top token reader state from the stack and restores the token reader to that state.
Push()
Pushes the token reader state onto a stack for later restoration, generally to begin a multiple token look-ahead operation.
Inherited Members
- TokenReaderBase.Advance()
- TokenReaderBase.AdvanceTo(int)
- TokenReaderBase.AdvanceTo(params int[])
- TokenReaderBase.AreNext(params int[])
- TokenReaderBase.GetLookAheadToken(int)
- TokenReaderBase.GetTokenText(IToken)
- TokenReaderBase.Initialize()
- TokenReaderBase.OffsetToPosition(int)
- TokenReaderBase.IsAtEnd
- TokenReaderBase.IsParserInErrorState
- TokenReaderBase.LookAheadToken
- TokenReaderBase.Offset
- TokenReaderBase.StackLevel
- TokenReaderBase.Token
- object.GetType()
- object.MemberwiseClone()
- object.ToString()
- object.Equals(object)
- object.Equals(object, object)
- object.ReferenceEquals(object, object)
- object.GetHashCode()