Struct Token

Namespace

BlazOrbit.SyntaxHighlight.Tokens

Assembly

BlazOrbit.SyntaxHighlight.dll

Represents a single token produced by the tokenizer.

public readonly struct Token

Constructors

Token(TokenType, string, int, int)

Represents a single token produced by the tokenizer.

public Token(TokenType Type, string Value, int StartIndex, int Length)

Parameters

Type TokenType

Value string

StartIndex int

Length int

Properties

EndIndex

Gets the end index of the token in the source text.

public int EndIndex { get; }

Property Value

int

Length

public int Length { get; init; }

Property Value

int

StartIndex

public int StartIndex { get; init; }

Property Value

int

Type

public TokenType Type { get; init; }

Property Value

TokenType

Value

public string Value { get; init; }

Property Value

string

Methods

Text(string, int)

Creates a text token at the specified start index.

public static Token Text(string value, int startIndex)

Parameters

value string

The token value.

startIndex int

The start index in the source text.

Returns

Token

A new text token.