dparse.lexer

Undocumented in source.

Members

Aliases

IdType
alias IdType = TokenIdType!(operators, dynamicTokens, keywords)

Token ID type for the D lexer.

Token
alias Token = std.experimental.lexer.TokenStructure!(IdType, extraFields)

The token type in the D lexer

str
alias str = tokenStringRepresentation!(IdType, operators, dynamicTokens, keywords)

Function used for converting an IdType to a string.

Enums

StringBehavior
enum StringBehavior

Configure string lexing behavior

WhitespaceBehavior
enum WhitespaceBehavior

Configure whitespace handling

Functions

byToken
auto byToken(ubyte[] range)

Creates a token range from the given source code. Creates a default lexer configuration and a GC-managed string cache.

byToken
auto byToken(ubyte[] range, StringCache* cache)

Creates a token range from the given source code. Uses the given string cache.

byToken
auto byToken(ubyte[] range, LexerConfig config, StringCache* cache)

Creates a token range from the given source code. Uses the provided lexer configuration and string cache.

getTokensForParser
const(Token)[] getTokensForParser(ubyte[] sourceCode, LexerConfig config, StringCache* cache)
isBasicType
bool isBasicType(IdType type)
isKeyword
bool isKeyword(IdType type)
isNumberLiteral
bool isNumberLiteral(IdType type)
isOperator
bool isOperator(IdType type)
isProtection
bool isProtection(IdType type)
isStringLiteral
bool isStringLiteral(IdType type)
newlineMask
ushort newlineMask(ubyte* )
rangeMatch
ulong rangeMatch(ubyte* )
skip
void skip(ubyte* , ulong* , ulong* )

Skips between 0 and 16 bytes that match (or do not match) one of the given chars.

unDecorateComment
void unDecorateComment(string comment, T outputRange)

Removes "decoration" such as leading whitespace, leading + and * characters, and places the result into the given output range

Structs

DLexer
struct DLexer

The D lexer struct.

LexerConfig
struct LexerConfig

Lexer configuration struct

StringCache
struct StringCache

The string cache is used for string interning.

Templates

ByteCombine
template ByteCombine(c...)
Undocumented in source.
tok
template tok(string token)

Template used to refer to D token types.

Meta