HCT::Language::Token - Definition of token.
The token consists of: name, priority and list of lexemes.
The token name is an unique name, that will be returned by lexer to parser.
The token lexemes is a list of lexemes, that will be used for definition of the token name. Also as a lexeme can be used not only strings, but regular expressions too.
The token priority is a number, that means, when the token have to be used. For instance, tokens can be divided on keywords, operators, numbers and identifiers, and the order in which lexer should use them will be: operators, numbers, keywords and identifiers. According to this the priorities can be: operators - 3, numbers - 2, keywrods - 1, identifiers - 0.
The functions above will be exported by default.
Exportable subroutine to get quick access to creating of token object. Returns object of the new token.
When the lexer reach the end of input,
it returns the '' empty token with an undef value.
Creates a new HCT::Language::Token object.
Returns a list of token lexemes.
Returns token name.
Returns token priority.
This document is part of HCT, the Hardware Complexity Tool.
Generated: Fri Jul 17 10:38:47 2009