Package org.antlr.grammar.v2

Class AssignTokenTypesWalker

java.lang.Object

antlr.TreeParser

org.antlr.grammar.v2.AssignTokenTypesWalker

All Implemented Interfaces:

AssignTokenTypesWalkerTokenTypes

Direct Known Subclasses:

AssignTokenTypesBehavior

public class AssignTokenTypesWalker
extends antlr.TreeParser
implements AssignTokenTypesWalkerTokenTypes

[Warning: TJP says that this is probably out of date as of 11/19/2005, but since it's probably still useful, I'll leave in. Don't have energy to update at the moment.] Compute the token types for all literals and rules etc.. There are a few different cases to consider for grammar types and a few situations within. CASE 1 : pure parser grammar a) Any reference to a token gets a token type. b) The tokens section may alias a token name to a string or char CASE 2 : pure lexer grammar a) Import token vocabulary if available. Set token types for any new tokens to values above last imported token type b) token rule definitions get token types if not already defined c) literals do NOT get token types CASE 3 : merged parser / lexer grammar a) Any char or string literal gets a token type in a parser rule b) Any reference to a token gets a token type if not referencing a fragment lexer rule c) The tokens section may alias a token name to a string or char which must add a rule to the lexer d) token rule definitions get token types if not already defined e) token rule definitions may also alias a token name to a literal. E.g., Rule 'FOR : "for";' will alias FOR to "for" in the sense that references to either in the parser grammar will yield the token type What this pass does: 0. Collects basic info about the grammar like grammar name and type; Oh, I have go get the options in case they affect the token types. E.g., tokenVocab option. Imports any token vocab name/type pairs into a local hashtable. 1. Finds a list of all literals and token names. 2. Finds a list of all token name rule definitions; no token rules implies pure parser. 3. Finds a list of all simple token rule defs of form " : ;" and aliases them. 4. Walks token names table and assign types to any unassigned 5. Walks aliases and assign types to referenced literals 6. Walks literals, assigning types if untyped 4. Informs the Grammar object of the type definitions such as: g.defineToken(, ttype); g.defineToken(, ttype); g.defineToken(, ttype); where some of the ttype values will be the same for aliases tokens.

Field Summary

Fields

Modifier and Type	Field	Description
static java.lang.String[]	_tokenNames
protected static GrammarAST	charAlias
protected static GrammarAST	charAlias2
protected java.lang.String	currentRuleName
protected Grammar	grammar
protected static GrammarAST	stringAlias
protected static GrammarAST	stringAlias2

Fields inherited from class antlr.TreeParser

_retTree, astFactory, ASTNULL, inputState, returnAST, tokenNames, traceDepth

Fields inherited from interface org.antlr.grammar.v2.AssignTokenTypesWalkerTokenTypes

ACTION, ACTION_CHAR_LITERAL, ACTION_ESC, ACTION_STRING_LITERAL, ALT, AMPERSAND, ARG, ARG_ACTION, ARGLIST, ASSIGN, BACKTRACK_SEMPRED, BANG, BLOCK, CHAR_LITERAL, CHAR_RANGE, CHARSET, CLOSE_ELEMENT_OPTION, CLOSURE, COLON, COMBINED_GRAMMAR, COMMA, COMMENT, DIGIT, DOC_COMMENT, DOLLAR, DOT, DOUBLE_ANGLE_STRING_LITERAL, DOUBLE_QUOTE_STRING_LITERAL, EOA, EOB, EOF, EOR, EPSILON, ESC, ETC, FORCED_ACTION, FRAGMENT, GATED_SEMPRED, ID, IMPLIES, IMPORT, INITACTION, INT, INTERNAL_RULE_REF, LABEL, LEXER, LEXER_GRAMMAR, LITERAL_catch, LITERAL_finally, LITERAL_grammar, LITERAL_lexer, LITERAL_private, LITERAL_protected, LITERAL_public, LITERAL_returns, LITERAL_throws, LITERAL_tree, LPAREN, ML_COMMENT, NESTED_ACTION, NESTED_ARG_ACTION, NOT, NULL_TREE_LOOKAHEAD, OPEN_ELEMENT_OPTION, OPTIONAL, OPTIONS, OR, PARSER, PARSER_GRAMMAR, PLUS, PLUS_ASSIGN, POSITIVE_CLOSURE, QUESTION, RANGE, RCURLY, RET, REWRITE, ROOT, RPAREN, RULE, RULE_REF, SCOPE, SEMI, SEMPRED, SL_COMMENT, SRC, STAR, STRAY_BRACKET, STRING_LITERAL, SYN_SEMPRED, SYNPRED, TEMPLATE, TOKEN_REF, TOKENS, TREE_BEGIN, TREE_GRAMMAR, WILDCARD, WS, WS_LOOP, WS_OPT, XDIGIT

Constructor Summary

Constructors

Constructor	Description
AssignTokenTypesWalker()

Method Summary

Modifier and Type	Method	Description
protected void	alias(GrammarAST t, GrammarAST s)
protected void	aliasTokenIDsAndLiterals(Grammar root)
void	alternative(antlr.collections.AST _t)
protected void	assignStringTypes(Grammar root)
protected void	assignTokenIDTypes(Grammar root)
void	ast_suffix(antlr.collections.AST _t)
void	atom(antlr.collections.AST _t)
void	attrScope(antlr.collections.AST _t)
void	block(antlr.collections.AST _t)
void	charSet(antlr.collections.AST _t)
void	charSetElement(antlr.collections.AST _t)
protected void	defineStringLiteralsFromDelegates()
protected void	defineTokenNamesAndLiteralsInGrammar(Grammar root)
void	defineTokens(Grammar root)
void	delegateGrammars(antlr.collections.AST _t)
void	ebnf(antlr.collections.AST _t)
void	element(antlr.collections.AST _t)
void	exceptionGroup(antlr.collections.AST _t)
void	exceptionHandler(antlr.collections.AST _t)
void	finallyClause(antlr.collections.AST _t)
void	grammar(antlr.collections.AST _t, Grammar g)
void	grammarSpec(antlr.collections.AST _t)
protected void	init(Grammar root)
protected void	initASTPatterns()
void	modifier(antlr.collections.AST _t)
void	option(antlr.collections.AST _t, java.util.Map opts)
java.util.Map	optionsSpec(antlr.collections.AST _t)
java.lang.Object	optionValue(antlr.collections.AST _t)
void	reportError(antlr.RecognitionException ex)
void	rewrite(antlr.collections.AST _t)
void	rule(antlr.collections.AST _t)
void	rules(antlr.collections.AST _t)
void	ruleScopeSpec(antlr.collections.AST _t)
void	tokenSpec(antlr.collections.AST _t)
void	tokensSpec(antlr.collections.AST _t)
protected void	trackString(GrammarAST t)
protected void	trackToken(GrammarAST t)
protected void	trackTokenRule(GrammarAST t, GrammarAST modifier, GrammarAST block)
void	tree(antlr.collections.AST _t)

Methods inherited from class antlr.TreeParser

getAST, getASTFactory, getTokenName, getTokenNames, match, match, matchNot, panic, reportError, reportWarning, setASTFactory, setASTNodeClass, setASTNodeType, traceIn, traceIndent, traceOut

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

grammar

protected Grammar grammar

currentRuleName

protected java.lang.String currentRuleName

stringAlias

protected static GrammarAST stringAlias

charAlias

protected static GrammarAST charAlias

stringAlias2

protected static GrammarAST stringAlias2

charAlias2

protected static GrammarAST charAlias2

_tokenNames

public static final java.lang.String[] _tokenNames

Constructor Detail

AssignTokenTypesWalker

public AssignTokenTypesWalker()

Method Detail

reportError

public void reportError(antlr.RecognitionException ex)

Overrides:

reportError in class antlr.TreeParser

initASTPatterns

protected void initASTPatterns()

trackString

protected void trackString(GrammarAST t)

trackToken

protected void trackToken(GrammarAST t)

trackTokenRule

protected void trackTokenRule(GrammarAST t,
                              GrammarAST modifier,
                              GrammarAST block)

alias

protected void alias(GrammarAST t,
                     GrammarAST s)

defineTokens

public void defineTokens(Grammar root)

defineStringLiteralsFromDelegates

protected void defineStringLiteralsFromDelegates()

assignStringTypes

protected void assignStringTypes(Grammar root)

aliasTokenIDsAndLiterals

protected void aliasTokenIDsAndLiterals(Grammar root)

assignTokenIDTypes

protected void assignTokenIDTypes(Grammar root)

defineTokenNamesAndLiteralsInGrammar

protected void defineTokenNamesAndLiteralsInGrammar(Grammar root)

init

protected void init(Grammar root)

grammar

public final void grammar(antlr.collections.AST _t,
                          Grammar g)
                   throws antlr.RecognitionException

Throws:

antlr.RecognitionException

grammarSpec

public final void grammarSpec(antlr.collections.AST _t)
                       throws antlr.RecognitionException

Throws:

antlr.RecognitionException

optionsSpec

public final java.util.Map optionsSpec(antlr.collections.AST _t)
                                throws antlr.RecognitionException

Throws:

antlr.RecognitionException

delegateGrammars

public final void delegateGrammars(antlr.collections.AST _t)
                            throws antlr.RecognitionException

Throws:

antlr.RecognitionException

tokensSpec

public final void tokensSpec(antlr.collections.AST _t)
                      throws antlr.RecognitionException

Throws:

antlr.RecognitionException

attrScope

public final void attrScope(antlr.collections.AST _t)
                     throws antlr.RecognitionException

Throws:

antlr.RecognitionException

rules

public final void rules(antlr.collections.AST _t)
                 throws antlr.RecognitionException

Throws:

antlr.RecognitionException

option

public final void option(antlr.collections.AST _t,
                         java.util.Map opts)
                  throws antlr.RecognitionException

Throws:

antlr.RecognitionException

optionValue

public final java.lang.Object optionValue(antlr.collections.AST _t)
                                   throws antlr.RecognitionException

Throws:

antlr.RecognitionException

charSet

public final void charSet(antlr.collections.AST _t)
                   throws antlr.RecognitionException

Throws:

antlr.RecognitionException

charSetElement

public final void charSetElement(antlr.collections.AST _t)
                          throws antlr.RecognitionException

Throws:

antlr.RecognitionException

tokenSpec

public final void tokenSpec(antlr.collections.AST _t)
                     throws antlr.RecognitionException

Throws:

antlr.RecognitionException

rule

public final void rule(antlr.collections.AST _t)
                throws antlr.RecognitionException

Throws:

antlr.RecognitionException

modifier

public final void modifier(antlr.collections.AST _t)
                    throws antlr.RecognitionException

Throws:

antlr.RecognitionException

ruleScopeSpec

public final void ruleScopeSpec(antlr.collections.AST _t)
                         throws antlr.RecognitionException

Throws:

antlr.RecognitionException

block

public final void block(antlr.collections.AST _t)
                 throws antlr.RecognitionException

Throws:

antlr.RecognitionException

exceptionGroup

public final void exceptionGroup(antlr.collections.AST _t)
                          throws antlr.RecognitionException

Throws:

antlr.RecognitionException

alternative

public final void alternative(antlr.collections.AST _t)
                       throws antlr.RecognitionException

Throws:

antlr.RecognitionException

rewrite

public final void rewrite(antlr.collections.AST _t)
                   throws antlr.RecognitionException

Throws:

antlr.RecognitionException

element

public final void element(antlr.collections.AST _t)
                   throws antlr.RecognitionException

Throws:

antlr.RecognitionException

exceptionHandler

public final void exceptionHandler(antlr.collections.AST _t)
                            throws antlr.RecognitionException

Throws:

antlr.RecognitionException

finallyClause

public final void finallyClause(antlr.collections.AST _t)
                         throws antlr.RecognitionException

Throws:

antlr.RecognitionException

atom

public final void atom(antlr.collections.AST _t)
                throws antlr.RecognitionException

Throws:

antlr.RecognitionException

ebnf

public final void ebnf(antlr.collections.AST _t)
                throws antlr.RecognitionException

Throws:

antlr.RecognitionException

tree

public final void tree(antlr.collections.AST _t)
                throws antlr.RecognitionException

Throws:

antlr.RecognitionException

ast_suffix

public final void ast_suffix(antlr.collections.AST _t)
                      throws antlr.RecognitionException

Throws:

antlr.RecognitionException