Package antlr

Class PythonCodeGenerator

  • public class PythonCodeGenerator
extends CodeGenerator
    Generate MyParser.java, MyLexer.java and MyParserTokenTypes.java

    • Field Detail

      • syntacticPredLevel

        protected int syntacticPredLevel

      • genAST

        protected boolean genAST

      • saveText

        protected boolean saveText

      • labeledElementType

        java.lang.String labeledElementType

      • labeledElementASTType

        java.lang.String labeledElementASTType

      • labeledElementInit

        java.lang.String labeledElementInit

      • commonExtraArgs

        java.lang.String commonExtraArgs

      • commonExtraParams

        java.lang.String commonExtraParams

      • commonLocalVars

        java.lang.String commonLocalVars

      • lt1Value

        java.lang.String lt1Value

      • exceptionThrown

        java.lang.String exceptionThrown

      • throwNoViable

        java.lang.String throwNoViable

      • initHeaderAction

        public static final java.lang.String initHeaderAction
        See Also:
        Constant Field Values

      • mainHeaderAction

        public static final java.lang.String mainHeaderAction
        See Also:
        Constant Field Values

      • lexerClassName

        java.lang.String lexerClassName

      • parserClassName

        java.lang.String parserClassName

      • treeWalkerClassName

        java.lang.String treeWalkerClassName

      • currentRule

        RuleBlock currentRule
        Tracks the rule being generated. Used for mapTreeId

      • currentASTResult

        java.lang.String currentASTResult
        Tracks the rule or labeled subrule being generated. Used for AST generation.

      • treeVariableMap

        java.util.Hashtable treeVariableMap
        Mapping between the ids used in the current alt, and the names of variables used to represent their AST values.

      • declaredASTVariables

        java.util.Hashtable declaredASTVariables
        Used to keep track of which AST variables have been defined in a rule (except for the #rule_name and #rule_name_in var's

      • astVarNumber

        int astVarNumber

      • NONUNIQUE

        protected static final java.lang.String NONUNIQUE
        Special value used to mark duplicate in treeVariableMap

    • Constructor Detail

      • PythonCodeGenerator

        public PythonCodeGenerator()

    • Method Detail

      • printTabs

        protected void printTabs()
        Create a Java code-generator using the given Grammar. The caller must still call setTool, setBehavior, and setAnalyzer before generating code.
        Overrides:
        printTabs in class CodeGenerator

      • addSemPred

        protected int addSemPred​(java.lang.String predicate)
        Adds a semantic predicate string to the sem pred vector These strings will be used to build an array of sem pred names when building a debugging parser. This method should only be called when the debug option is specified

      • exitIfError

        public void exitIfError()

      • checkCurrentOutputStream

        protected void checkCurrentOutputStream()

      • extractIdOfAction

        protected java.lang.String extractIdOfAction​(java.lang.String s,
                                             int line,
                                             int column)
        Get the identifier portion of an argument-action. For Python the ID of an action is assumed to be everything before the assignment, as Python does not support a type.
        Overrides:
        extractIdOfAction in class CodeGenerator
        Parameters:
        s - The action text
        line - Line used for error reporting.
        column - Line used for error reporting.
        Returns:
        A string containing the text of the identifier

      • extractTypeOfAction

        protected java.lang.String extractTypeOfAction​(java.lang.String s,
                                               int line,
                                               int column)
        Get the type portion of an argument-action. Python does not have a type declaration before an identifier, so we just return the empty string.
        Overrides:
        extractTypeOfAction in class CodeGenerator
        Parameters:
        s - The action text
        line - Line used for error reporting.
        Returns:
        A string containing the text of the type

      • flushTokens

        protected void flushTokens()

      • gen

        public void gen()
        Generate the parser, lexer, treeparser, and token types in Java
        Specified by:
        gen in class CodeGenerator

      • gen

        public void gen​(ActionElement action)
        Generate code for the given grammar element.
        Specified by:
        gen in class CodeGenerator
        Parameters:
        blk - The {...} action to generate

      • gen

        public void gen​(AlternativeBlock blk)
        Generate code for the given grammar element.
        Specified by:
        gen in class CodeGenerator
        Parameters:
        blk - The "x|y|z|..." block to generate

      • gen

        public void gen​(BlockEndElement end)
        Generate code for the given grammar element.
        Specified by:
        gen in class CodeGenerator
        Parameters:
        blk - The block-end element to generate. Block-end elements are synthesized by the grammar parser to represent the end of a block.

      • gen

        public void gen​(CharLiteralElement atom)
        Generate code for the given grammar element.
        Specified by:
        gen in class CodeGenerator
        Parameters:
        blk - The character literal reference to generate

      • toString

        java.lang.String toString​(boolean v)

      • gen

        public void gen​(CharRangeElement r)
        Generate code for the given grammar element.
        Specified by:
        gen in class CodeGenerator
        Parameters:
        blk - The character-range reference to generate

      • gen

        public void gen​(LexerGrammar g)
         throws java.io.IOException
        Generate the lexer Java file
        Specified by:
        gen in class CodeGenerator
        Throws:
        java.io.IOException

      • genHeaderMain

        protected void genHeaderMain​(Grammar grammar)

      • genHeaderInit

        protected void genHeaderInit​(Grammar grammar)

      • printMainFunc

        protected void printMainFunc​(java.lang.String s)

      • gen

        public void gen​(OneOrMoreBlock blk)
        Generate code for the given grammar element.
        Specified by:
        gen in class CodeGenerator
        Parameters:
        blk - The (...)+ block to generate

      • gen

        public void gen​(ParserGrammar g)
         throws java.io.IOException
        Generate the parser Java file
        Specified by:
        gen in class CodeGenerator
        Throws:
        java.io.IOException

      • gen

        public void gen​(RuleRefElement rr)
        Generate code for the given grammar element.
        Specified by:
        gen in class CodeGenerator
        Parameters:
        blk - The rule-reference to generate

      • gen

        public void gen​(StringLiteralElement atom)
        Generate code for the given grammar element.
        Specified by:
        gen in class CodeGenerator
        Parameters:
        blk - The string-literal reference to generate

      • gen

        public void gen​(TokenRangeElement r)
        Generate code for the given grammar element.
        Specified by:
        gen in class CodeGenerator
        Parameters:
        blk - The token-range reference to generate

      • gen

        public void gen​(TokenRefElement atom)
        Generate code for the given grammar element.
        Specified by:
        gen in class CodeGenerator
        Parameters:
        blk - The token-reference to generate

      • gen

        public void gen​(TreeWalkerGrammar g)
         throws java.io.IOException
        Generate the tree-parser Java file
        Specified by:
        gen in class CodeGenerator
        Throws:
        java.io.IOException

      • gen

        public void gen​(WildcardElement wc)
        Generate code for the given grammar element.
        Specified by:
        gen in class CodeGenerator
        Parameters:
        wc - The wildcard element to generate

      • gen

        public void gen​(ZeroOrMoreBlock blk)
        Generate code for the given grammar element.
        Specified by:
        gen in class CodeGenerator
        Parameters:
        blk - The (...)* block to generate

      • genAlt

        protected void genAlt​(Alternative alt,
                      AlternativeBlock blk)
        Generate an alternative.
        Parameters:
        alt - The alternative to generate
        blk - The block to which the alternative belongs

      • genBitsets

        protected void genBitsets​(Vector bitsetList,
                          int maxVocabulary)
        Generate all the bitsets to be used in the parser or lexer Generate the raw bitset data like "long _tokenSet1_data[] = {...}" and the BitSet object declarations like "BitSet _tokenSet1 = new BitSet(_tokenSet1_data)" Note that most languages do not support object initialization inside a class definition, so other code-generators may have to separate the bitset declarations from the initializations (e.g., put the initializations in the generated constructor instead).
        Parameters:
        bitsetList - The list of bitsets to generate.
        maxVocabulary - Ensure that each generated bitset can contain at least this value.

      • genBlockInitAction

        protected void genBlockInitAction​(AlternativeBlock blk)
        Generate the init action for a block, which may be a RuleBlock or a plain AlternativeBLock.

      • genBlockPreamble

        protected void genBlockPreamble​(AlternativeBlock blk)
        Generate the header for a block, which may be a RuleBlock or a plain AlternativeBLock. This generates any variable declarations and syntactic-predicate-testing variables.

      • genCases

        protected void genCases​(BitSet p)
        Generate a series of case statements that implement a BitSet test.
        Parameters:
        p - The Bitset for which cases are to be generated

      • genCommonBlock

        public PythonBlockFinishingInfo genCommonBlock​(AlternativeBlock blk,
                                               boolean noTestForSingle)
        Generate common code for a block of alternatives; return a postscript that needs to be generated at the end of the block. Other routines may append else-clauses and such for error checking before the postfix is generated. If the grammar is a lexer, then generate alternatives in an order where alternatives requiring deeper lookahead are generated first, and EOF in the lookahead set reduces the depth of the lookahead. @param blk The block to generate @param noTestForSingle If true, then it does not generate a test for a single alternative.

      • genASTDeclaration

        protected void genASTDeclaration​(AlternativeElement el,
                                 java.lang.String node_type)

      • genASTDeclaration

        protected void genASTDeclaration​(AlternativeElement el,
                                 java.lang.String var_name,
                                 java.lang.String node_type)

      • genHeader

        protected void genHeader()
        Generate a header that is common to all Python files

      • genLexerTest

        protected void genLexerTest()
        Generate an automated test for Python CharScanner (sub)classes.

      • genMatch

        protected void genMatch​(BitSet b)

      • genMatch

        protected void genMatch​(GrammarAtom atom)

      • genMatchUsingAtomText

        protected void genMatchUsingAtomText​(GrammarAtom atom)

      • genMatchUsingAtomTokenType

        protected void genMatchUsingAtomTokenType​(GrammarAtom atom)

      • genNextToken

        public void genNextToken()
        Generate the nextToken() rule. nextToken() is a synthetic lexer rule that is the implicit OR of all user-defined lexer rules.

      • genRule

        public void genRule​(RuleSymbol s,
                    boolean startSymbol,
                    int ruleNum)
        Gen a named rule block. ASTs are generated for each element of an alternative unless the rule or the alternative have a '!' modifier. If an alternative defeats the default tree construction, it must set _AST to the root of the returned AST. Each alternative that does automatic tree construction, builds up root and child list pointers in an ASTPair structure. A rule finishes by setting the returnAST variable from the ASTPair.
        Parameters:
        rule - The name of the rule to generate
        startSymbol - true if the rule is a start symbol (i.e., not referenced elsewhere)

      • genSemPred

        protected void genSemPred​(java.lang.String pred,
                          int line)

      • genSemPredMap

        protected void genSemPredMap()
        Write an array of Strings which are the semantic predicate expressions. The debugger will reference them by number only

      • genSynPred

        protected void genSynPred​(SynPredBlock blk,
                          java.lang.String lookaheadExpr)

      • genTokenStrings

        public void genTokenStrings()
        Generate a static array containing the names of the tokens, indexed by the token type values. This static array is used to format error messages so that the token identifers or literal strings are displayed instead of the token numbers. If a lexical rule has a paraphrase, use it rather than the token label.

      • genTokenASTNodeMap

        protected void genTokenASTNodeMap()
        Create and set Integer token type objects that map to Java Class objects (which AST node to create).

      • genTokenTypes

        protected void genTokenTypes​(TokenManager tm)
                      throws java.io.IOException
        Generate the token types Java file
        Throws:
        java.io.IOException

      • getASTCreateString

        public java.lang.String getASTCreateString​(Vector v)
        Get a string for an expression to generate creation of an AST subtree.
        Specified by:
        getASTCreateString in class CodeGenerator
        Parameters:
        v - A Vector of String, where each element is an expression in the target language yielding an AST node.

      • getASTCreateString

        public java.lang.String getASTCreateString​(GrammarAtom atom,
                                           java.lang.String astCtorArgs)
        Get a string for an expression to generate creating of an AST node
        Specified by:
        getASTCreateString in class CodeGenerator
        Parameters:
        atom - The grammar node for which you are creating the node
        str - The arguments to the AST constructor

      • getASTCreateString

        public java.lang.String getASTCreateString​(java.lang.String astCtorArgs)
        Get a string for an expression to generate creating of an AST node. Parse the first (possibly only) argument looking for the token type. If the token type is a valid token symbol, ask for it's AST node type and add to the end if only 2 arguments. The forms are #[T], #[T,"t"], and as of 2.7.2 #[T,"t",ASTclassname].
        Parameters:
        str - The arguments to the AST constructor

      • getLookaheadTestExpression

        protected java.lang.String getLookaheadTestExpression​(Lookahead[] look,
                                                      int k)

      • getLookaheadTestExpression

        protected java.lang.String getLookaheadTestExpression​(Alternative alt,
                                                      int maxDepth)
        Generate a lookahead test expression for an alternate. This will be a series of tests joined by '&&' and enclosed by '()', the number of such tests being determined by the depth of the lookahead.

      • getLookaheadTestTerm

        protected java.lang.String getLookaheadTestTerm​(int k,
                                                BitSet p)
        Generate a depth==1 lookahead test expression given the BitSet. This may be one of: 1) a series of 'x==X||' tests 2) a range test using >= && <= where possible, 3) a bitset membership test for complex comparisons
        Parameters:
        k - The lookahead level
        p - The lookahead set for level k

      • getRangeExpression

        public java.lang.String getRangeExpression​(int k,
                                           int[] elems)
        Return an expression for testing a contiguous renage of elements
        Parameters:
        k - The lookahead level
        elems - The elements representing the set, usually from BitSet.toArray().
        Returns:
        String containing test expression.

      • lookaheadIsEmpty

        protected boolean lookaheadIsEmpty​(Alternative alt,
                                   int maxDepth)
        Is the lookahead for this alt empty?

      • mapTreeId

        public java.lang.String mapTreeId​(java.lang.String idParam,
                                  ActionTransInfo transInfo)
        Map an identifier to it's corresponding tree-node variable. This is context-sensitive, depending on the rule and alternative being generated
        Specified by:
        mapTreeId in class CodeGenerator
        Parameters:
        idParam - The identifier name to map
        Returns:
        The mapped id (which may be the same as the input), or null if the mapping is invalid due to duplicates

      • processActionForSpecialSymbols

        protected java.lang.String processActionForSpecialSymbols​(java.lang.String actionStr,
                                                          int line,
                                                          RuleBlock currentRule,
                                                          ActionTransInfo tInfo)
        Lexically process $var and tree-specifiers in the action. This will replace #id and #(...) with the appropriate function calls and/or variables etc...
        Specified by:
        processActionForSpecialSymbols in class CodeGenerator

      • isEmpty

        static boolean isEmpty​(java.lang.String s)

      • processActionCode

        protected java.lang.String processActionCode​(java.lang.String actionStr,
                                             int line)

      • printActionCode

        protected void printActionCode​(java.lang.String actionStr,
                               int line)

      • setupOutput

        public void setupOutput​(java.lang.String className)
                 throws java.io.IOException
        This method exists so a subclass, namely VAJCodeGenerator, can open the file in its own evil way. JavaCodeGenerator simply opens a text file...
        Throws:
        java.io.IOException

      • isspace

        protected boolean isspace​(char c)

      • _printAction

        protected void _printAction​(java.lang.String s)
        Description copied from class: CodeGenerator
        Print an action without leading tabs, attempting to preserve the current indentation level for multi-line actions Ignored if string is null.
        Overrides:
        _printAction in class CodeGenerator
        Parameters:
        s - The action string to output

      • od

        protected void od​(java.lang.String s,
                  int i,
                  int end,
                  java.lang.String msg)

      • printAction

        protected void printAction​(java.lang.String s)
        Description copied from class: CodeGenerator
        Print an action with leading tabs, attempting to preserve the current indentation level for multi-line actions Ignored if string is null.
        Overrides:
        printAction in class CodeGenerator
        Parameters:
        s - The action string to output

      • printGrammarAction

        protected void printGrammarAction​(Grammar grammar)

      • _printJavadoc

        protected void _printJavadoc​(java.lang.String s)

      • genJavadocComment

        protected void genJavadocComment​(Grammar g)

      • genJavadocComment

        protected void genJavadocComment​(RuleSymbol g)