> Does the ANTLR parser use the standard token stream if it's not lexing the > source?
For AS, where we use JFlex to do the lexing (for better performance), we implement our own token stream. For CSS, where ANTLR is doing the lexing, we use its standard token stream. > Would there be a way to create a linked list token stream with what the lexer > creates? We don't buffer visited tokens for AS, but it would be straightforward to implement your own linked list token stream. > Is there any plan on adding FXG grammar and DOM? I forgot to mention that Falcon does support compiling FXG. But we didn't innovate much in this area... we mainly just brought over the FXG code from the old compiler, which you can take a look at. I believe it uses a SAX parser to parse the XML of the FXG file, and it builds a custom DOM with classes like RectNode and TextNode which it then uses for semantic analysis and code generation. One improvement is that the FXG compilation takes place in the same multithreaded fashion as other Falcon compilation. - Gordon -----Original Message----- From: Michael Schmalle [mailto:m...@teotigraphix.com] Sent: Thursday, January 19, 2012 4:48 PM To: flex-dev@incubator.apache.org Subject: RE: Falcon compiler source code / Falcon architecture Gordon, when you have time; Does the ANTLR parser use the standard token stream if it's not lexing the source? Would there be a way to create a linked list token stream with what the lexer creates? Is there any plan on adding FXG grammar and DOM (this would be awesome but maybe something the community could implement)? Thanks, Mike Quoting Gordon Smith <gosm...@adobe.com>: > ANTLR is used to parse (but not lex) AS, and to both lex and parse CSS. > > - Gordon. > > -----Original Message----- > From: Michael Schmalle [mailto:m...@teotigraphix.com] > Sent: Thursday, January 19, 2012 11:16 AM > To: flex-dev@incubator.apache.org > Subject: RE: Falcon compiler source code / Falcon architecture > > Gordon this is about the best news we could get. > > It's great to see this structure! I can't wait. > > What parts are using ANTLR grammar? > > Thanks, > Mike > > Quoting Gordon Smith <gosm...@adobe.com>: > >> Until we release specs, Javadoc, or code for Falcon, here is a very >> brief overview of Falcon's architecture: >> >> Falcon is designed from the beginning to support compiling multiple >> targets in multiple projects in a workspace (as you have in an IDE >> like Flash Builder). By contrast, asc was designed to compile a >> single AS file, and then mxmlc was built on top of that. >> >> Falcon uses multiple threads to compile multiple files at the same >> time. The more cores you have, the faster it goes. >> >> Critical data structures such as the symbol table (which stores >> information about which classes are known, what methods they have, >> etc.) are shared across the entire workspace, to minimize memory >> usage. >> >> Critical data structures are maintained in memory to support both >> compilation and IDS code intelligence in an efficient and consistent >> way. For example: If you open a file in the IDE, Falcon builds a >> syntax tree and symbol table for it to support intelligent editing. >> Compiling the file requires just one additional code generation step. >> By contrast, in the current Flash Builder which uses the compiler in >> the SDK, Flash Builder builds its own parse trees and symbol tables >> to support editing, and then when you compile the compiler in the SDK >> builds another set of parse trees and symbol tables. This is slow and >> a waste of memory. >> >> Falcon understands .as, .mxml, .css, and .properties files. >> >> The parse trees for AS consist of nodes from about 100 classes, such >> as LiteralNode, BinaryOperatorNode, FunctionNode, ClassNode. The >> parse trees for MXML consist of nodes from about 50 classes, such as >> MXMLDocumentNode, MXMLInstanceNode, MXMLScriptNode, etc. The >> ActionScript-y parts of MXML are represented by AS nodes inside of >> MXML nodes. >> >> The symbol table consists of objects representing the things your >> code defines, such as ClassDefiniition, FunctionDefinition, and >> VariableDefinition, arranged into a hierarchy of scope objects. >> >> Falcon uses 3rd-party grammars like JFlex, ANTLR, and JBurg to >> generate various lexers, parsers, and code generators. >> >> MXML is compiled directly to ABC, not to ActionScript source code or >> an ActionScript parse tree. >> >> - Gordon Smith, Adobe >> >> P.S. After working on the Flex framework since its inception for >> about >> 8 years, I joined the Falcon team 18 months ago. >> >> >> >> -----Original Message----- >> From: Raju Bitter [mailto:rajubit...@googlemail.com] >> Sent: Thursday, January 19, 2012 2:10 AM >> To: flex-dev@incubator.apache.org >> Subject: Re: Falcon compiler source code / Falcon architecture >> >> 2012/1/19 Alex Harui <aha...@adobe.com>: >>> I was in the Falcon code in November. It was way easier to figure >>> out how to make changes than with MXMLC. Even if it is not >>> documented, I think it will be easier to make progress that with the >>> Falcon code base. >> I agree, that sounds like it would be much better to use the Falcon >> code base. There are normally fewer community members with much >> experience in language/compiler design, and the cleaner the >> architecture is, the better for the community. >> > > >
