> This is a parser which takes a sequence of ASToken objects as output and > creates an AST consisting of NodeBase objects.
Sorry for the typo... I meant "input" rather than "output". - Gordon Sent from my iPad > On Jul 23, 2014, at 9:06 AM, "Gordon Smith" <gsmit...@hotmail.com> wrote: > > CSSTree.g isn't involved in compiling AS files. It's involved only in > compiling CSS files. > > For AS compilation: We use JFlex to create RawASTokenizer.java from > RawASTokenizer.lex. This is a lexer which takes an AS file as input and > creates a sequence of ASToken objects as output. We use Antlr 3 to create > ASParser.java from ASParser.g. This is a parser which takes a sequence of > ASToken objects as output and creates an AST consisting of NodeBase objects. > This is a custom AST, not the kind of generic Antlr AST that Antlr can create > automatically. > > If Antlr 4 can tokenize as fast as JFlex can, then I think it would be great > to have a unified Antlr 4 lexer/parser. However, I strongly recommend keeping > Falcon's custom AST classes. > > I'm not an expert on JBurg and wasn't aware that it had a dependency on > Antlr. Can you please show me where this depencency is? The "JBurg guys" are > basically one person, Tom Harwood, who used to work at Adobe but no longer > does. I don't know whether he is still supporting JBurg or not. > > As for eliminating JBurg... This is certainly doable, but would have serious > implications. The main reason that Falcon does code generation using JBurg is > in order to perform various code optimizations such as constant propagation. > These code optimizations would be much harder to do without JBurg. Ideally, > these optimizations would be done in the ActionScript Virtual Machine rather > than in the compiler (and this was the plan for AS4, which Adobe killed) but > since they are NOT done in the current AVM, if the compiler doesn't do them > then the code will execute slower. However, if cross-compilation to > JavaScript is the main goal for the future, then perhaps slower AVM execution > is tolerable. > > - Gordon > > >> From: christofer.d...@c-ware.de >> To: dev@flex.apache.org >> Subject: AW: AW: AW: AW: Falcon and Antlr4 >> Date: Wed, 23 Jul 2014 12:54:15 +0000 >> >> Hi Eric, >> >> The question is definitely not silly. >> >> I'm not planning on changing anything here. The package tangling seems to be >> related with JBurg needing access to >> some objects that are not part of the interface but part of the internal >> model. I would untangle this by extending the >> Interface to provide anything JBurg needs so there shouldn't be any change >> in the Falcon AST. But after all that's just >> an optimization. Currently I'm digging in the inner workings of JBurg ... >> gee I sort of forgot why I looked in the inner >> workings of Falcon in the first place ;-) >> >> I would however like to remove as many processing phases and model >> conversions as possible as each one costs time >> and memory and makes things more compilcated to understand. >> >> Chris >> >> ________________________________________ >> Von: Erik de Bruin <e...@ixsoftware.nl> >> Gesendet: Mittwoch, 23. Juli 2014 14:47 >> An: dev@flex.apache.org >> Betreff: Re: AW: AW: AW: Falcon and Antlr4 >> >> I'm not at all familiar with the inner workings of Falcon, but I know that >> FalconJX relies heavily on the current Falcon AST. Will the new AST be >> compatible with the old (that may be a silly question, but it's the only >> one I know to ask)? >> >> EdB >> >> >> >> On Wed, Jul 23, 2014 at 2:36 PM, Christofer Dutz <christofer.d...@c-ware.de> >> wrote: >> >>> Ok ... so I used some idle time setting up a clean falcon-antlr4 module >>> starting with the CSS parsing. >>> >>> As far as I can see it, Falcon is currently using Antlr3 to generate the >>> lexer and a parser that generates an Antlr AST from that >>> Then we generate a second parser CSSTree.g to convert that AST into the >>> falcon-internal object structure, which is then >>> passed to JBurg to generate the output. >>> >>> As you mentioned the ability to manipulate the AST. Are you talking about >>> the first (antlr) AST or the one the Falcon AST? >>> I would like to elminiate the second parsing step and generate the >>> internal model directly from the first parser using >>> Antlr4s Listeners. Are there any objections against this? I doubt that >>> anyone is manipulating the Antlr AST directly, much >>> more would I expect operations to be performed on the Falcon AST. >>> >>> I managed to get the CSS Lexer and Parser finished. The grammar is way >>> more readable now and the general structure is >>> much more straight forward :-) Now I would like to Build the internal >>> Falcon object model for that. >>> Guess I would have to contact the JBurg guys for continuing from then as I >>> definitely need Antlr4 support but I would >>> bet that either JBurg is dead or they are working or at least thinking >>> about that. >>> >>> I did detect some ugly cyclic dependencies in the falcon classes though >>> ... for example we have interfaces for defining the >>> internal models interface and have implementations of these interfaces in >>> the internal packages, but unfortunately the >>> interfaces reference classes in the internal model ... I think that's bad >>> design and would like to clean that up. >>> >>> Any objections? (Keep in mind ... it's all in a dedicated branch, but I >>> don't want to make it hard to merge back as soon as I'm finished) >>> >>> There is no DOM or SAX involved in Falcon ... I was using a metaphor to >>> explain what I was thinking about by using >>> an example from the XML processing world. Here you can load an XML >>> document to memory (DOM), transform that into >>> a second memory representation and so on in contrast to SAX processing >>> where parser generates a stream of events which >>> can be processed allowing you to process petabyte big documents with as >>> much ram as a calculator ;-) >>> >>> >>> Chris >>> >>> >>> ________________________________________ >>> Von: Alex Harui <aha...@adobe.com> >>> Gesendet: Mittwoch, 23. Juli 2014 08:48 >>> An: dev@flex.apache.org >>> Betreff: Re: AW: AW: AW: Falcon and Antlr4 >>> >>>> On 7/22/14 12:07 PM, "Christofer Dutz" <christofer.d...@c-ware.de> wrote: >>>> >>>> The way I understood JBurg is that it seems to have been designed to >>>> manipulate ASTs produced by Antlr2 and Antlr3. >>>> Therefore it has a compile and runtime dependency to Antlr (I also sort >>>> of dislike them bundling the generator together with the runtime stuff) >>>> but nevertheless ... I can't really use JBurg with Antlr4 in my project >>>> not without splitting the module up but I think that's rather ugly. >>> I'm definitely not an expert on the subject. I'm surprised the AST has >>> any Antlr-isms in it. Maybe Gordon can confirm. When I look at the AST, >>> I don't see anything I find surprising. >>> >>>> >>>> I too was thinking about making JBurg obsolete too ... the way I >>>> understand Antlr4 it looked as if it provided a lot which could make life >>>> a lot easier. For example currently the ASParser creates the AST objects >>>> and JBurg then performs it's magic on that AST tree. In Antlr4 I could >>>> create a Parser-Listener that produces the AST object (Probably needed >>>> for IDE suppoer) but could probably create a second listener that >>>> directly outputs flash bytecode. I think this approach should be maximum >>>> performant, maximum simple and use only a fragment of the memory the >>>> current solutions need ... Sort of XML Dom processing (old AST parsing) >>>> and SAX (Antlr4 direct Bytecode output). >>> I have not played with SAX much nor Antlr, so I have no idea about >>> parser/listener. I will say that there are times folks want to create the >>> AST and then manipulate it before generating the output, so unless there >>> is a huge performance win, making sure there is a way to do that would be >>> preferred. >>> >>>> >>>> But I also think that this would be quite an effort. Perhaps I should >>>> split up my falcon-antlr4 branch even more and sort of start with the CSS >>>> parser and as soon as that's up and running get the output running ... >>>> then we could compare the performance of both and see if it's worth going >>>> down that path. Don't want to waste time I could be working on Flexmojos, >>>> or the new Flex-Maven-Plugin for a less performant solution. But if it >>>> was faster (I would expect it to be) it could be quite breakthough ... >>>> after all I have never seen several datastructure conversions be faster >>>> than direct output. >>> Up to you since you are doing the work. I just want to make sure you >>> don't feel like you have to keep all of these subsystems like Jburg and >>> Jflex around. >>> >>> -Alex >> >> >> -- >> Ix Multimedia Software >> >> Jan Luykenstraat 27 >> 3521 VB Utrecht >> >> T. 06-51952295 >> I. www.ixsoftware.nl >
