Hi Le-chun and Taras, You are right that instead of categorizing the plugin API into "production", "research", etc, we can just introduce several layers of abstraction that will be useful for different purposes such as quick prototyping or tightly coupled plugins, etc. We will update the Wiki to reflect that and avoid misunderstandings ...
Zbigniew and I are preparing a patch that will include a higher abstraction layer that will be relying on the lower-level abstraction you documented (we just need to clean it now and we should be able to do it by the end of this week). We also provide high-level routines for pass manipulation since it's also critical for us and our current users - we can already add new code analysis/instrumentation passes or select/de-select/reorder existing passes easily - we will just need to synchronize on that implementation altogether to avoid duplicate work ... Will keep in touch, Grigori > -----Original Message----- > From: Le-Chun Wu [mailto:l...@google.com] > Sent: Wednesday, February 18, 2009 1:35 AM > To: Grigori Fursin > Cc: Basile STARYNKEVITCH; Diego Novillo; gcc@gcc.gnu.org; Sean Callanan; > Taras Glek; Gerald > Pfeifer; Zbigniew Chamski; Cupertino Miranda > Subject: Re: [plugins] Comparison of plugin mechanisms > > Hi Grigori and Zbigniew, > > I just took a look at the wiki page you guys put together. While I > have some reservations about categorizing the plugin APIs into > "production", "research", and "new pass", I totally agreed with you > that the plugin support should be implemented in layers, starting from > the core support that provides basic APIs and exposes all the internal > GCC data structures, to more complicated support that allows more > abstract APIs and maybe drastic changes in GCC compilation pipeline. > > I agree with Basile that the support for pass management should not be > in a different category. I think allowing plugin modules to hook new > passes into GCC is an important requirement for plugins to be really > useful, so we should provide basic pass management support (such as > adding a new pass or replacing an existing pass) even in the core (or > "production") APIs. And in the advanced/extended (or "research") APIs, > we can allow the whole pass manager to be replaced (probably like what > you did in ICI). In our plugin prototype, we have implemented basic > support for adding new passes and defined APIs that allow the plugin > writers to specify where/when to hook in the new passes. I will modify > the GCC Plugin API wiki with our proposal so that people can comment > on it. > > You mentioned that you will be sending out a patch this week. Will > your patch contains the implementation for both the "production" and > "research" APIs? If your patch contains only the research APIs, we can > prepare an official patch for the "production" APIs (based on the > patch that I send out a while back ago) and send it out for review > this week (if no one else would like to do it, that is). > > Thanks, > > Le-chun > > > On Fri, Feb 13, 2009 at 1:26 AM, Grigori Fursin <grigori.fur...@inria.fr> > wrote: > > Hi Basile et al, > > > > Thanks a lot for detailed explanations. > > > > In addition to Zbigniew's reply: > > I didn't specifically put MELT into new category (I actually just > > updated the page and added it to the other categories too), > > but I still keep the new pass integration plugin, since > > some of our colleagues would like to add new passes including > > new optimizations, but have difficulties to do that since there > > is no full documented API, they may not want to write it in C > > (as you mentioned) and they are concerned about support overheads > > if the internal API will be changing all the time. Providing > > standard API would be of great help for such cases, but it rises > > other issues such as opening GCC to proprietary plugins, etc > > which has been heavily discussed here for some time. So I personally > > think we should leave it for the future and just implement minimalistic > > Plugin API that will already make many current and prospective users > > happy and then we can gradually extend it ... > > > > By the way, Sean, what do you think about the proposed APIs? > > Will it suit your purposes or some changes are also required? > > I would like to add info about your plugin system but just > > don't have much knowledge about that ... > > > > Cheers, > > Grigori > > > >> -----Original Message----- > >> From: Basile STARYNKEVITCH [mailto:bas...@starynkevitch.net] > >> Sent: Friday, February 13, 2009 8:38 AM > >> To: Grigori Fursin > >> Cc: 'Diego Novillo'; gcc@gcc.gnu.org; 'Sean Callanan'; 'Taras Glek'; > >> 'Le-Chun Wu'; 'Gerald > >> Pfeifer'; 'Zbigniew Chamski'; 'Cupertino Miranda' > >> Subject: Re: [plugins] Comparison of plugin mechanisms > >> > >> Hello All, > >> > >> > >> Grigori Fursin wrote: > >> > Basically, we currently see 3 complementary categories of GCC plugins, > >> > depending > >> > on the nature of the extension: production, experimentation/research, > >> > and new pass > >> > integration. Each category naturally calls for slightly different API > >> > features. > >> > > >> > > >> I am not sure of the relevance of the "new pass integration plugins" > >> examplified by MELT. > >> [on the other hand, I do know Grigori and I believe he thought quite a > >> lot about plugins, which I didn't. I only implemented one particular > >> plugin machinery = MELT, knowing well that my approach is quite peculiar > >> both in its goals and its implementation. I never thought of MELT as a > >> universal plugin machinery). > >> > >> In my view, MELT fits quite precisely in the "production plugins" > >> definition, while indeed I expect it to be useful mostly for > >> "experimental/research" plugins. > >> > >> In my view also, the "new pass integration plugin" category should not > >> really exist, because it probably can fit inside one (or both) of the > >> above categories. > >> > >> MELT definitely claims to fit into the "production plugins" slot, > >> because MELT always was concerned by efficiency and most importantly > >> close integration of GCC internal structures. The major point of MELT > >> is its several idioms to fit into the *evolving* GCC internals API, > >> and I claim that the various MELT idioms (see my GROW paper) make > >> close integration into GCC internals possible, and perhaps even easy > >> (for each internal "feature" of GCC, it is really easy to code the > >> couple of MELT line to use it). > >> > >> Of course, MELT is mostly motivated by "experimental/research plugins", > >> in the sense that MELT will be mostly useful for experimental and > >> prototyping. I never thought that MELT would be useful for coding > >> definitive optimisation passes, but it should be useful to at least > >> prototype them. Actually, I really believe that for ordinary > >> optimisation, any plugin machinery is not used. In other words, I tend > >> to think that even in -O3 (or a future -O4) no plugin will be dlopen-ed > >> by default in GCC. [by the way, I believe that this last fact is > >> unfortunate; I would like plugins to be routinely used inside GCC, but I > >> do know that most of the GCC community disagree.] > >> > >> Also, I don't understand why "production plugins" or > >> "experimental/research plugins" could not be coded in another language > >> than C. For sure, they could probably be coded in a suitable subdialect > >> of C++ at least. > >> > >> I do like Grigori's plugin API proposal. (but again, I definitely do not > >> claim to be a plugin theorist, only a particular plugin implementor, > >> with MELT having specific needs & solutions.). > >> > >> I did not understood yet how exactly can Grigori's production plugin API > >> be used to add e.g. one plugin pass inside the pass manager. Eg how to > >> add one pass fooplugin_pass provided by a plugin just after the > >> pass_inline_parameters, or another plugin pass barplugin_pass just after > >> pass_ipa_struct_reorg? > >> I assume such things could be possible... > >> > >> There is another issue which has not been discussed enough in my view. > >> The interaction between some GCC plugins and the GGC memory > >> manager/garbage collector. More precisely: > >> > >> 1. How can a plugin fooplugin.c use GTY() notation, and therefore > >> generate a gt-fooplugin.h which is #include-d at the end of fooplugin.c? > >> conceptually it could be quite easy: extend gengtype so that, in > >> addition to it peculiar current use [invocation from the Makefile > >> without program arguments] it is able to take two program arguments, the > >> input file -here fooplugin.c- and the output file -here gt-fooplugin.h > >> > >> 2. Even more important, how can a plugin register static or global GTY > >> roots and their marking routines (generated inside gt-fooplugin.h > >> above)? The point is that after a plugin has been loaded, the > >> ggc_collect routine should invoke some marking routines defined by the > >> plugin! > >> A simplistic very low level approach whould be to very simply extend > >> ggc_collect to take an additional marking routine & data. I did provide > >> a very small patch for that > >> http://gcc.gnu.org/ml/gcc-patches/2009-01/msg00431.html but very sadly > >> it has been rejected. Has it been accepted, one could easily e.g. add a > >> root registration in the plugin machinery and have all the dynamic roots > >> be marked with a very simple wrapper above the proposed > >> ggc_collect_extra_marking. > >> I still hope that some dynamic roots (and dynamic marking routines) > >> will be possible one day (otherwise, in my opinion, the whole plugin > >> effort would be doomed). > >> > >> Regarding MELT, I believe that if a plugin facility has, in addition of > >> Gregori's feature, the ability to insert new dynamic passes and the > >> ability to add some extra GTY root & marking routine, I could fit MELT > >> into that. If that happens, MELT would become somehow a "metaplugin" > >> machinery, in the sense of a plugin which will itself load some dynamic > >> code. > >> > >> In Grigori's table > >> > >> > >> Side-by-side comparison of production and research/experimental plugin > >> APIs > >> > >> > >> I believe that MELT probably don't need an extra column, and that for > >> the purpose, impact, implementation,aspects it fit into the first column > >> "production plugins" while for most other aspects MELT fits into the > >> second column. > >> > >> > >> Regards. > >> > >> PS. In all this email, dynamic means simply obtained thru dlsym. > >> > >> > >> > >> -- > >> Basile STARYNKEVITCH http://starynkevitch.net/Basile/ > >> email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359 > >> 8, rue de la Faiencerie, 92340 Bourg La Reine, France > >> *** opinions {are only mines, sont seulement les miennes} *** > > > >
