I have notes inline below, following is my summary of libplugin from what i understand of your posts: * It exists as a fraemwork that works with GCC now * It uses xml files to define plugins (Allows making new plugins as combinations of others without making a new shared library, i.e. just create an xml file that describes the plugin) * It handles issues with inter-dependencies between plugins * It uses a "push" framework, where function pointers are replaced/chained in the original application rather than explicit calls to plugins (Provides more extensibility in a application that makes heavy use of function pointers, but produces a less explicit set of entry points or hooks for plugins) * Currently it provides automatic loading of plugins without specific user request * It already has a framework for allowing plugins to interact with the pass manager
If you can think of any other points to summarize the features it might be helpful as you are closer to it. The issues i see with this framework: * it seems to provide a lot of features that we may not necessarily need (That should be up for discussion) * plugin entry points are not well defined but can be "any function pointer call" Some questions: * How does the framework interact with the compile command line arguments? * Does this work on platforms that dont support -rdynamic or can it be modified to do so in the future? Hugh Leather wrote: > *Separating Plugin system from appliction* > Libplugin ships as a library. Apart from a few lines of code in > toplev.c, the only other changes to GCC will be refactorings and > maybe calling a few functions through pointers. As i understand the difference between the pull vs push, a plugin will load, and then modify existing function pointers in GCC to insert its own code and chain the existing code to be called after it. Is this correct? Doing this will be able to make use of existing function pointers as plugin hook locations, but some hooks we may want are not already called by function pointers and so would need to be changed. This means that plugin hook locations are not explicitly defined, but rather any place where a function pointer is used can be modified. Personally i prefer explicit specification of plugin hook locations. > I think it's important to separate the plugin system from the > application. Doing plugins well, IMO, requires a lot of code. It > shouldn't be spread through the app. It also cleanly separates > plugin mechanism from the actual extensions the app wants. > Finally, plugins have to be extensible too. They should really be on > a nearly equal footing with the app. Otherwise plugin developers > who want the plugins to be extensible will need to reimplement there > own extensibility system. Without the use of plugin meta-data in XML files and auto-loading and many of the things discussed, i am not so sure that plugins will be such a large body of code. It is really a matter of deciding if such features that libplugin provides are desirable for GCC. If so, then there is a lot of code required for plugins and libplugin becomes a good idea IMO. If not, then libplugin may just be more than we need. It really depends on what "doing plugins well" means for the specific application. > *Scalable and Granularity* > The system is very scalable. Really this is due to the push > architecture. The granularity as i understand it is only as fine/coarse as the number of function pointers in the system that can be overwritten. This is no different from the pull method (i.e. The granularity depends on where you put the hook locations) except that function pointers "may already exist". Though i may have mis-understood something... I.e. For the "pull" method you can: Add a "pull" for firePluginEvent() or add a "pull" inside each existing event handler. Where as the push method requires that the existing event handlers are called via function pointers and the "push" chains itself to that. I have used a similar method for the "push" plugin in python. The advantage here is that basically "anything" can be pushed in python so the system becomes very flexible to extend via "plugins". In C/C++ the areas that can be extended need to be defined and turned into function pointers for the push method to work. Again, assuming i have understood how it works. > *Mutliple cooperating plugins > *I think some of the proposals don't allow multiple plugins or > plugins aren't able to be extended in the same way that the > application is. In libplugin you can have lots of plugins all > depending on each other. Plugins can provide extension points as > well as the application - this means it isn't just a matter of the > application deciding what's important and everyone else having to > make do. > > In some senses, this is the difference between a plugin system and > loading a few shared libraries. A plugin system provides an > holistic framework for building and using plugins. One thing this has brought up, is plugin inter-dependencies. I am not certain that we need to be able to have plugins for plugins, however some plugins may require others to also be present to operate. My first thought is that such dependencies would best be avoided but is that an acceptable restriction? I.e. If pluginfoo requires pluginblah to have been loaded previously to work. Do we automatically load pluginblah first or do we expect the user to specify: -fplugin=blah -fplugin=foo? The inter-dependency issue between plugins is something we need to decide on. I.e. Is it necessary, and if so what mechanism to use? > * > Separation of plugins from shared libraries* > If a plugin is only a shared library, then you have to load that > library to find out anything about it. Not exactly. I would propose to embed certain symbols or data into the shared library to provide the necessary meta-data. You can use libbfd to extract such data from a shared library without having to load it. This could be as simple as does the plugin export a symbol: gcc_version_4_3_2. As far as i remember the 3D engine "Crystal Space" used a combination of the two methods. I.e. It had an XML file that contained meta-data about the plugin, on platforms that supported it this data was embedded into the binaries. On others it was a separate file. What we really need to do is determine what sorts of meta-data we need embedded into the plugin and after identifying that is an XML file good for this or is it overkill. For example: plugin-name plugin-version application the plugin is for application version it works with But if we require lots of other plugin meta-data, it may not be suitable to use a simple mechanism such as requiring certain exported symbols and an XML file might be a good idea. > The other problem with plugins being only shared libraries is that > you always have to write C code. What if you just want something > really simple? Do you always have to go to the effort of writing a > shared library, compiling it, putting it in the right place, and so > on? That again is another feature we need to decide if it is desirable? > Libplugin, though, makes every plugin an XML file, not a shared > library. The plugin might use a shared library but it might not. > It can also use more than one. That is an interesting concept. > > *Passes* > Here's a quick description about what happens with passes. I've split > the discussion in two. The first, short part describes what will go > into the next release, in mid-October(ish). The next part is working > but I'm not happy with it yet, so it will wait. Interesting. So far, we have not looked in detail at how to work with the pass manager. I only briefly read this but it sounds nice. > *Licensing* > > I don't know anything about licensing, but we could do something > similar to the approach that Joern suggested. We could only load > plugins that included the GPL or other approved OSS lisence at the > top of the file. The plugin would then declare that it and > everything it used was good. I don't think people could avoid that > declaration. Maybe I'm wrong. In previous discussions (See wiki page on GCC plugins) the main issue was not that the plugins themselves were GPL or not. As i understand it they will have to also be GPL'ed under the existing GPL (They will need to link with a GCC internals code to work). The issue was rather GPL plugins that exported the AST in some proprietary format to a file and then writing a proprietary program that loads that data and does something with it. I.e. GCC doing the hard work of generating the tree and then the proprietary application doing something with that data. With this in mind, technical measures such as requiring a plugin to return a specific licence text string etc may not be necessary. If they are, then we can worry about them when we get more information. Anyhow, this is not a list for discussing legal stuff so i will be quiet after this. Everything that people are saying again in other threads has previously been said on this list and will not produce anything productive until we get more information. Just check the wiki page or archives for the histroy on this debate if you are more interested.
