Hello, I have prepared a first draft of 'dynamic SLOT' specification. This is my proposal in attempt to solve the problem of building packages for multiple Python and Ruby versions. It could also be reused for multilib.
The spec tries to explain the broad idea, and all problems relevant to it. It also lists a few problems which are still unsolved and I think they will cause the spec to change after hearing your ideas. To be honest, I tried to keep it as simple as possible. Please don't say it doesn't solve all the world problems because it simply won't. I'm attaching a reStructuredText version of the spec. You can view it rendered as a gist[1]. But please keep the replies on the list, rather than forking the gist. [1]:https://gist.github.com/2943774 -- Best regards, Michał Górny
=============
Dynamic SLOTs
=============
1. Motivation
-------------
There is currently no good way to handle building and installing
ebuilds for multi-ABI languages like Perl, Python or Ruby.
The Perl modules are bound to a particular Perl version. Since we
support having only one Perl version around, this issue will supposedly
be solved via ABI_SLOT solution which is not part of this spec.
However, we support having multiple Python & Ruby versions installed
(via SLOTs). Therefore, we need to have a good solution to support
having multiple versions of modules, each one built against respective
ABI version.
Currently, this problem is worked-around through USE flags. This
solution is incomplete and has a few drawbacks including:
- necessity of rebuilding *all* package versions when the list
of enabled ABIs changes,
- dynamically changing IUSE with new Python/Ruby versions being
released (supported).
The dynamic SLOT solution is supposed to solve these problems through
providing a way to:
- build multiple package variants (dynamic SLOTs) from the same ebuild,
- have those variants installed in parallel,
- automatically handle cross-package dynamic SLOT dependencies.
2. Definitions
--------------
dynamic SLOT
A dynamically assigned, additional SLOT specification for a package,
representing ABIs for which it was built. Dynamic SLOT is defined
at build-time. A single package may have more than one dynamic SLOT
defined.
dynamic SLOT variant of a package
A single package built against chosen dynamic SLOT specification.
3. Defining supported SLOTs in an ebuild
----------------------------------------
An ebuild supporting building for multiple dynamic SLOTs has to declare
the supported slots using ``DYNAMIC_SLOTS`` variable. The variable can
be inherited from eclasses. The one-of (``|| ( )``) group can be used
in this variable to specify exclusive SLOT groups.
For example, an ebuild supporting building for multiple Python ABIs
would declare (either explicitly or implicitly through an eclass)::
DYNAMIC_SLOTS='|| ( py2.6 py2.7 py3.1 py3.2 )'
which would mean that when the package is built, one of the ``pyX.Y``
SLOTs must be used (and only one).
An ebuild may also declare multiple dynamic SLOTs to be specified
at a time::
DYNAMIC_SLOTS='|| ( py2.6 py2.7 ) || ( lib32 lib64 )'
In this case, both one of ``pyX.Y`` and ``libX`` SLOTs need to be
declared.
4. Building the ebuild against chosen SLOTs
-------------------------------------------
In ``pkg_*`` and ``src_*`` phases, the build environment is provided
with currently enabled dynamic SLOTs via ``CURRENT_SLOTS`` array.
The ebuild must use the contents of this variable to adjust the build
process accordingly which can be done either directly or via an eclass.
If multiple ABIs were chosen, they are propagated onto next indices
of the array.
5. Relevance to binary and installed packages
---------------------------------------------
It is necessary to store the dynamic SLOTs for which a package was built
in the binary package and installed package metadata. The exact
semantics are left to be implementation-specific. The implementation
must ensure, however, that a multiple dynamic SLOT variants of the same
package can be installed at the same time.
For example, dynamic SLOT could be appended after package version::
dev-python/foo-1.2.3+py2.6
dev-libs/bar-1.0+lib32
6. Installed file collissions
-----------------------------
Due to the specifics of dynamic SLOT implementation, it is possible that
files installed by various dynamic SLOTs collide. For example, each
dynamic SLOT variant of a Python module may install a README file
in the same docdir.
The ebuilds must ensure that the colliding files are exactly the same
in all dynamic SLOT variants of the package. The package manager may
either choose to overwrite those files when installing each consecutive
dynamic ABI variant of a package, or leave the earlier version.
The package must not remove the file unless all of dynamic SLOT variants
of the package were unmerged.
7. Inter-package dependencies
-----------------------------
If an ebuild supporting dynamic SLOTs depends on another ebuild doing
so, the package manager must implicitly depend on matching dynamic SLOT
variant of the dependant package.
In other words, if ``dev-python/foo`` depends on ``dev-python/bar``
and both of those ebuilds support dynamic SLOTs for Python ABIs, then
the package manager must ensure that building ``dev-python/foo`` with
``py2.7`` dynamic SLOT will pull in ``dev-python/bar`` with ``py2.7``
SLOT.
8. User interface
-----------------
The choice of dynamic SLOTs for ebuilds whenever not directly caused
by implicit dependencies is to be implementation-defined.
The package manager should provide an ability to control the default
dynamic SLOT choice to external tools like eselect. For example,
the default Python SLOT would be controlled by eselect-python.
The package manager must provide user with an ability to explicitly
choose dynamic SLOT when merging a package and to store such a SLOT
specification in the world file.
The package manager may support specifying more than a single exclusive
dynamic SLOT variants. In this case, the package manager should split
the request into merging multiple dynamic SLOT variants of the package.
For example, the following call::
emerge dev-python/foo+py2.6+py2.7
may cause the package manager to either reject the request (because of
colliding dynamic SLOTs) or build two dynamic SLOT variants
of the package.
9. Unsolved problems
--------------------
a) Dependencies on package versions with single unsupported dyn. SLOTs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Example: some Python packages support Python 3 while older versions
of their dependencies don't. With the current text of the spec, there's
no way for package manager to ensure that newer versions will be pulled
in due to that.
The specification doesn't distinguish between particular kind of dynamic
SLOTs being unsupported (i.e. dependency which is not a Python package)
and a particular dynamic SLOT being unsupported (dependency
not supporting a particular Python version).
This is probably to be fixed via introducing global definitions of
supported *dynamic SLOT groups* (in ``profiles/``). Then, the packages
would be enforced a particular SLOT when it supports at least one SLOT
from the group.
b) Large common parts of packages supporting dyn. SLOTs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Often packages supporting dynamic SLOTs have large, common parts.
In most common case, it is API documentation which gets generated
at build time. Supporting multiple dynamic SLOTs would mean that
documentation is going to be regenerated for every SLOT (or at least
SLOTs supporting documentation generation).
This get worse when various versions of the SLOT introduce tiny,
irrelevant differences in the generated files. As the spec does disallow
installing colliding files with different contents, this has to be
handled some other way.
Possible solutions:
1. Let each dynamic SLOT install own copy of files (i.e. separate docs
for each version),
2. Move the common files into separate ebuild,
3. Introduce a special dynamic SLOT which would install just the common
files, and depend on it (requires adding support for explicit dynamic
SLOT dependencies).
c) Packages installing multiple kinds of dynamic SLOTs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Example: packages installing bindings for multiple languages. The usual
way of defining dynamic SLOTs, i.e.::
DYNAMIC_SLOTS='|| ( py2.6 py2.7 ) || ( ruby1.8 ruby1.9 )'
would mean that each time both a Python version, and a Ruby version has
to be chosen. This would have to be backed up at least with USE flags
and it gets more ugly...
An interesting solution would be to use a global one-of block like::
DYNAMIC_SLOTS='|| ( py2.6 py2.7 ruby1.8 ruby1.9 )'
which would mean that each build involves building just one
of the bindings for a particular language of interest. Sadly, this will
probably make the eclasses more complex.
Another solution would be to simply split the package.
.. vim:ft=rst:tw=72:noet
signature.asc
Description: PGP signature
