On 11/20/2012 12:18 PM, Hugo Trippaers wrote:
Hey all,
Packaging is a work in progress at the moment. Wido, me and others are working
on this, but we are not there yet. Partially this is because there are a
multitude of things to consider when we discuss packaging. Hence this mail to
share a lot of the thoughts that went into packaging.
First of all is how we look at the packages. With the ASF it is pretty clear that the "release" is
the source code. We tag a particular state of the source tree and voila, we have a release. So
"packaging" our "release" is a simple as making at tarball or zip of the code and making
it available to users (aside from the ASF process to name something a release).
Compiling the code, as part of our build procedure generates artifacts (using
the mvn package target). Artifacts in this sense are jar, war and zip files
containing a mix of compiled java classes, scripts, documentation (and
optionally dependencies). Most of these artifacts are already automagically
sent to the apache snapshot repositories by jenkins because we inherit the
maven configuration from the apache master (or as a final release if we wanted
to).
Finally "packaging" is taking the artifacts generated by the compile/build step
and turning it into some kind of OS specific package like an RPM or DEB.
The current build system based around maven is designed for this way of
working. Maven is only taking responsibility for turning the source code into
artifacts and additional scripts can take those artifacts and combine them into
packages. This is explicitly done this way to not clutter a generic and
multiplatform java build process with very os specific packaging steps. The
packaging step should be something like, download sources as tarball, extract,
run mvn package (with any additional profiles you need), generate package from
the artifacts. In the packaging directory there is a centos63 spec file which
does this (see %build and %install sections)
Currently we have released the 4.0.0 version as a source release and some
members have graciously offered to compile, build and package the source and
make that available to the community for their convenience. I think the
explicit wish is that we are able to provide a distribution ourselves (meaning
the Apache CloudStack community) instead of just providing source code. This
brings us back to the original discussion regarding licensing and how to deal
with dependencies in packages.
Wido and myself had a lengthy discussion on this subject at ApacheConEU and we
haven't reached a conclusion yet, but a good enough understanding of the
problem that we can bring our ideas and discussions to the list.
One of the main arguments is how to deal with dependencies in packages. There
are a couple of options here:
* Create self-contained artifacts in step 2 (compile/build) that
contain all required dependencies. Packages made from these artifacts need no
outside dependencies save the bare essentials like java runtime, tomcat and
python.
* Create "bare" artifacts with just our compiled code in step 2
(compile/build). Packages need to include all dependencies for all jars that are
generated as part of our code.
* Hybrid mix and match of the two options above.
* The old waf build is even worse. We compile bare artifacts in step 2
(build/compile) and then package downloaded dependencies as far as the ASF
permits and let the package dependencies deal with the others (like
mysql-connector-java).
In my view one of the big issues here is version management of our dependencies.
Managing dependencies can be conceived as a two pronged effort. However,
it can also be boiled down to letting packagers deal with it and thus it
can be externalized/delegated.
One can look at dependency management from a developers or users point
of view and can decide that the project should provide, in some way, the
dependencies to potential developers or users. This is reasonable,
however it leads to the point where packagers for distributions manage
dependencies and the project manages dependencies. Thus, effort is
duplicated across the open source community and people's time is wasted.
Considering that developers and users run on some distribution it would
be a more efficient approach to let packagers worry about the
dependencies. Then a developer that wants to join the effort can just
get all the packages from the distro of her/his choice and clone the git
tree and voila everything is just there and works (famous last words, I
know ;) ). Users are most likely to try and find the packages in the
repositories of their distro of choice first before they will go looking
for packages from the project. Thus if we can get the word out and make
it easy to package CloudStack we will get good support in the
distributions and should not have to worry about having packages from
the project. This way the license concern issue is also avoided.
Therefore, IMHO the project should not really worry too much about
dependencies, other than clearly documenting them.
The goal should be to provide a build setup that:
- does not download anything
- builds the artifacts
- places the artifacts in a reasonably nice directory structure
- provide a generic guide for packagers that explains what artifacts are
part of the various CloudStack components
If we want to ship packages for certain operating systems we need to align our
dependency versions to exactly the versions shipped by those distributions.
And that will be a nightmare for the project. It is much better to leave
that rat hole alone and let people that contribute to the project worry
about the core
- how to build it
- how do the various components work together
- cloud management functionality
- documentation
That said, my question is, is the 4.0 release there?
- will maven produce all the artifacts needed or is there still some
ugly mix of waf, maven, or whatever?
- is there documentation about how the components are supposed to fit
together?
- is there a document that describes all the dependencies and when they
come into play. For example in a KVM setup one would not have a
dependencies on the xen-java stuff.
Unfortunately I have lost track of where things stand, thus please
excuse me if these questions have been answered or the answers are
obvious from previous communication during the 4.0 cycle.
Later,
Robert
--
Robert Schweikert MAY THE SOURCE BE WITH YOU
SUSE-IBM Software Integration Center LINUX
Tech Lead
rjsch...@suse.com
rschw...@ca.ibm.com
781-464-8147
