Thanks for your quick replies.
I have spent some time reading the doc and poking at the code, and I
must admit that I'm still confused :)
However I have not tried building the package locally.
Before more specific comments/questions on some of your answers, here
are some more general questions?
- What is the root of the decision to build everything from source? In a
situation where the upstream provider has a clear policy about what goes
in and out, with a relatively simple and uniform build process (and the
output is signed), I'm wondering if there can't be a trust system that
gets established so everything does not have to be rebuilt.
- What is the limit of the actions that can be run in an install script?
For example, could parts of the application being installed be used
during the install script (e.g. call a utility to generate files)?
- What drives the decision to package certain Eclipse libraries as
system jars and not others (e.g. swt and osgi, but not emf or jface
whereas they can run on plain java as well)?
- Do we have stats about what the user is downloading? The reason I'm
asking is to understand if all the packages are needed.
See also embedded questions below
[...]
I believe Markus already touched on some of these. I only have an
incomplete list. None that some of my remarks / follow comments here
are based on my time as Debian maintainer of eclipse (mostly active
during 2009-2011ish). Anyhow:
* DFSG compatible licenses. Here Eclipse general excels with its IP
and rigid handling of copyright/licenses.
* Rebuild things from source. Some of it works, except for the
bootstrap problem. Namely to compile eclipse you (used to?) require
eclipse.
This was indeed the case in the past, and it is still the same. The
difference is that now everything is built with Maven rather than PDE
build. More specifically, it is built using a Maven plug-in called Tycho
which embeds parts of Eclipse (mostly osgi, ecf and p2).
* No / Avoid bundling of libraries and avoid static linking.
* Rebuild everything *without internet*. Mostly an issue with maven.
In my time, Eclipse did not have an issue here, but I heard they
considered moving to maven or so?
Really? Without internet access? With Tycho in the middle of this, it
means that we would first have to build all the Maven dependencies
(which may already be handled), plus all the Tycho dependencies which
themselves depend on Eclipse bits and pieces. Definitely not a piece of
cake, but it seems that the ppl from Fedora have managed to get this
working by using artifacts from a previous build.
* Debian generally tries to avoid "= $version" dependencies, which are
fairly common in the Java world (if I am not mistaken).
Yes, this is true especially in the Maven world since it guarantees
build reproducibility. Eclipse is a bit more flexible in how plug-ins
specify their dependencies on other plug-ins, but control is happening
higher up in features or in product since it is for us a way to make
sure the user is running a combination of software that has been tested
together (at least to some degree).
Again note that my remarks here are based from my experience and may
have changed since 2011. Hopefully you will find it easier - your time
as Eclipse committer might make a lot of things easier for you. Also
please note that some of these issues are not specific to Debian (e.g.
Fedora have similar concerns).
* Eclipse upstream and Debian had different ideas on certain things
like "how much should be rebuilt from source".
- Debian: Everything *must* be rebuildable and everything *should* be
be rebuilt during a package build.
- Eclipse: Automatic rebuilds are /sometimes/ an afterthought or not
done at all (e.g. because they "consume the official" binaries of
XYZ).
Originally (back in 2001) the source files were provided for
"viewing pleasure" and to allow users to fix a plug-in by working in the
IDE. Also the thing is that with the original build system (PDE), you
could not simply download the dependencies to compile your plug-in,
which made the inclusion of build scripts moot. However the eclipse
platform provided a huge tar.gz files with all the source from the
eclipse SDK along side with a bunch of ant scripts (some of which were
generated). This was not ideal either but solved the purpose of
rebuilding from source but this was only done for the SDK and not
generalized to other plug-ins.
Now with the massive adoption of Maven/Tycho (and some other
wrapping scripts called CBI), it should be possible to build anything
independently and have the right thing to happen. But, this does not
resolve the more fundamental difference between the Java and Linux
ecosystems of reusing vs rebuilding.
[...]
* Providing OSGi support for (non-eclipse) JAR files in other debian
packages.
- We spent a few month adding those back in 2009 and 2010.
Hopefully most of it is still correct. But it was a lot of work
to migrate from eclipse 3.2 to 3.5.
Sigh... Nowadays, more jars have manifest but work will still be
required.
* When build breaks, it is sometimes painful to figure out what went
wrong. Especially when the upstream build does not fail on first
error (but instead continues and explodes later or creates broken
binaries/jar files).
With Tycho, this has changed since the build proceeds depth first
(it compiles and produces the jar before moving to the next one) whereas
PDE build proceeded breath first.
But in summary, have a look at the eclipse-build project[2]. This is
probably the best possible entry point to compiling a newer eclipse.
It is definitely doable, but it required a larger commitment than I
was able to provide more than a couple of years (as a volunteer).
Kudos to you and the other people for what you've done so far.
Getting the package done and maintain it for so long was already a big
task especially with how things were upstream.
Now what strikes me with the current approach is the amount of
effort required to maintain an individual package, and I would really
hope that we can find a more systematic way of bringing various parts of
Eclipse to debian, especially that we want to provide more eclipse based
applications to the user (e.g. there is a standalone C debugger
(https://wiki.eclipse.org/CDT/StandaloneDebugger), or trace compass
(https://projects.eclipse.org/proposals/trace-compass)).
Again, thanks for your help and time. I will talk with the people
funding the effort to see what are the next steps, so don't take it
personally if you never hear from me :)
Regards,
Pascal
~Niels
PS: You will want a machine with sufficient RAM. I suspect today's
average hardware have sufficient capacity, but it is the difference
between a 15 minutes and a 90 minutes build (and possibly your build or
desktop being OOM-killed).
[1]https://www.debian.org/doc/debian-policy/
[2]http://wiki.eclipse.org/Linux_Tools_Project/Eclipse_Build
[3] The debian/eclipse-build-generatedScripts.tar.bz2 variant was
because we needed a rebuilt after the one provided from eclipse-build
and I did not want to override a binary file in the "orig" tarball.
[4] Fedora uses /bin/bash as /bin/sh. Debian migrated to /bin/dash, so
a lot of the "extra" features in bash are not present in our /bin/sh.
These bashism problems occur in many different places (even outside
eclipse-build).
