Howdy all,
Tonight while browsing through the latest commons projects, it
occurred to me the once modest commons project has grown substantially
over time and has a lot of great stuff. There are now lots of sub
projects. Actually, it's 37 active subprojects to be exact. Plus the
archived projects. And the release numbering, JRE requirements, and
dependency graph can get a bit confusing. (Took me several hours to
map it out.)
So I started thinking about what really makes it important to separate projects:
+ Different large external dependencies you wish to keep separate.
+ Projects that aren't related.
+ Projects that have very different user communities (e.g.
commons-math versus commons-dbcp).
And on the flip side, the benefits that come from having one artifact
and one release number to track, instead of a whole heap of release
numbers and dependencies to manage.
Then it hit me... most of the dependencies are with other commons
projects, and on Java itself. And quite a few of the projects run on
the same version of Java. Could this network of interrelated projects
be simplified by consolidating a handful of projects together as a
single project?
If so, what are the tradeoffs and what might it look like.
So that's the question... would it potentially be beneficial to
consolidate some of the projects? What do we lose? What do we gain?
To help get discussion and thoughts flowing, I've attached some notes
to the bottom of this email (and in attached txt files). Please don't
interpret these considerations as an actual final proposal... I'm
including them simply as a basis to initiate a possible discussion.
Regards,
Doug
Attachment 1: A possible consolidation plan.
Attachment 2: A starter list of pros and cons.
Attachment 3: A map of project dependencies and minimum Java versions
requirements.
Caveat: Please forgive me for any spelling errors or missing words
that went unnoticed while pulling all this together. Spelling isn't
my strong suit.
*Attachment 1: A possible consolidation plan...*
Commons-Lang (JDK 1.1/1.2 Compatible)
- lang
- collections
- logging
- primatives
- discovery
Commons-Utils (JDK 1.5... plus create one-time releases for JDK 1.3
and 1.4, using the appropriate past releases of the items below)
- attributes client api
- beanutils
- command line utils (CLI)
- codec
- compress/zip
- dbutils
- exec
- io
- jxpath
- pool
- proxy (unclear... this might not belong as it has some *optional*
dependencies)
- validator
Commons-Config (Could be in rolled into Commons-Utils)
- configuration
- digester
Commons-Net (JDK 1.4+. Keep old release of original commons-net
available for those needing jdk 1.2 support.)
- net
- email
- vfs
Commons-EE (JavaEE 1.4+.)
- dbcp
- el
- fileupload
- transaction
- chain (Note: The non-JavaEE stuff could go into utils group above if desired.)
Keep as Independent Subprojects:
- sanselan (hasn't reached 1.0 yet)
- betwixt (hasn't reached 1.0 yet)
- scxml (hasn't reached 1.0 yet)
- daemon (contains native code)
- modeler (requires jmx jars unless jdk 1.6+)
- math (maintain focus on math/sci community)
- launcher (ant dependency)
- jelly
- attributes' ant tasks
*Attachment 2: Pros and Cons*
Benefits of the separate projects in the status quo:
+ *JRE Versions*: Some projects run on Java releases as old as Java
1.1. Other projects have generics and need Java 5.0.
+ *Fewer Transitive Dependencies*: By keeping project A and B
separate, a project can use A without cluttering its transitive
dependency graph with all of B's dependencies.
+ *Smaller Jar Files*: Rather than having one single 1 MB jar file, I
can pick out just the handful of 150kb jar files I really need.
+ *Separate Release Cycles*: I can release project A without releasing
project B.
+ *Separate Committer Lists*: Project A and Project B can have
separate lists of committers, and access to the SVN directories can be
controlled accordingly.
+ *Legacy*: We've always done it this way, and it works. Why confuse
people with change.
Important Observations:
+ *Similar Committer Lists*: When you look at the projects, there is a
core group of individuals who are listed as committers on the majority
of projects. The committer lists between projects aren't all that
different. Particularly since the subprojects are all so
interdependent on each other, one project is likely to find and fix
another projects bugs.
+ *Very Few Transitive Dependencies*: Most of the commons subprojects
simply depend on each other. There are actually very few required
dependencies on external artifacts that could be undesirably included,
+ *Small Jars*: Most of the jars are only about 150kb. If they
combined the resulting jar would still only be about 1MB, which is
hardly problematic for 90% of the apps of there.
+ *Stable Projects*: Subprojects that support old versions of Java
tend to be the more mature and stable projects. Generally speaking,
new versions incorporate new features rather than bug fixes for very
very old features (there are exceptions, of course).
+ *Java 1.4 is 7 years old*: Java 1.4 came out in 2002. People
running Java 1.1, 1.2, and 1.3 are most likely more interested in the
existing commons releases than they are desiring of the newest stuff.
(Certainly those who run old Java but want the newest stuff fall in
the <10% category. As Craig once said... Apache designs primarily for
the more common uses (80%) than the exceptional uncommon cases (<20%).
Craig... did I get the quote correct this time around. ;-)
+ *Java 5.0 is 5 years old*: Java 5.0 came out in 2004. Sure, lots of
vendors were slow to fully support 5.0 (ahemIBMahem), but it's out and
established now. Even dbutils has elected to support Java 1.5 for
it's newest releases. If you need a Java 1.4 version of dbutils, one
can simply pickup the older releases.
Benefits of consolidating
+ *Simpler Dependency Management*: Rather than tracking down the
required versions of various dependencies and transitive dependencies,
it really can be much easier to just grap a single JAR with
everything, so long as you don't accidentally introduce new transitive
dependencies you didn't want.
+ *Easier Refactoring*: It's easier to manage clean design within a
single project (for example, there might be some sharable code between
commons-net and commons-vfs).
Other minor benefits of consolidating:
+ *Easier Corporate Approval*: So this might not seem such a big deal,
but organizations which have to strong-arm the corporate lawyers to
get approval for various open-source libraries don't do so well
handing the lawyers a list of 37 separate projects to approve.
+ *Less Bewildering*: A new person looking at the website is hit by a
list of 37 projects, and tends to instinctively become overwhelmed
before getting started.
*Attachment 3: Commons Projects and their dependencies*
Disclaimer: I put this list together based on a variety of sources,
ranging from truck/pom.xml to project.properties to project webpages.
Please don't flame me if I didn't get everything exactly right. I put
this list together quickly to get a sense of what the issues are in
consolidating projects and dependency lists.
attributes
- JDK: 1.4
- Required: qdox
- Optional/Provided: ant, maven-xdoc
beanutils
- JDK: 1.3
- Required: commons-logging
- Optional/Provided: commons-collections
betwixt
- Note: Not yet release 1.0
- JDK: 1.3 + xml
- Required: beanutils, collections, digester, logging
- Optional/Provided:
chain
- JDK: 1.3
- JavaEE: 1.3
- Required: commons-digester
- Optional/Provided: javax.servlet 2.3, javax.portlet 1.0, javax.faces 1.1
cli
- JDK: 1.4
- Required: None
- Optional/Provided: None
codec
- JDK: 1.4
- Required:
- Optional/Provided:
collections
- JDK: 1.2 with partial 1.1 support, pom.xml in trunk targets jdk 1.5
- Required: None
- Optional/Provided: None
compress
- JDK: 1.4
- Required: None
- Optional/Provided: None
configuration
- JDK: 1.3 + xml, 1.4, or 1.5
- Required: commons-collections, commons-lang, commons-logging,
commons-digester, commons-beanutils
- Optional/Provided: commons-jxpath, commons-codec, commons-jexl
commons-vfs, javax.servlet, ant
daemon
- Note: Includes native C code
- JDK: 1.3
- Required: None
- Optional/Provided: None
dbcp
- JDK: 1.4 (1.3 version comes with JDBC 3.0 support)
- Required: commons-pool
- Optional/Provided: JavaEE, concurrent
dbutils
- JDK: 1.5 (generics)
- Required: None
- Optional/Provided: None
digester
- JDK: 1.5
- Required: commons-loggging, commons-beanutils
- Optional/Provided: None
discovery
- JDK: 1.1
- Required: commons-logging
- Optional/Provided: None
el
- JDK: 1.4
- JavaEE: 1.4
- Required: commons-logging
- Optional/Provided: javax.servlet 2.4 (for JSP 2.0 support)
email
- JDK: 1.4
- Required: javax.activation, javax.email
- Optional/Provided:
exec
- JDK: 1.3
- Required: None
- Optional/Provided: None
fileupload
- JDK: 1.3
- JavaEE: 2.4
- Required: commons-io
- Optional/Provided: javax.portlet, javax.servlet 2.4
io
- JDK: 1.5
- Required: None
- Optional/Provided: None
jci
- JDK: 1.4
- Required: None
- Optional/Provided: A compiler
jelly
- JDK: 1.3 + xml
- Required: commons-beanutils, commons-cli, commons-collections
commons-jexl, commons-lang, commons-logging
dom4j, forehead, jaxen
- Optional/Provided: javax.servlet, jstl
jexl
- JDK: 1.5
- Required: commons-logging
- Optional/Provided: None
jxpath
- JDK: 1.3 + xml
- Required: commongs-logging
- Optional/Provided: javax.servlet, jdom, commons-beanutil
lang
- JDK: 1.2
- Required: None
- Optional/Provided: None
launcher
- JDK: 1.3 + xml
- Required: ant
- Optional/Provided:
logging
- JDK: 1.1
- Required: None
- Optional/Provided: log4j, logkit, avalon, javax.servlet
math
- JDK: 1.5
- Required: None
- Optional/Provided: None
modeler
- JDK: 1.3 + xml
- Required: commons-digester, commons-logging, jmx
- Optional/Provided: ant
net
- JDK: 1.2
- Required: oro
- Optional/Provided: None
pool
- JDK: 1.3
- Required: None
- Optional/Provided: None
primitives
- JDK: 1.1
- Required: commons-collections
- Optional/Provided:
proxy
- JDK: 1.4
- Required: None
- Optional/Provided: Lots, but they're all optional
sanselan
- Note: Not yet release 1.0
- JDK: 1.4
- Required: None
- Optional/Provided: None
scxml
- JDK: 1.4
- Required: commons-logging, commons-digester, commons-beanutils
- Optional/Provided: javax.servlet, javax.faces, commons-el, commons-jexl
transaction
- JDK: 1.2
- JavaEE: 1.4
- Required: commons-codec, commons-logging, log4j
- Optional/Provided: JavaEE 1.4 (servlet, jta, etc, etc.)
validator
- JDK: 1.4
- Required: commons-beanutils, commons-digester, commons-logging,
- Optional/Provided: oro
vfs
- JDK: 1.4
- Required: None
- Optional/Provided: None
Commons-Lang (JDK 1.1/1.2 Compatible)
- lang
- collections
- logging
- primatives
- discovery
Commons-Utils (JDK 1.5+ ... create one-time releases for JDK 1.3 and 1.4, using
the appropriate past releases of the items below)
- attributes client api
- beanutils
- command line utils (CLI)
- codec
- compress/zip
- dbutils
- exec
- io
- jxpath
- pool
- proxy (unclear... this might not belong as it has some *optional*
dependencies)
- validator
Commons-Config (Could be in rolled into Commons Utils)
- configuration
- digester
Commons-Net (JDK 1.4+. Keep old release of original commons-net available for
those needing jdk 1.2 support.)
- net
- email
- vfs
Commons-EE (JavaEE 1.4+.)
- chain (The non-JavaEE stuff could go into utils group above if desired.)
- dbcp
- el
- fileupload
- transaction
Keep as Independent Subprojects:
- sanselan (hasn't reached 1.0 yet)
- betwixt (hasn't reached 1.0 yet)
- scxml (hasn't reached 1.0 yet)
- daemon (contains native code)
- modeler (requires jmx jars unless jdk 1.6+)
- math (maintain focus on math/sci community)
- launcher (ant dependency)
- jelly
- attributes' ant tasks
*Attachment 2: Pros and Cons*
Benefits of the separate projects in the status quo:
+ *JRE Versions*: Some projects run on Java releases as old as Java 1.1. Other
projects have generics and need Java 5.0.
+ *Fewer Transitive Dependencies*: By keeping project A and B separate, a
project can use A without cluttering its transitive dependency graph with all
of B's dependencies.
+ *Smaller Jar Files*: Rather than having one single 1 MB jar file, I can pick
out just the handful of 150kb jar files I really need.
+ *Separate Release Cycles*: I can release project A without releasing project
B.
+ *Separate Committer Lists*: Project A and Project B can have separate lists
of committers, and access to the SVN directories can be controlled accordingly.
+ *Legacy*: We've always done it this way, and it works. Why confuse people
with change.
Important Observations:
+ *Similar Committer Lists*: When you look at the projects, there is a core
group of individuals who are listed as committers on the majority of projects.
The committer lists between projects aren't all that different. Particularly
since the subprojects are all so interdependent on each other, one project is
likely to find and fix another projects bugs.
+ *Very Few Transitive Dependencies*: Most of the commons subprojects simply
depend on each other. There are actually very few required dependencies on
external artifacts that could be undesirably included,
+ *Small Jars*: Most of the jars are only about 150kb. If they combined the
resulting jar would still only be about 1MB, which is hardly problematic for
90% of the apps of there.
+ *Stable Projects*: Subprojects that support old versions of Java tend to be
the more mature and stable projects. Generally speaking, new versions
incorporate new features rather than bug fixes for very very old features
(there are exceptions, of course).
+ *Java 1.4 is 7 years old*: Java 1.4 came out in 2002. People running Java
1.1, 1.2, and 1.3 are most likely more interested in the existing commons
releases than they are desiring of the newest stuff. (Certainly those who run
old Java but want the newest stuff fall in the <10% category. As Craig once
said... Apache designs primarily for the more common uses (80%) than the
exceptional uncommon cases (<20%). Craig... did I get the quote correct this
time around. ;-)
+ *Java 5.0 is 5 years old*: Java 5.0 came out in 2004. Sure, lots of vendors
were slow to fully support 5.0 (ahemIBMahem), but it's out and established now.
Even dbutils has elected to support Java 1.5 for it's newest releases. If you
need a Java 1.4 version of dbutils, one can simply pickup the older releases.
Benefits of consolidating
+ *Simpler Dependency Management*: Rather than tracking down the required
versions of various dependencies and transitive dependencies, it really can be
much easier to just grap a single JAR with everything, so long as you don't
accidentally introduce new transitive dependencies you didn't want.
+ *Easier Refactoring*: It's easier to manage clean design within a single
project (for example, there might be some sharable code between commons-net and
commons-vfs).
Other minor benefits of consolidating:
+ *Easier Corporate Approval*: So this might not seem such a big deal, but
organizations which have to strong-arm the corporate lawyers to get approval
for various open-source libraries don't do so well handing the lawyers a list
of 37 separate projects to approve.
+ *Less Bewildering*: A new person looking at the website is hit by a list of
37 projects, and tends to instinctively become overwhelmed before getting
started.
attributes
- JDK: 1.4
- Required: qdox
- Optional/Provided: ant, maven-xdoc
beanutils
- JDK: 1.3
- Required: commons-logging
- Optional/Provided: commons-collections
betwixt
- Note: Not yet release 1.0
- JDK: 1.3 + xml
- Required: beanutils, collections, digester, logging
- Optional/Provided:
chain
- JDK: 1.3
- JavaEE: 1.3
- Required: commons-digester
- Optional/Provided: javax.servlet 2.3, javax.portlet 1.0, javax.faces 1.1
cli
- JDK: 1.4
- Required: None
- Optional/Provided: None
codec
- JDK: 1.4
- Required:
- Optional/Provided:
collections
- JDK: 1.2 with partial 1.1 support, pom.xml in trunk targets jdk 1.5
- Required: None
- Optional/Provided: None
compress
- JDK: 1.4
- Required: None
- Optional/Provided: None
configuration
- JDK: 1.3 + xml, 1.4, or 1.5
- Required: commons-collections, commons-lang, commons-logging,
commons-digester, commons-beanutils
- Optional/Provided: commons-jxpath, commons-codec, commons-jexl
commons-vfs, javax.servlet, ant
daemon
- Note: Includes native C code
- JDK: 1.3
- Required: None
- Optional/Provided: None
dbcp
- JDK: 1.4 (1.3 version comes with JDBC 3.0 support)
- Required: commons-pool
- Optional/Provided: JavaEE, concurrent
dbutils
- JDK: 1.5 (generics)
- Required: None
- Optional/Provided: None
digester
- JDK: 1.5
- Required: commons-loggging, commons-beanutils
- Optional/Provided: None
discovery
- JDK: 1.1
- Required: commons-logging
- Optional/Provided: None
el
- JDK: 1.4
- JavaEE: 1.4
- Required: commons-logging
- Optional/Provided: javax.servlet 2.4 (for JSP 2.0 support)
email
- JDK: 1.4
- Required: javax.activation, javax.email
- Optional/Provided:
exec
- JDK: 1.3
- Required: None
- Optional/Provided: None
fileupload
- JDK: 1.3
- JavaEE: 2.4
- Required: commons-io
- Optional/Provided: javax.portlet, javax.servlet 2.4
io
- JDK: 1.5
- Required: None
- Optional/Provided: None
jci
- JDK: 1.4
- Required: None
- Optional/Provided: A compiler
jelly
- JDK: 1.3 + xml
- Required: commons-beanutils, commons-cli, commons-collections
commons-jexl, commons-lang, commons-logging
dom4j, forehead, jaxen
- Optional/Provided: javax.servlet, jstl
jexl
- JDK: 1.5
- Required: commons-logging
- Optional/Provided: None
jxpath
- JDK: 1.3 + xml
- Required: commongs-logging
- Optional/Provided: javax.servlet, jdom, commons-beanutil
lang
- JDK: 1.2
- Required: None
- Optional/Provided: None
launcher
- JDK: 1.3 + xml
- Required: ant
- Optional/Provided:
logging
- JDK: 1.1
- Required: None
- Optional/Provided: log4j, logkit, avalon, javax.servlet
math
- JDK: 1.5
- Required: None
- Optional/Provided: None
modeler
- JDK: 1.3 + xml
- Required: commons-digester, commons-logging, jmx
- Optional/Provided: ant
net
- JDK: 1.2
- Required: oro
- Optional/Provided: None
pool
- JDK: 1.3
- Required: None
- Optional/Provided: None
primitives
- JDK: 1.1
- Required: commons-collections
- Optional/Provided:
proxy
- JDK: 1.4
- Required: None
- Optional/Provided: Lots, but they're all optional
sanselan
- Note: Not yet release 1.0
- JDK: 1.4
- Required: None
- Optional/Provided: None
scxml
- JDK: 1.4
- Required: commons-logging, commons-digester, commons-beanutils
- Optional/Provided: javax.servlet, javax.faces, commons-el, commons-jexl
transaction
- JDK: 1.2
- JavaEE: 1.4
- Required: commons-codec, commons-logging, log4j
- Optional/Provided: JavaEE 1.4 (servlet, jta, etc, etc.)
validator
- JDK: 1.4
- Required: commons-beanutils, commons-digester, commons-logging,
- Optional/Provided: oro
vfs
- JDK: 1.4
- Required: None
- Optional/Provided: None
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org
