Re: commons-compress 1.26.0 optional dependency on commons-codec causes runtime failure.
On Tue, Feb 27, 2024 at 1:43 PM Piotr P. Karwasz wrote: > > Hi Elliotte, > > On Tue, 27 Feb 2024 at 13:20, Elliotte Rusty Harold > wrote: > > > > On Tue, Feb 27, 2024 at 8:27 AM Piotr P. Karwasz > > wrote: > > e will not be loaded even if it is available. > > > > > > Fortunately Commons Compress is well-engineered and easy to split. The > > > code that depends on: > > > > What about creating: > * separate `commons-compress-zstd`, `commons-compress-brotli`, > `commons-compress-pack200`, `commons-compress-xz` artifacts with the > packages that require external dependencies, > * a new `commons-compress-core` (or `-api` and `-impl`) with the rest > of the code, > * an empty `commons-compress` artifact with a hard dependency on the > remaining ones. It just needs a JPMS module descriptor with `requires > transitive` directives to the other modules. > > This way won't have problems when they automatically upgrade > `commons-compress`. They'll just end up with many additional > dependencies. > I'm not quite sure what you're proposing. This works if you also change the package to something like org.apache.commons.compress2. It does not work without changing the package names. The bottom line is: 1. A classpath MUST NOT have the same fully package qualified name in more than one JAR. This is a hard requirement in Java 9+ with JPMS. The JVM will not run a program that violates this. It is a very important requirement in Java 8. 2. Maven and Gradle cannot resolve dependency conflicts between jars with different groupID:artifactID. They both treat them as fully independent artifacts, even if they're not, and will add both to a classpath, thereby violating rule #1. Consequently if a widely used low level library like commons-compress ships two different artifacts (i.e. different groupId:artifactID) that both contain com.example.package.ClassName, it breaks dependents in hard to debug, hard to work around ways. It is a painful and confusing problem that is often hidden behind multiple levels of transitive dependencies. This can happen even in projects that don't directly depend on commons-compress at all. commons-compress is not a green field. It MUST NOT break existing clients by releasing new jars that split packages in existing classpaths. There are only two options: 1. Change Maven coordinates AND package names 2. Keep Maven coordinates AND package names. More detailed discussion of this at: https://jlbp.dev/JLBP-5 https://jlbp.dev/JLBP-6 -- Elliotte Rusty Harold elh...@ibiblio.org - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
Re: commons-compress 1.26.0 optional dependency on commons-codec causes runtime failure.
Hi all, Is this the wrong place for such a question? If so, would someone be so kind as to let me know how I should find an answer / raise this, please? Thanks, Andy On Thu, Feb 22, 2024 at 12:01 PM Andrew Coates wrote: > Hi all, > > I'm seeing a runtime failure using TarArchiveOutputStream when updating to > commons-compress 1.26.0. > > java.lang.NoClassDefFoundError: org/apache/commons/codec/Charsets > at org.apache.commons.compress@1.26.0 > /org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.(TarArchiveOutputStream.java:212) > at org.apache.commons.compress@1.26.0 > /org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.(TarArchiveOutputStream.java:157) > at org.apache.commons.compress@1.26.0 > /org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.(TarArchiveOutputStream.java:147) > at testcontainers@1.19.5 > /org.testcontainers.containers.ContainerState.copyFileToContainer(ContainerState.java:350) > ... > > Commons-compress 1.26.0 contains changes to make use of commons-codec, > rather than its own copy of files, but I see that the POM marks > commons-codec as *optional*. Excuse my potential ignorance, but I thought > optional dependencies shouldn't cause runtime failures if not present. Is > this not the case? > > Obviously, I can just add commons-codec as an explicit dependency. But > this seems wrong IMHO. > > Should I sign up for an account and raise this as a bug in Jira? > > Thanks, > > Andy >
Re: commons-compress 1.26.0 optional dependency on commons-codec causes runtime failure.
Hi Andrew. > Is this the wrong place for such a question? It is the right place. > how I should find an answer For your own usage you can just add such dependency. For a long-time solution, this usually takes months(sometime even years) for community to decide how to do. Of course this sounds quite slow but this is just how open source works... Andrew Coates 于2024年2月28日周三 21:19写道： > Hi all, > > Is this the wrong place for such a question? If so, would someone be so > kind as to let me know how I should find an answer / raise this, please? > > Thanks, > > Andy > > On Thu, Feb 22, 2024 at 12:01 PM Andrew Coates > wrote: > > > Hi all, > > > > I'm seeing a runtime failure using TarArchiveOutputStream when updating > to > > commons-compress 1.26.0. > > > > java.lang.NoClassDefFoundError: org/apache/commons/codec/Charsets > > at org.apache.commons.compress@1.26.0 > > > /org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.(TarArchiveOutputStream.java:212) > > at org.apache.commons.compress@1.26.0 > > > /org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.(TarArchiveOutputStream.java:157) > > at org.apache.commons.compress@1.26.0 > > > /org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.(TarArchiveOutputStream.java:147) > > at testcontainers@1.19.5 > > > /org.testcontainers.containers.ContainerState.copyFileToContainer(ContainerState.java:350) > > ... > > > > Commons-compress 1.26.0 contains changes to make use of commons-codec, > > rather than its own copy of files, but I see that the POM marks > > commons-codec as *optional*. Excuse my potential ignorance, but I thought > > optional dependencies shouldn't cause runtime failures if not present. > Is > > this not the case? > > > > Obviously, I can just add commons-codec as an explicit dependency. But > > this seems wrong IMHO. > > > > Should I sign up for an account and raise this as a bug in Jira? > > > > Thanks, > > > > Andy > > >
Re: commons-compress 1.26.0 optional dependency on commons-codec causes runtime failure.
> This works if you also change the package to something like org.apache.commons.compress2 Yas, I mean this. Elliotte Rusty Harold 于2024年2月28日周三 20:44写道： > On Tue, Feb 27, 2024 at 1:43 PM Piotr P. Karwasz > wrote: > > > > Hi Elliotte, > > > > On Tue, 27 Feb 2024 at 13:20, Elliotte Rusty Harold > wrote: > > > > > > On Tue, Feb 27, 2024 at 8:27 AM Piotr P. Karwasz > > > wrote: > > > e will not be loaded even if it is available. > > > > > > > > Fortunately Commons Compress is well-engineered and easy to split. > The > > > > code that depends on: > > > > > > What about creating: > > * separate `commons-compress-zstd`, `commons-compress-brotli`, > > `commons-compress-pack200`, `commons-compress-xz` artifacts with the > > packages that require external dependencies, > > * a new `commons-compress-core` (or `-api` and `-impl`) with the rest > > of the code, > > * an empty `commons-compress` artifact with a hard dependency on the > > remaining ones. It just needs a JPMS module descriptor with `requires > > transitive` directives to the other modules. > > > > This way won't have problems when they automatically upgrade > > `commons-compress`. They'll just end up with many additional > > dependencies. > > > > I'm not quite sure what you're proposing. This works if you also > change the package to something like org.apache.commons.compress2. It > does not work without changing the package names. The bottom line is: > > 1. A classpath MUST NOT have the same fully package qualified name in > more than one JAR. This is a hard requirement in Java 9+ with JPMS. > The JVM will not run a program that violates this. It is a very > important requirement in Java 8. > > 2. Maven and Gradle cannot resolve dependency conflicts between jars > with different groupID:artifactID. They both treat them as fully > independent artifacts, even if they're not, and will add both to a > classpath, thereby violating rule #1. > > Consequently if a widely used low level library like commons-compress > ships two different artifacts (i.e. different groupId:artifactID) that > both contain com.example.package.ClassName, it breaks dependents in > hard to debug, hard to work around ways. It is a painful and confusing > problem that is often hidden behind multiple levels of transitive > dependencies. This can happen even in projects that don't directly > depend on commons-compress at all. > > commons-compress is not a green field. It MUST NOT break existing > clients by releasing new jars that split packages in existing > classpaths. There are only two options: > > 1. Change Maven coordinates AND package names > 2. Keep Maven coordinates AND package names. > > More detailed discussion of this at: > > https://jlbp.dev/JLBP-5 > https://jlbp.dev/JLBP-6 > > -- > Elliotte Rusty Harold > elh...@ibiblio.org > > - > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > >
Re: commons-compress 1.26.0 optional dependency on commons-codec causes runtime failure.
Hi Elliotte, On Wed, 28 Feb 2024 at 13:44, Elliotte Rusty Harold wrote: > I'm not quite sure what you're proposing. This works if you also > change the package to something like org.apache.commons.compress2. It > does not work without changing the package names. The bottom line is: > > 1. A classpath MUST NOT have the same fully package qualified name in > more than one JAR. This is a hard requirement in Java 9+ with JPMS. > The JVM will not run a program that violates this. It is a very > important requirement in Java 8. > > 2. Maven and Gradle cannot resolve dependency conflicts between jars > with different groupID:artifactID. They both treat them as fully > independent artifacts, even if they're not, and will add both to a > classpath, thereby violating rule #1. Currently Commons Compress 1.26.0 has: * a single `commons-compress` artifact with packages: o.a.c.c.compressors, o.a.c.c.compressors.brotli, etc. What I am proposing is for 1.27.0 to have: * an (almost) empty `commons-compress` artifact that depends on `commons-compress-core`, `commons-compress-brotli`, etc. This artifact should only have a JPMS module descriptor with `requires transitive` directives. * a `commons-compress-brotli` artifact that depends on `org.brotli:dec` and a single package: o.a.c.c.compressors.brotli, * a `commons-compress-core` artifact with packages o.a.c.c.compressors, etc. JPMS does not really care which module has which package. As long as they are disjoint it will be fine. Also, since `commons-compress` will have `requires transitive` directives, libraries compiled against 1.26.0 that use `requires o.a.c.compress;` will still be able to read the `o.a.c.compress.core` module. Piotr - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
Re: commons-compress 1.26.0 optional dependency on commons-codec causes runtime failure.
Hmm, splitting will require a package change at least for osgi. Thats said as mentionned here the dependencies are used by laziness but it is trivial to copy the few needed class so no need to break consumers IMHO. Le mer. 28 févr. 2024 à 15:03, Piotr P. Karwasz a écrit : > Hi Elliotte, > > On Wed, 28 Feb 2024 at 13:44, Elliotte Rusty Harold > wrote: > > I'm not quite sure what you're proposing. This works if you also > > change the package to something like org.apache.commons.compress2. It > > does not work without changing the package names. The bottom line is: > > > > 1. A classpath MUST NOT have the same fully package qualified name in > > more than one JAR. This is a hard requirement in Java 9+ with JPMS. > > The JVM will not run a program that violates this. It is a very > > important requirement in Java 8. > > > > 2. Maven and Gradle cannot resolve dependency conflicts between jars > > with different groupID:artifactID. They both treat them as fully > > independent artifacts, even if they're not, and will add both to a > > classpath, thereby violating rule #1. > > Currently Commons Compress 1.26.0 has: > > * a single `commons-compress` artifact with packages: > o.a.c.c.compressors, o.a.c.c.compressors.brotli, etc. > > What I am proposing is for 1.27.0 to have: > > * an (almost) empty `commons-compress` artifact that depends on > `commons-compress-core`, `commons-compress-brotli`, etc. This artifact > should only have a JPMS module descriptor with `requires transitive` > directives. > * a `commons-compress-brotli` artifact that depends on > `org.brotli:dec` and a single package: o.a.c.c.compressors.brotli, > * a `commons-compress-core` artifact with packages o.a.c.c.compressors, > etc. > > JPMS does not really care which module has which package. As long as > they are disjoint it will be fine. Also, since `commons-compress` will > have `requires transitive` directives, libraries compiled against > 1.26.0 that use `requires o.a.c.compress;` will still be able to read > the `o.a.c.compress.core` module. > > > Piotr > > - > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > >
Re: commons-compress 1.26.0 optional dependency on commons-codec causes runtime failure.
Hi Romain, On Wed, 28 Feb 2024 at 16:30, Romain Manni-Bucau wrote: > > Hmm, splitting will require a package change at least for osgi. OSGi should be painless: the common practice is to use `Import-Package` instead of `Require-Bundle`, this way it is up to the OSGi environment to figure out that a certain package is in `commons-compress-core` instead of `commons-compress`. Remark that I am talking about moving whole packages to new artifacts. If we were to split a package in two parts, I agree with the previous posts that a new major version, Maven coordinates and package names are required. Piotr - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
Re: commons-compress 1.26.0 optional dependency on commons-codec causes runtime failure.
On Wed, 28 Feb 2024 at 20:06, Piotr P. Karwasz wrote: > > Hi Romain, > > On Wed, 28 Feb 2024 at 16:30, Romain Manni-Bucau > wrote: > > > > Hmm, splitting will require a package change at least for osgi. > > OSGi should be painless: the common practice is to use > `Import-Package` instead of `Require-Bundle`, this way it is up to the > OSGi environment to figure out that a certain package is in > `commons-compress-core` instead of `commons-compress`. > > Remark that I am talking about moving whole packages to new artifacts. Will these packages be renamed? If not, then I don't see how you can prevent possible class duplication. > If we were to split a package in two parts, I agree with the previous > posts that a new major version, Maven coordinates and package names > are required. > > Piotr > > - > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
Re: commons-compress 1.26.0 optional dependency on commons-codec causes runtime failure.
Hi sebb, On Wed, 28 Feb 2024 at 23:48, sebb wrote: > > Remark that I am talking about moving whole packages to new artifacts. > > Will these packages be renamed? > > If not, then I don't see how you can prevent possible class duplication. Do they need to be renamed? This breaks backward compatibility. A user could have duplicated classes if he has an old `commons-compress` 1.26.0 together with a newer `commons-compress-core`, but dependency management can be used to prevent version mismatches. Since an example is better than a thousand words I submitted a PoC on how to split Commons Compress: https://github.com/apache/commons-compress/pull/490 A simple `mvn clean verify` succeeds. I just need to smoothen out the CI's failures. Piotr - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
