Thanks Jack/Ryan for the feedback! I am happy to go with option 3 if that is what people agree on. Its the simplest change.
To be clear: We will just skip tests in the Nessie module on Java8. This will still create the jars for releases and still run tests for Java11. I will wait a day or two to see if anyone else has any feedback and if not will do Option 3. Best, Ryan On Mon, Jun 14, 2021 at 9:48 PM Ryan Blue <b...@apache.org> wrote: > I would also vote for option 3. That seems like the easiest way to go if > the Nessie server can't run in JDK8. Assuming that people have multiple > JDKs available seems brittle to me, too. And using a container seems like a > bit more development work in the Iceberg build. How long can we continue > running as we currently do? > > On Mon, Jun 14, 2021 at 9:26 AM Jack Ye <yezhao...@gmail.com> wrote: > >> My thinking is that currently we already have CI tasks for JDK 8 and 11 >> separately, so it would be sufficient to only run build on JDK8 and run >> build and test on JDK11, which leads to option 3. >> >> For option 1, I think people do mostly have JDK 11, but might not want to >> have 8 and 11 at the same time in classpath, so we should be more cautious >> about it. For option 2, yes the major concern will be the large Docker >> dependency, but since it is only test dependency it should be fine, so that >> is also a good way to go. >> >> -Jack Ye >> >> On Mon, Jun 14, 2021 at 6:14 AM Ryan Murray <rym...@gmail.com> wrote: >> >>> Hi All, >>> >>> Currently the iceberg CI/development build relies on a custom gradle >>> plugin to start a Nessie server for the nessie module tests. The underlying >>> tech that powers a nessie server is Quarkus. Unfortunately Quarkus is >>> dropping support for building and running from jdk8. This means that the >>> Java8 build and the Nessie module are no longer compatible. Note this is >>> the server only, and has no effect on the client. The client and the nessie >>> iceberg catalog are still jdk8/jdk11 compatible. >>> >>> We have a few ways to fix this and I wanted to solicit the opinion of >>> the community before working on one: >>> >>> 1) Have a JDK11 available on the PATH for builds. JDK11 based builds >>> work fine and JDK8 based builds can use the JDK11 on PATH to start the >>> Nessie server. This is easy in CI but it requires all devs to have a JDK11 >>> available. I am assuming that most developers have jdk11 on their path >>> already? >>> 2) Use testcontainers. The testcontainers plugin would start the Nessie >>> docker image for nessie tests. This would add a dependency of Docker to >>> both CI builds and developer builds. I am aware of other Apache projects >>> who use testcontainers so I don't think there are license issues but Docker >>> is potentially a large dependency. >>> 3) Skip Nessie module for JDK8 builds. This has little effect on CI and >>> developers but it complicates the release as to get all release modules a >>> build would have to happen for JDK8 and JDK11. >>> 4) We rebuild the Nessie server to work w/ Jersey. This has no effect on >>> iceberg devs but it's quite expensive for us. >>> >>> The order of preference for us is 1->4. I understand that most people >>> will prefer 4 but I am hoping that 1, 2, or 3 may be possible and we can >>> save some dev time on project Nessie. >>> >>> Best, >>> Ryan >>> >>> >>> > > -- > Ryan Blue >
