My thinking is that currently we already have CI tasks for JDK 8 and 11 separately, so it would be sufficient to only run build on JDK8 and run build and test on JDK11, which leads to option 3.
For option 1, I think people do mostly have JDK 11, but might not want to have 8 and 11 at the same time in classpath, so we should be more cautious about it. For option 2, yes the major concern will be the large Docker dependency, but since it is only test dependency it should be fine, so that is also a good way to go. -Jack Ye On Mon, Jun 14, 2021 at 6:14 AM Ryan Murray <rym...@gmail.com> wrote: > Hi All, > > Currently the iceberg CI/development build relies on a custom gradle > plugin to start a Nessie server for the nessie module tests. The underlying > tech that powers a nessie server is Quarkus. Unfortunately Quarkus is > dropping support for building and running from jdk8. This means that the > Java8 build and the Nessie module are no longer compatible. Note this is > the server only, and has no effect on the client. The client and the nessie > iceberg catalog are still jdk8/jdk11 compatible. > > We have a few ways to fix this and I wanted to solicit the opinion of the > community before working on one: > > 1) Have a JDK11 available on the PATH for builds. JDK11 based builds work > fine and JDK8 based builds can use the JDK11 on PATH to start the Nessie > server. This is easy in CI but it requires all devs to have a JDK11 > available. I am assuming that most developers have jdk11 on their path > already? > 2) Use testcontainers. The testcontainers plugin would start the Nessie > docker image for nessie tests. This would add a dependency of Docker to > both CI builds and developer builds. I am aware of other Apache projects > who use testcontainers so I don't think there are license issues but Docker > is potentially a large dependency. > 3) Skip Nessie module for JDK8 builds. This has little effect on CI and > developers but it complicates the release as to get all release modules a > build would have to happen for JDK8 and JDK11. > 4) We rebuild the Nessie server to work w/ Jersey. This has no effect on > iceberg devs but it's quite expensive for us. > > The order of preference for us is 1->4. I understand that most people will > prefer 4 but I am hoping that 1, 2, or 3 may be possible and we can save > some dev time on project Nessie. > > Best, > Ryan > > >
