I would also vote for option 3. That seems like the easiest way to go if the Nessie server can't run in JDK8. Assuming that people have multiple JDKs available seems brittle to me, too. And using a container seems like a bit more development work in the Iceberg build. How long can we continue running as we currently do?
On Mon, Jun 14, 2021 at 9:26 AM Jack Ye <yezhao...@gmail.com> wrote: > My thinking is that currently we already have CI tasks for JDK 8 and 11 > separately, so it would be sufficient to only run build on JDK8 and run > build and test on JDK11, which leads to option 3. > > For option 1, I think people do mostly have JDK 11, but might not want to > have 8 and 11 at the same time in classpath, so we should be more cautious > about it. For option 2, yes the major concern will be the large Docker > dependency, but since it is only test dependency it should be fine, so that > is also a good way to go. > > -Jack Ye > > On Mon, Jun 14, 2021 at 6:14 AM Ryan Murray <rym...@gmail.com> wrote: > >> Hi All, >> >> Currently the iceberg CI/development build relies on a custom gradle >> plugin to start a Nessie server for the nessie module tests. The underlying >> tech that powers a nessie server is Quarkus. Unfortunately Quarkus is >> dropping support for building and running from jdk8. This means that the >> Java8 build and the Nessie module are no longer compatible. Note this is >> the server only, and has no effect on the client. The client and the nessie >> iceberg catalog are still jdk8/jdk11 compatible. >> >> We have a few ways to fix this and I wanted to solicit the opinion of the >> community before working on one: >> >> 1) Have a JDK11 available on the PATH for builds. JDK11 based builds work >> fine and JDK8 based builds can use the JDK11 on PATH to start the Nessie >> server. This is easy in CI but it requires all devs to have a JDK11 >> available. I am assuming that most developers have jdk11 on their path >> already? >> 2) Use testcontainers. The testcontainers plugin would start the Nessie >> docker image for nessie tests. This would add a dependency of Docker to >> both CI builds and developer builds. I am aware of other Apache projects >> who use testcontainers so I don't think there are license issues but Docker >> is potentially a large dependency. >> 3) Skip Nessie module for JDK8 builds. This has little effect on CI and >> developers but it complicates the release as to get all release modules a >> build would have to happen for JDK8 and JDK11. >> 4) We rebuild the Nessie server to work w/ Jersey. This has no effect on >> iceberg devs but it's quite expensive for us. >> >> The order of preference for us is 1->4. I understand that most people >> will prefer 4 but I am hoping that 1, 2, or 3 may be possible and we can >> save some dev time on project Nessie. >> >> Best, >> Ryan >> >> >> -- Ryan Blue
