+1 for option 3 We do similar things for Hive3 tests where only jdk8 is supported.
> On Jun 14, 2021, at 23:02, Ryan Murray <rym...@gmail.com> wrote: > > Thanks Jack/Ryan for the feedback! > > I am happy to go with option 3 if that is what people agree on. Its the > simplest change. > > To be clear: We will just skip tests in the Nessie module on Java8. This will > still create the jars for releases and still run tests for Java11. > > I will wait a day or two to see if anyone else has any feedback and if not > will do Option 3. > > Best, > Ryan > > On Mon, Jun 14, 2021 at 9:48 PM Ryan Blue <b...@apache.org > <mailto:b...@apache.org>> wrote: > I would also vote for option 3. That seems like the easiest way to go if the > Nessie server can't run in JDK8. Assuming that people have multiple JDKs > available seems brittle to me, too. And using a container seems like a bit > more development work in the Iceberg build. How long can we continue running > as we currently do? > > On Mon, Jun 14, 2021 at 9:26 AM Jack Ye <yezhao...@gmail.com > <mailto:yezhao...@gmail.com>> wrote: > My thinking is that currently we already have CI tasks for JDK 8 and 11 > separately, so it would be sufficient to only run build on JDK8 and run build > and test on JDK11, which leads to option 3. > > For option 1, I think people do mostly have JDK 11, but might not want to > have 8 and 11 at the same time in classpath, so we should be more cautious > about it. For option 2, yes the major concern will be the large Docker > dependency, but since it is only test dependency it should be fine, so that > is also a good way to go. > > -Jack Ye > > On Mon, Jun 14, 2021 at 6:14 AM Ryan Murray <rym...@gmail.com > <mailto:rym...@gmail.com>> wrote: > Hi All, > > Currently the iceberg CI/development build relies on a custom gradle plugin > to start a Nessie server for the nessie module tests. The underlying tech > that powers a nessie server is Quarkus. Unfortunately Quarkus is dropping > support for building and running from jdk8. This means that the Java8 build > and the Nessie module are no longer compatible. Note this is the server only, > and has no effect on the client. The client and the nessie iceberg catalog > are still jdk8/jdk11 compatible. > > We have a few ways to fix this and I wanted to solicit the opinion of the > community before working on one: > > 1) Have a JDK11 available on the PATH for builds. JDK11 based builds work > fine and JDK8 based builds can use the JDK11 on PATH to start the Nessie > server. This is easy in CI but it requires all devs to have a JDK11 > available. I am assuming that most developers have jdk11 on their path > already? > 2) Use testcontainers. The testcontainers plugin would start the Nessie > docker image for nessie tests. This would add a dependency of Docker to both > CI builds and developer builds. I am aware of other Apache projects who use > testcontainers so I don't think there are license issues but Docker is > potentially a large dependency. > 3) Skip Nessie module for JDK8 builds. This has little effect on CI and > developers but it complicates the release as to get all release modules a > build would have to happen for JDK8 and JDK11. > 4) We rebuild the Nessie server to work w/ Jersey. This has no effect on > iceberg devs but it's quite expensive for us. > > The order of preference for us is 1->4. I understand that most people will > prefer 4 but I am hoping that 1, 2, or 3 may be possible and we can save some > dev time on project Nessie. > > Best, > Ryan > > > > > -- > Ryan Blue
