+1 for option 3
We do similar things for Hive3 tests where only jdk8 is supported.

> On Jun 14, 2021, at 23:02, Ryan Murray <rym...@gmail.com> wrote:
> 
> Thanks Jack/Ryan for the feedback!
> 
> I am happy to go with option 3 if that is what people agree on. Its the 
> simplest change.
> 
> To be clear: We will just skip tests in the Nessie module on Java8. This will 
> still create the jars for releases and still run tests for Java11.
> 
> I will wait a day or two to see if anyone else has any feedback and if not 
> will do Option 3.
> 
> Best,
> Ryan
> 
> On Mon, Jun 14, 2021 at 9:48 PM Ryan Blue <b...@apache.org 
> <mailto:b...@apache.org>> wrote:
> I would also vote for option 3. That seems like the easiest way to go if the 
> Nessie server can't run in JDK8. Assuming that people have multiple JDKs 
> available seems brittle to me, too. And using a container seems like a bit 
> more development work in the Iceberg build. How long can we continue running 
> as we currently do?
> 
> On Mon, Jun 14, 2021 at 9:26 AM Jack Ye <yezhao...@gmail.com 
> <mailto:yezhao...@gmail.com>> wrote:
> My thinking is that currently we already have CI tasks for JDK 8 and 11 
> separately, so it would be sufficient to only run build on JDK8 and run build 
> and test on JDK11, which leads to option 3. 
> 
> For option 1, I think people do mostly have JDK 11, but might not want to 
> have 8 and 11 at the same time in classpath, so we should be more cautious 
> about it. For option 2, yes the major concern will be the large Docker 
> dependency, but since it is only test dependency it should be fine, so that 
> is also a good way to go.
> 
> -Jack Ye
> 
> On Mon, Jun 14, 2021 at 6:14 AM Ryan Murray <rym...@gmail.com 
> <mailto:rym...@gmail.com>> wrote:
> Hi All,
> 
> Currently the iceberg CI/development build relies on a custom gradle plugin 
> to start a Nessie server for the nessie module tests. The underlying tech 
> that powers a nessie server is Quarkus. Unfortunately Quarkus is dropping 
> support for building and running from jdk8. This means that the Java8 build 
> and the Nessie module are no longer compatible. Note this is the server only, 
> and has no effect on the client. The client and the nessie iceberg catalog 
> are still jdk8/jdk11 compatible.
> 
> We have a few ways to fix this and I wanted to solicit the opinion of the 
> community before working on one:
> 
> 1) Have a JDK11 available on the PATH for builds. JDK11 based builds work 
> fine and JDK8 based builds can use the JDK11 on PATH to start the Nessie 
> server. This is easy in CI but it requires all devs to have a JDK11 
> available. I am assuming that most developers have jdk11 on their path 
> already?
> 2) Use testcontainers. The testcontainers plugin would start the Nessie 
> docker image for nessie tests. This would add a dependency of Docker to both 
> CI builds and developer builds. I am aware of other Apache projects who use 
> testcontainers so I don't think there are license issues but Docker is 
> potentially a large dependency.
> 3) Skip Nessie module for JDK8 builds. This has little effect on CI and 
> developers but it complicates the release as to get all release modules a 
> build would have to happen for JDK8 and JDK11.
> 4) We rebuild the Nessie server to work w/ Jersey. This has no effect on 
> iceberg devs but it's quite expensive for us. 
> 
> The order of preference for us is 1->4. I understand that most people will 
> prefer 4 but I am hoping that 1, 2, or 3 may be possible and we can save some 
> dev time on project Nessie.
> 
> Best,
> Ryan
> 
> 
> 
> 
> -- 
> Ryan Blue

Reply via email to