I think we can remove 11. Looks like we always do this: JAVA_HOME: /opt/hostedtoolcache/Java_Temurin-Hotspot_jdk/11.0.27-6/x64
However, we probably need to change that direct github workflow to remove :sdks:java:container:java8 and :sdks:java:container:java11 and add :sdks:java:container:java17. On Wed, Apr 30, 2025 at 10:23 AM Tobi Kaymak <kay...@google.com> wrote: > I do have a silly question, I am struggling to find the reason for the > fallback need of Java 11 in the beam_PreCommit_Java_Debezium_IO_Direct.yml > [0] > I was trying to not interfere with the build system outside of the module > by doing some tricks within the build.gradle of Debzium IO in the second > PR. [1] > However, it seems that the CI always picks up Java 11 and therefore fails > the build. It builds completely fine with Java 17 on my machine though. > Is there a way to force Java 17 for the module by, for example, removing > the 11 fallback from the PreCommit Workflow or does this have known > side-effects? > > Thank you! > > [0] > https://github.com/apache/beam/pull/34751/files#diff-de752bcab62398ee28adb75a8fdb1933d8d855f2ae46417ec1d47180174a1800R91 > [1] https://github.com/apache/beam/pull/34763 > > On Mon, Apr 28, 2025 at 5:04 PM Tobi Kaymak <kay...@google.com> wrote: > >> I did the second PR and removed the enforcement of proto3 in the >> build.gradle ( >> https://github.com/apache/beam/pull/34763/files#diff-1572bcc322ede7d3a757381b84791a29b691df4d8947f6ad39ce4597653787edL95) >> - >> All tests are green on my local machine, but they will of course fail on >> the CI since the first PR has not been merged yet. >> >> I would be very happy about a review by Danny McCormick, who did work on >> this a few months ago in his PR and added the enforcement ( >> https://github.com/apache/beam/pull/33526). >> >> >> On Sun, Apr 27, 2025 at 5:00 PM Tobi Kaymak <kay...@google.com> wrote: >> >>> I did create the first PR, after building the IO on my local machine >>> with gradle and Java 17. This should not have an effect other than changing >>> the build system. Feedback is very welcome: >>> https://github.com/apache/beam/pull/34751 >>> >>> On Sat, Apr 26, 2025 at 12:49 AM Tobi Kaymak <kay...@google.com> wrote: >>> >>>> Thank you both! >>>> I have checked and couldn't find a matching issue, so I created this >>>> one: https://github.com/apache/beam/issues/34747 >>>> >>>> On Sat, Apr 26, 2025 at 12:32 AM XQ Hu via dev <dev@beam.apache.org> >>>> wrote: >>>> >>>>> Can we create the GitHub issue to track this work if we haven't? >>>>> >>>>> On Fri, Apr 25, 2025, 4:39 PM Yi Hu via dev <dev@beam.apache.org> >>>>> wrote: >>>>> >>>>>> Hi Tobi, >>>>>> >>>>>> Thanks for your interest and willing to take on the task. First of >>>>>> all, could you please create a GitHub Issue to track the effort? >>>>>> >>>>>> I would like to share some comments if find useful >>>>>> >>>>>> - We already have infrastructure for the goal of "PR 1". We use >>>>>> BeamPluginModulePlugin [1] to initialize java modules and there is a >>>>>> setJavaVerOptions method. Here is a module could be an example how >>>>>> compiling with different Java version works in Beam: [2]. >>>>>> >>>>>> - We'll need to handle Debezium with newer Java version as well >>>>>> (That's the reason test currently failing on #34733). We'll need to >>>>>> modify >>>>>> test workflow file [3]. However due to GitHub limitation changes to >>>>>> .github/workflows won't be effective on unsubmitted PRs. You'd need to >>>>>> test >>>>>> your change locally. >>>>>> >>>>>> - Given that current Debezium (1.x) is already old I'm +1 with the >>>>>> change. >>>>>> >>>>>> [1] >>>>>> https://github.com/apache/beam/blob/master/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy >>>>>> >>>>>> [2] >>>>>> https://github.com/apache/beam/blob/master/sdks/java/container/agent/build.gradle >>>>>> >>>>>> [3] >>>>>> https://github.com/apache/beam/blob/master/.github/workflows/beam_PreCommit_Java_Debezium_IO_Direct.yml >>>>>> >>>>>> >>>>>> Thanks. >>>>>> >>>>>> Yi >>>>>> >>>>>> On Fri, Apr 25, 2025 at 3:06 PM Tobi Kaymak via dev < >>>>>> dev@beam.apache.org> wrote: >>>>>> >>>>>>> Hi Beam Devs, >>>>>>> >>>>>>> I'm interested in helping to upgrade the Debezium IO connector ( >>>>>>> sdks:java:io:debezium) to Debezium version 3.1.1.Final. This >>>>>>> version of Debezium requires its dependencies (like debezium-core) >>>>>>> to be compiled against Java 17 >>>>>>> <https://debezium.io/releases/3.0/release-notes#breaking_changes_14> >>>>>>> bytecode. >>>>>>> >>>>>>> I took the work from Danny McCormick who started doing something >>>>>>> similar for Debezium 2.7.4 and created a WIP branch >>>>>>> <https://github.com/apache/beam/pull/34733> to get my feet wet with >>>>>>> Java and Gradle's build system again. >>>>>>> >>>>>>> To manage this upgrade gracefully and make the review process >>>>>>> smoother, I was thinking to split the work into a two-step PR: >>>>>>> >>>>>>> 1. >>>>>>> >>>>>>> *PR 1 (Build Infrastructure):* A smaller, focused PR to enable >>>>>>> Java 17 compilation specifically for the sdks:java:io:debezium >>>>>>> module. This would likely involve: >>>>>>> - Modifying the sdks:java:io:debezium/build.gradle to allow >>>>>>> forking its compileJava (and compileTestJava) tasks using a >>>>>>> JDK 17 (e.g., via a java17Home Gradle property derived from a >>>>>>> CI environment variable like JAVA_HOME_17_X64). >>>>>>> - Updating the relevant PreCommit workflow ( >>>>>>> beam_PreCommit_Java_Debezium_IO_Direct.yml) >>>>>>> >>>>>>> <https://github.com/apache/beam/blob/master/.github/workflows/beam_PreCommit_Java_Debezium_IO_Direct.yml#L87> >>>>>>> to ensure a JDK 17 is available and its path is passed to Gradle >>>>>>> for this >>>>>>> module. >>>>>>> - The goal of this first PR would be to get the module to >>>>>>> compile successfully with Java 17 against the existing (older) >>>>>>> Debezium >>>>>>> version. >>>>>>> 2. >>>>>>> >>>>>>> *PR 2 (Debezium 3.1.1 Upgrade & API Adaptation):* A subsequent >>>>>>> PR that would: >>>>>>> - Actually update the Debezium dependencies to 3.1.1.Final. >>>>>>> - Incorporate the necessary code changes within >>>>>>> sdks:java:io:debezium to adapt to any API changes in Debezium >>>>>>> 3.1.1. >>>>>>> - Address any new functionality or considerations with >>>>>>> Debezium 3.1.1 (which I understand uses now libprotoc-dev 1.5, >>>>>>> >>>>>>> <https://github.com/debezium/container-images/pull/427/files#diff-bf3b0b8b023cf3cd20ca9d862bb8301523cdb9c934dec1ce96cd9640a613ae7e> >>>>>>> - >>>>>>> according to its release notes >>>>>>> <https://debezium.io/releases/3.1/release-notes#other_changes_5> >>>>>>> . >>>>>>> >>>>>>> My reasoning for this split is to isolate the build system changes >>>>>>> from the actual library upgrade and code adaptation, allowing for >>>>>>> smaller >>>>>>> code reviews. >>>>>>> >>>>>>> I'd appreciate any feedback or suggestions on the best way to >>>>>>> upgrade Debezium IO. >>>>>>> >>>>>>> Best >>>>>>> Tobi >>>>>>> >>>>>>
