I think the issue is independent of releasing since it is a CI thing. And I think Yi is actually pointing out that it will be a user thing (since users will want to use newer JDK also)
But still I agree that as the project matures the pro/con of making modules more independent changes. So if that is true, then we might think: - Perhaps the SamzaRunner community is happy with old Samza on JDK 1.8. - Perhaps the SparkRunner community is happy with Spark < 3.4.0 or using just JDK 1.8. Or the same situation if the communities are too small to keep up with upgrades. If either of these are true, what would the steps be? Kenn On Tue, Nov 26, 2024 at 11:23 AM Jan Lukavský <je...@seznam.cz> wrote: > Hi, > > issues like this arise from the fact that we have tight coupling between > various parts of our ecosystem - the model, the core and runners. We should > decouple this and enable runners to have their own release cycles, because > anything other will not scale in the long run. We cannot have more and more > runners in our release dependencies, because various conflicts will block > our ability to create a stable release, at some point. We might want to > define a roadmap to avoid this. > > Best, > > Jan > On 11/26/24 15:46, Kenneth Knowles wrote: > > Are we using --target 8 in the Ci/CD and/or --source 8 for some modules? > Are the problems independent of what those flags control? > > Just curious - I am not advocating for anything. > > Kenn > > On Mon, Nov 25, 2024 at 1:38 PM Yi Hu via dev <dev@beam.apache.org> wrote: > >> Hi Beam community, >> >> As Beam repo CI/CD has moved to Java 11 [1], I would like to bring up >> attention to two outstanding open Issues that could affect Spark and Samza >> runner users [2, 3] >> >> For Spark Runner issue, currently Spark runner does not support Spark >> 3.4.0+ with Java11. This is due to Spark 3.4.0 upgraded to use slf4j 2. >> However the same Spark version still runs fine with Java8. >> >> For Samza Runner, currently it does not support Java11. Newer Samza does >> support Java11, however, the upgrade appears nontrivial. >> >> It would be great if someone actively using Spark and Samza runner could >> volunteer to add or complete Java 11 support, or share their solution for >> these issues. >> >> Thanks for your attention, >> >> Yi >> >> [1] https://github.com/apache/beam/issues/31677 >> [2] https://github.com/apache/beam/issues/32207 >> [3] https://github.com/apache/beam/issues/32208 >> >
