Thanks Sergey for the great work! ZGC in JDK21 is now a multi-generational garbage collector. In jdk17, we tried using ZGC as the default garbage collector for streaming computations, but found that while it reduced STW, it did have a negative impact on job throughput because a single-generation collector was not as efficient as a generational collector. This issue might be resolved in JDK21.
Looking forward to this! Best Regards, Xiangyu Sergey Nuyanzin <snuyan...@gmail.com> 于2023年12月11日周一 07:06写道： > thanks for checking and creation the ticket > > yes, that probably makes sense > > On Thu, Nov 30, 2023 at 1:08 PM Yun Tang <myas...@live.com> wrote: > > > Hi Sergey, > > > > I checked the CI [1] which was executed with Java21, and noticed that the > > StatefulJobSnapshotMigrationITCase-related tests have passed, which > proves > > what I guessed before, most checkpoints/savepoints should be restored > > successfully. > > > > I think we shall introduce such snapshot migration tests, which restore > > snapshots containing scala code. I also create a ticket focused on Java17 > > [2] > > > > > > [1] > > > https://dev.azure.com/snuyanzin/flink/_build/results?buildId=2620&view=logs&j=0a15d512-44ac-5ba5-97ab-13a5d066c22c&t=9a028d19-6c4b-5a4e-d378-03fca149d0b1 > > [2] https://issues.apache.org/jira/browse/FLINK-33707 > > > > > > Best > > Yun Tang > > ________________________________ > > From: Sergey Nuyanzin <snuyan...@gmail.com> > > Sent: Thursday, November 30, 2023 14:41 > > To: dev@flink.apache.org <dev@flink.apache.org> > > Subject: Re: [ANNOUNCE] Experimental Java 21 support now available on > > master > > > > Thanks Yun Tang > > > > One question to clarify: since the scala version was also bumped for java > > 17, shouldn't there be a similar task for java 17? > > > > On Thu, Nov 30, 2023 at 3:43 AM Yun Tang <myas...@live.com> wrote: > > > > > Hi Sergey, > > > > > > You can leverage all tests extending SnapshotMigrationTestBase[1] to > > > verify the logic. I believe all binary _metadata existing in the > > resources > > > folder[2] were built by JDK8. > > > > > > I also create a ticket FLINK-33699[3] to track this. > > > > > > [1] > > > > > > https://github.com/apache/flink/blob/master/flink-tests/src/test/java/org/apache/flink/test/checkpointing/utils/SnapshotMigrationTestBase.java > > > [2] > > > > > > https://github.com/apache/flink/tree/master/flink-tests/src/test/resources > > > [3] https://issues.apache.org/jira/browse/FLINK-33699 > > > > > > Best > > > Yun Tang > > > ________________________________ > > > From: Sergey Nuyanzin <snuyan...@gmail.com> > > > Sent: Wednesday, November 29, 2023 22:56 > > > To: dev@flink.apache.org <dev@flink.apache.org> > > > Subject: Re: [ANNOUNCE] Experimental Java 21 support now available on > > > master > > > > > > thanks for the response > > > > > > > > > >I feel doubt about the conclusion that "don't try to load a savepoint > > from > > > a Java 8/11/17 build due to bumping to scala-2.12.18", since the > > > snapshotted state (operator/keyed state-backend), and most key/value > > > serializer snapshots are generated by pure-java code. > > > >The only left part is that the developer uses scala UDF or scala types > > for > > > key/value types. However, since all user-facing scala APIs have been > > > deprecated, I don't think we have so many cases. Maybe we can give > > > descriptions without such strong suggestions. > > > > > > That is the area where I feel I lack the knowledge to answer this > > > precisely. > > > My assumption was that statement about Java 21 regarding this should be > > > similar to Java 17 which is almost same [1] > > > Sorry for the inaccuracy > > > Based on your statements I agree that the conclusion could be more > > relaxed. > > > > > > I'm curious whether there are some tests or anything which could > clarify > > > this? > > > > > > [1] https://lists.apache.org/thread/mz0m6wqjmqy8htob3w4469pjbg9305do > > > > > > On Wed, Nov 29, 2023 at 12:25 PM Yun Tang <myas...@live.com> wrote: > > > > > > > Thanks Sergey for the great work. > > > > > > > > I feel doubt about the conclusion that "don't try to load a savepoint > > > from > > > > a Java 8/11/17 build due to bummping to scala-2.12.18", since the > > > > snapshotted state (operator/keyed state-backend), and most key/value > > > > serializer snapshots are generated by pure-java code. The only left > > part > > > is > > > > that the developer uses scala UDF or scala types for key/value types. > > > > However, since all user-facing scala APIs have been deprecated [1], I > > > don't > > > > think we have so many cases. Maybe we can give descriptions without > > such > > > > strong suggestions. > > > > > > > > Please correct me if I am wrong. > > > > > > > > > > > > [1] https://issues.apache.org/jira/browse/FLINK-29740 > > > > > > > > Best > > > > Yun Tang > > > > > > > > ________________________________ > > > > From: Rui Fan <1996fan...@gmail.com> > > > > Sent: Wednesday, November 29, 2023 16:43 > > > > To: dev@flink.apache.org <dev@flink.apache.org> > > > > Subject: Re: [ANNOUNCE] Experimental Java 21 support now available on > > > > master > > > > > > > > Thanks Sergey for the great work! > > > > > > > > Best, > > > > Rui > > > > > > > > On Wed, Nov 29, 2023 at 4:42 PM Leonard Xu <xbjt...@gmail.com> > wrote: > > > > > > > > > Cool ! > > > > > > > > > > Thanks Sergey for the great effort and all involved. > > > > > > > > > > > > > > > Best, > > > > > Leonard > > > > > > > > > > > 2023年11月29日 下午4:31，Swapnal Varma <swapnal...@gmail.com> 写道： > > > > > > > > > > > > Congratulations Sergey, and everyone involved! > > > > > > > > > > > > Excited to work with and on this! > > > > > > > > > > > > Best, > > > > > > Swapnal > > > > > > > > > > > > > > > > > > On Wed, 29 Nov 2023, 13:58 Sergey Nuyanzin, <snuyan...@gmail.com > > > > > > wrote: > > > > > > > > > > > >> The master branch now builds and runs with Java 21 > out-of-the-box. > > > > > >> > > > > > >> Notes: > > > > > >> - a nightly cron build was set up. > > > > > >> - In Java 21 builds, Scala is being bumped to 2.12.18 > > > > > >> which causes incompatibilities within Flink; > > > > > >> i.e. don't try to load a savepoint from a Java 8/11/17 build > > > > > >> - All the tests that are being skipped on Java 11/17 > > > > > >> are also skipped on Java 21. > > > > > >> > > > > > >> Huge shout-out to everyone participating > > > > > >> in review of my Java 21 related PRs > > > > > >> > > > > > >> If you run into any issues, please report it in FLINK-33163 > > > > > >> <https://issues.apache.org/jira/browse/FLINK-33163> . > > > > > >> > > > > > >> -- > > > > > >> Best regards, > > > > > >> Sergey > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > -- > > > Best regards, > > > Sergey > > > > > > > > > -- > > Best regards, > > Sergey > > > > > -- > Best regards, > Sergey >
