Hi Jing, Flink Shaded exists so that Flinks internal usage of commonly used packages such as Guava, Jackson inside of Flink don't clash with different versions that users might use when creating a Flink application. When I did the upgrade of Flink Shaded, we already ran into a bunch of problems because a lot of the externalized connectors relied on Flink Shaded, which made it problematic to get the connector to work on both Flink 1.17 and Flink 1.18. There's been quite a lot of effort put into making sure that externalized connectors don't rely on Flink Shaded at all anymore, by either using their own versions of shaded artifacts (which was the case with the Pulsar connector) or just removing the dependency on Flink Shaded all together, by using regular Java.
If you would upgrade flink-shaded from 16.1 to 17.0 in Flink 1.17, you would break all externalized connectors that rely on Flink Shaded's Guava version, plus you potentially would impact the runtime given that there's a newer Netty version etc. Flink-Shaded is usually only updated whenever a new Flink minor version is released and only at the beginning of the release cycle, so that there's enough time to stabilize Flink. All in all, we shouldn't upgrade flink-shaded in Flink 1.17. Best regards, Martijn On Tue, Sep 12, 2023 at 7:26 PM Jing Ge <j...@ververica.com.invalid> wrote: > Hi Dev, > > Currently Flink 1.17 is using flink-shaded 16.1 and Flink 1.18 is using > flink-shaded 17.0. Do we need to consider any compatibility rules between > them? E.g. is there any concern to upgrade flink-shaded from 16.1 to 17.x > for Flink 1.17? Or there are some implicit dependency rules between > them. Looking > forward to hearing from you. > > Best regards, > Jing >
