Thanks for bringing this discussion back. When we decided to decouple the connectors, we already discussed that we will only realize the full benefit when the connectors actually become independent from the Flink minor releases. Until that happens we have a ton of extra work but limited gain. Based on the assumption that getting to the binary compatibility guarantee is our goal - not just for the connectors managed within the Flink project but for the ecosystem as a whole - I don't see the benefit of mono repo or similar approach that targets the symptom rather than the cause.
In the final picture we would only need connector releases if/when a specific connector changes and the repository per connector layout would work well. I also agree with Danny that we may not have to wait for Flink 2.0 for that. How close are we to assume compatibility of the API surface that affects connectors? It appears that practically there have been little to no known issues in the last couple of releases? Would it be possible to verify that by running e2e tests of connector binaries built against an earlier Flink minor version against the latest Flink minor release candidate as part of the release? Thanks, Thomas On Tue, Jun 11, 2024 at 11:05 AM Chesnay Schepler <ches...@apache.org> wrote: > On 10/06/2024 18:25, Danny Cranmer wrote: > > This would > > mean we would usually not need to release a new connector version per > Flink > > version, assuming there are no breaking changes. > We technically can't do this because we don't provide binary > compatibility across minor versions. > That's the entire reason we did this coupling in the first place, and > imo /we/ shouldn't take a shortcut but still have our users face that > very problem. > We knew this was gonna by annoying for us; that was intentional and > meant to finally push us towards binary compatibility /guarantees/.
