Hi Chesnay, I think the advantage of also doing a release is that we have then completely externalized the connector. If we don't make a release, I think it would appear as partially externalized (since the binaries are still only created with Flink core, not from the external repository). It would also mean that in our documentation we would still point to the binary created with the Flink core release.
There are also other connectors (like Opensearch and I believe RabbitMQ) that will end up straight in their own repositories. Since we also would like to document those, I don't think the situation will be messy. We can also solve it with an information hint in the documentation. With regards to point 6, do you have an alternative? Would you prefer to remove a connector in a Flink patch release? Best regards, Martijn On Fri, 18 Feb 2022 at 16:17, Chesnay Schepler <ches...@apache.org> wrote: > Why do you want to immediately do a release for the elasticsearch > connector? What does that provide us? > > I'd rather first have a fully working setup and integrated documentation > before thinking about releasing anything. > Once we have that we may be able to externalize all connectors within 1 > release cycle and do a clean switch; otherwise we end up with a bit of a > messy situation for users where some connectors use version scheme A and > others B. > > I also doubt the value of 6). They'll have to update the version anyway > (and discover at some point that the version scheme has changed), so I > don't see what this makes easier. > > On 18/02/2022 14:54, Martijn Visser wrote: > > Hi everyone, > > > > As a follow-up to earlier discussions [1] [2] to externalize the > connectors > > from the Flink repository, I would like to propose a plan to externalize > > these connectors. The goal of this plan is to start with moving > connectors > > to its own repositories without introducing regressions for connector > > developers. > > > > The plan is as follows: > > > > 1. A new repository is requested for a connector. > > 2. The code for that connector is moved to its individual repository, > > including the commit history > > 3. Any first release made for a connector in an external connector > > repository starts with major version 3, so 3.0.0. The reason for that is > > that we want to decouple the Flink releases from a connector release. If > we > > would start with major version 2, it could cause some confusion because > > people could think a Flink 2.0 has been released. This does mean that > each > > connector needs to have a compatibility matrix generated, stating which > > version number of the connector is compatible with the correct Flink > > version. > > 4. The group id and artifact id for the connector will remain the same, > > only the version is different. > > 5. The connector dependencies on the Flink website are updated to point > to > > the newly released connector artifact. > > 6. If a connector is moved, there is one release cycle where there will > be > > binary releases for that connector in both Flink core and from the > > connector repository. This is to make Flink users who are upgrading > > slightly easier. We will have to make a note in the release notes that a > > connector has been moved and that a user should update any references > from > > the original connector artifact (from the Flink connector) to the new > > connector artifact (from the external conenctor version) > > > > We propose to first try to execute this plan for the Elasticsearch > > connector as follows: > > > > 1. We wait until the Flink 1.15 release branch is cut > > 2. When that's done, the Elasticsearch code (including commit history) > from > > Flink's 1.15 release branch will be moved to the > > flink-connector-elasticsearch main branch. > > 3. When Flink 1.15 is released, we will also release an Elasticsearch > > connector for the external connector repository with version 3.0.0. > > 4. Bugfixes or improvements will be made first pointing to the external > > connector repository and will be cherry-picked back to the release-1.15 > > branch in the Flink core repository. > > 5. The Elasticsearch code, test etc will be removed from the master > branch > > in the Flink core repository and dropped with Flink 1.16 > > > > Looking forward to your thoughts on this! > > > > Best regards, > > > > Martijn Visser > > https://twitter.com/MartijnVisser82 > > > > [1] https://lists.apache.org/thread/bywh947r2f5hfocxq598zhyh06zhksrm > > [2] https://lists.apache.org/thread/bk9f91o6wk66zdh353j1n7sfshh262tr > > > >
