Thanks for starting the discussion Robert. I agree that we should try to treat the Elasticsearch connectors in a similar fashion to the multiple Kafka connector versions.
+1 for flink-connector-elasticsearch-base +1 to consolidating the documents +1 for keeping the ES connector in Flink I’d also like to proceed with this with https://github.com/apache/flink/pull/2767, by opening another new PR based on #2767 that also refactors the ES connectors & consolidates documentation. After that, we can proceed with https://github.com/apache/flink/pull/2861 for the proper at least once support across all ES versions. - Gordon On December 19, 2016 at 11:58:03 PM, Fabian Hueske ([email protected]) wrote: Thanks for looking into this Robert. I agree, it would be good to share as much code as possible between the Elasticsearch connectors. +1 for flink-connector-elasticsearch-base +1 for keeping the connectors in Flink 2016-12-19 16:46 GMT+01:00 Robert Metzger <[email protected]>: > Hi, > I'm wondering how we want to maintain the ElasticSearch sinks in Flink. > > Right now, we have two connectors, supporting the 1.x and 2.x versions. > There is a pending PR [1] for adding ES 5 support. > > All three connectors share a lot of code, and there is a pull request [2] > improving the ES 2.x connector. If we want to make the ES sinks have proper > at least once support, we need to do some more changes, and I would like to > do them once for all connectors. > I propose to create a new "flink-connector-elasticsearch-base" module with > the common code and then create modules for 1.x, 2.x and 5.x. > Also, I would merge the documentation pages into one. > > I would also be open to discuss whether we want to move the EL connectors > into Bahir. > For now, I'm against that, given that ES is among the more popular > connectors in Flink. > > > Regards, > Robert > > [1] https://github.com/apache/flink/pull/2767 > [2] https://github.com/apache/flink/pull/2861 >
