+1. Torally agree On Sat, 12 May 2018, 18:14 Christophe Jolif, <cjo...@gmail.com> wrote:
> Hi all, > > There is quite some time Flink Elasticsearch sink is broken for > Elastisearch 5.x (nearly a year): > > https://issues.apache.org/jira/browse/FLINK-7386 > > And there is no support for Elasticsearch 6.x: > > https://issues.apache.org/jira/browse/FLINK-8101 > > However several PRs were issued: > > https://github.com/apache/flink/pull/4675 > https://github.com/apache/flink/pull/5374 > > I also raised the issue on the mailing list in the 1.5 timeframe: > > http://apache-flink-mailing-list-archive.1008284.n3. > nabble.com/DISCUSS-Releasing-Flink-1-5-0-td20867.html#a20905 > > But things are still not really moving. However this seems something people > are looking for, so I would really like the community to consider that for > 1.6. > > The problems I see from comments on the PRs: > > - getting something that is following the Flink APIs initially created is a > nightmare because Elastic is pretty good at breaking compatibility the hard > way (see in particular in the last PR the cast that have to be made to get > an API that works in all cases) > - Elasticsearch is moving away from their "native" API Flink is using to > the REST APIs so there is little common ground between pre 6 and post 6 > even if Elasticsearch tried to get some level of compatibility in the APIs. > > My fear is that by trying to kill two birds with one stone, we actually get > nothing done. > > In the hope of moving that forward I would like to propose for 1.6 a new > Elasticsearch 6.x+ sink that would follow the design of the previous ones > BUT only leverage the new REST API and not inherit from existing classes. > It would really be close to what is in my previous PR: > https://github.com/apache/flink/pull/5374 but just focus on E6+/REST and > so > avoid any "strange" cast. > > This would not fill the gap of the 5.2+ not working but at least we would > be back on track with something for the future as REST API is where Elastic > is going. > > If people feel there is actual interest and chances this can be merged I'll > be working on issuing a new PR around that. > > Alternative is to get back working on the existing PR but it seems to be a > dead-end at the moment and not necessarily the best option long term as > anyway Elasticsearch is looking into promoting the REST API. > > Please let me know what you think? > > -- > Christophe >
