Hi Christophe, great! Thanks for your contribution. I'm quite busy right now, but I agree that we should have support for ES 5.3 and Es 6.x for the next minor release 1.5.
Best, Fabian 2018-01-26 23:09 GMT+01:00 Christophe Jolif <cjo...@gmail.com>: > Ok, I got it "done". I have a PR for ES5.3 (FLINK-7386) just rebasing the > original one that was never merged (#4675). And added ES 6.X through > RestHighLevelClient on top (FLINK-8101). This is: > https://github.com/apache/flink/pull/5374. And believe it or not but > someone else submitted a PR for those two as well today! See: > https://github.com/apache/flink/pull/5372. So looks like there is some > traction to get it done? If would really be good if a committer could look > at those PRs and let us know which one is closer to get merge so we focus > on it instead of duplicating work ;) > > Thanks, > -- > Christophe > > On Fri, Jan 26, 2018 at 1:46 PM, Christophe Jolif <cjo...@gmail.com> > wrote: > >> Fabien, >> >> Unfortunately I need more than that :) But this PR is definitely a first >> step. >> >> My real need is Elasticsearch 6.x support through RestHighLevel client. >> FYI Elastic has deprecated the TransportClient that Flink connector >> leverages and it will be removed in Elasticsearch 8 (presumably ~1.5 years >> from now at their current release pace). Also TransportClient is not >> working with hosted version of Elasticsearch like Compose.io. So I think it >> makes a lot of sense to start introduce a sink based on RestHighLevel >> client. I'll be looking at creating a PR for that. >> >> Thanks, >> >> -- >> Christophe >> >> On Fri, Jan 26, 2018 at 10:11 AM, Fabian Hueske <fhue...@gmail.com> >> wrote: >> >>> Great, thank you! >>> Hopefully, this pushes the PR forward. >>> >>> Thanks, Fabian >>> >>> 2018-01-25 22:30 GMT+01:00 Christophe Jolif <cjo...@gmail.com>: >>> >>>> Hi Fabian, >>>> >>>> FYI I rebased the branch and tested it and it worked OK on a sample. >>>> >>>> -- >>>> Christophe >>>> >>>> On Mon, Jan 22, 2018 at 2:53 PM, Fabian Hueske <fhue...@gmail.com> >>>> wrote: >>>> >>>>> Hi Adrian, >>>>> >>>>> thanks for raising this issue again. >>>>> I agree, we should add support for newer ES versions. >>>>> I've added 1.5.0 as target release for FLINK-7386 and bumped the >>>>> priority up. >>>>> >>>>> In the meantime, you can try Flavio's approach (he responded to the >>>>> mail thread you linked) and fork and fix the connector. >>>>> You could also try the PR for FLINK-7386 [1] and comment on the pull >>>>> request whether it works for you or not. >>>>> >>>>> Best, Fabian >>>>> >>>>> [1] https://github.com/apache/flink/pull/4675 >>>>> >>>>> >>>>> 2018-01-22 13:54 GMT+01:00 Adrian Vasiliu <vasi...@fr.ibm.com>: >>>>> >>>>>> Hello, >>>>>> >>>>>> With a local run of Flink 1.4.0, ElasticsearchSink fails for me with >>>>>> a local run of Elasticsearch 5.6.4 and 5.2.1, while the same code >>>>>> (with adjusted versions of dependencies) works fine with Elasticsearch >>>>>> 2.x >>>>>> (tried 2.4.6). >>>>>> I get: >>>>>> java.lang.NoSuchMethodError: org.elasticsearch.action.bulk. >>>>>> BulkProcessor.add(Lorg/elasticsearch/action/ActionRequest;)L >>>>>> org/elasticsearch/action/bulk/BulkProcessor >>>>>> >>>>>> (env: Mac OSX 10.13.2, oracle jdk 1.8.0_112) >>>>>> >>>>>> Now, this looks similar to the issue referred in >>>>>> http://apache-flink-user-mailing-list-archive.2336050.n4.nab >>>>>> ble.com/Elasticsearch-Sink-Error-td15246.html >>>>>> which points to >>>>>> "Flink Elasticsearch 5 connector is not compatible with Elasticsearch >>>>>> 5.2+ client" >>>>>> https://issues.apache.org/jira/browse/FLINK-7386 >>>>>> >>>>>> Side-remark: when trying with Elasticsearch 5.6.4 via a docker >>>>>> container, for some reason the error I get is different: >>>>>> "RuntimeException: >>>>>> Client is not connected to any Elasticsearch nodes!" (while Elasticsearch >>>>>> 2.4.6 works fine via docker too). >>>>>> >>>>>> FLINK-7386 <https://issues.apache.org/jira/browse/FLINK-7386> being >>>>>> pending since August 2017, would it mean that there is nowadays still no >>>>>> way to make Flink 1.4.0's sink work with Elasticsearch 5.2+? My use-case >>>>>> involves Compose for Elasticsearch 5.6.3, shared by different apps, and I >>>>>> can't really downgrade its Elasticsearch version. >>>>>> Or would there be signs it will be fixed in Flink 1.5.0? >>>>>> >>>>>> Any lights welcome. >>>>>> >>>>>> Thanks, >>>>>> Adrian >>>>>> >>>>>> >>>>>> Sauf indication contraire ci-dessus:/ Unless stated otherwise above: >>>>>> Compagnie IBM France >>>>>> Siège Social : 17 avenue de l'Europe >>>>>> <https://maps.google.com/?q=17+avenue+de+l'Europe&entry=gmail&source=g>, >>>>>> 92275 Bois-Colombes Cedex >>>>>> RCS Nanterre 552 118 465 >>>>>> Forme Sociale : S.A.S. >>>>>> Capital Social : 657.364.587 € >>>>>> SIREN/SIRET : 552 118 465 03644 - Code NAF 6202A >>>>>> >>>>> >>>>> >>>> >>>> >>> >> >> >> -- >> Christophe >> > > > > -- > Christophe >
