Re: ElasticsearchSink in Flink 1.4.0 with Elasticsearch 5.2+

Mon, 22 Jan 2018 06:03:09 -0800

OK, thanks a lot Fabian.
Adrian
 
----- Original message -----
From: Fabian Hueske <fhue...@gmail.com>
To: Adrian Vasiliu <vasi...@fr.ibm.com>
Cc: user <user@flink.apache.org>
Subject: Re: ElasticsearchSink in Flink 1.4.0 with Elasticsearch 5.2+
Date: Mon, Jan 22, 2018 2:54 PM
 
Hi Adrian,
 
thanks for raising this issue again.
I agree, we should add support for newer ES versions.
I've added 1.5.0 as target release for FLINK-7386 and bumped the priority up.
 
In the meantime, you can try Flavio's approach (he responded to the mail thread you linked) and fork and fix the connector.
You could also try the PR for FLINK-7386 [1] and comment on the pull request whether it works for you or not.
 
Best, Fabian
 
2018-01-22 13:54 GMT+01:00 Adrian Vasiliu <vasi...@fr.ibm.com>:
Hello,
 
With a local run of Flink 1.4.0, ElasticsearchSink fails for me with a local run of Elasticsearch 5.6.4 and 5.2.1, while the same code (with adjusted versions of dependencies) works fine with Elasticsearch 2.x (tried 2.4.6).
I get:
java.lang.NoSuchMethodError: org.elasticsearch.action.bulk.BulkProcessor.add(Lorg/elasticsearch/action/ActionRequest;)Lorg/elasticsearch/action/bulk/BulkProcessor
 
(env: Mac OSX 10.13.2, oracle jdk 1.8.0_112)
 
Now, this looks similar to the issue referred in
which points to 
"Flink Elasticsearch 5 connector is not compatible with Elasticsearch 5.2+ client"

Side-remark: when trying with Elasticsearch 5.6.4 via a docker container, for some reason the error I get is different: "RuntimeException: Client is not connected to any Elasticsearch nodes!" (while Elasticsearch 2.4.6 works fine via docker too).
 
FLINK-7386 being pending since August 2017, would it mean that there is nowadays still no way to make Flink 1.4.0's sink work with Elasticsearch 5.2+? My use-case involves Compose for Elasticsearch 5.6.3, shared by different apps, and I can't really downgrade its Elasticsearch version.
Or would there be signs it will be fixed in Flink 1.5.0?
 
Any lights welcome.
 
Thanks,
Adrian
 
 
Sauf indication contraire ci-dessus:/ Unless stated otherwise above:
Compagnie IBM France
Siège Social : 17 avenue de l'Europe, 92275 Bois-Colombes Cedex
RCS Nanterre 552 118 465
Forme Sociale : S.A.S.
Capital Social : 657.364.587 €
SIREN/SIRET : 552 118 465 03644 - Code NAF 6202A
 
Sauf indication contraire ci-dessus:/ Unless stated otherwise above:
Compagnie IBM France
Siège Social : 17 avenue de l'Europe, 92275 Bois-Colombes Cedex
RCS Nanterre 552 118 465
Forme Sociale : S.A.S.
Capital Social : 657.364.587 €
SIREN/SIRET : 552 118 465 03644 - Code NAF 6202A

Reply via email to