Hi all,
Just following up on this thread (+dev) regarding recommendations for using the Elasticsearch 8.x connector. I haven’t seen any artifacts related to it available in Maven, and it appears there’s an Ali Baba published one that looks similar (but I’d obviously prefer to the use official one from Flink directly).
What’s the current approach to using the connector? Are most people forking the repository? Or building it themselves and uploading it to something like an Artifactory to use? I’m just curious what/how others are using it.
Any advice would be appreciated!
Thanks,
Rion On Jul 31, 2024, at 9:39 AM, Rion Williams <rionmons...@gmail.com> wrote:
Hi all,
I’m bubbling this request up from a thread that I had started on both the Flink Slack channel as well as the User mailing list . I recently had the need to upgrade a series of my Flink jobs that heavily interact with Elasticsearch and noticed that many started failing after a rather unexpected cluster upgrade (due to incompatibility with the RHLC).
I’ve been trying to find a recommendation for how to go about using or referencing it similar to previous connectors (ala Maven) but it appears artifacts aren’t yet published for it. I’m not sure what the schedule looks like for doing so, as I’d prefer not to fork the ES 8.x connector bits if possible.
Please advise and thanks in advance,
Rion Williams
Begin forwarded message:
From: Ahmed Hamdy <hamdy10...@gmail.com>
Date: July 31, 2024 at 9:18:18 AM CDT
To: Rion Williams <rionmons...@gmail.com>
Cc: Flink User List <user@flink.apache.org>
Subject: Re: Elasticsearch 8.x Connector in Maven
Hi Rion It seems that ES 8 was supported ahead of 3.1 release[1], which seems to not be released yet hence not published to maven. Given the importance of ES 8 and the fact that elastic search still depends on Flink 1.18 while we are releasing 1.20, I would suggest nudging the dev list[2] for a release manager to pick up the 3.1 release.
Hi again all,
Just following up on this as I’ve scoured around trying to find any documentation for using the ES 8.x connector, however everything only appears to reference 6/7.
The ES 8.x seems to have been released for quite a bit of time, so I’m curious how others are using it. I’d really like to avoid doing something like forking the bits I need in my local repository if possible and building it on my own.
Thanks in advance,
Rion
> On Jul 30, 2024, at 1:00 PM, Rion Williams <rionmons...@gmail.com> wrote:
>
> Hi all,
>
> I see that the Elasticsearch Connector for 8.x is supported per the repo (and completed JIRAs). Is there a way to reference this via Maven? Or is it required to build the connector from the source directly?
>
> We recently upgraded an Elasticsearch cluster to 8.x and some of the writes are now failing from the 7.x RHLC, so trying to upgrade to get to parity (and stop the jobs from crashing).
>
> Thanks in advance,
>
> Rion
|
