Hi, I was building something that puts data in ES and found that the current connector is not compatible with an ES 8 cluster.
I found that - The currently employed `RestHighLevelClient` has been deprecated by Elastic . - There is an `ApiCompatibilityMode` you can use in the latest ES 7 RestHighLevelClient to connect to an ES 8 cluster (works on my machine). - Between the 7.10 (used now) and the 7.17 (latest) they have introduced in addition to the ApiCompatibilityMode also several breaking API changes by changing the packages of several classes we use. - If you update to the latest 7x then the `flink-connector-elasticsearch-base` can no longer be shared between ES 6 and ES 7 simply because of these package changes. - In 2022 there was still an ES 6 release but according to https://www.elastic.co/support/eol it is now EOL. So I now see several ways forward: 1) We drop ES6, update ES7 and use this compatibility mode as a quick solution. 2) We keep ES6, duplicating the `flink-connector-elasticsearch-base` (into a 6.x and 7.x version with only these few classes being different), update ES7 and use this compatibility mode as a quick solution. Later on someone should then create a connector based on the new ElasticSearch 8 Java client API. See also: https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high-compatibility.html https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/master/migrate-hlrc.html My question is - Which do we do? - I vote for the "We drop ES6, update ES7 and use this compatibility mode" - Where do we do it? - We have these connectors in both the 'flink' and 'flink-connector-elasticsearch' repos. Do we update only the new one? Or both? -- Best regards / Met vriendelijke groeten, Niels Basjes
