Re: [ANNOUNCE] New Apache Flink PMC Member - Matthias Pohl

2023-08-06 Thread Andriy Redko
Congrats Matthias, well deserved!!

DC> Congrats Matthias!

DC> Very well deserved, thankyou for your continuous, consistent contributions.
DC> Welcome.

DC> Thanks,
DC> Danny

DC> On Fri, Aug 4, 2023 at 9:30 AM Feng Jin  wrote:

>> Congratulations, Matthias!
>>
>> Best regards
>>
>> Feng
>>
>> On Fri, Aug 4, 2023 at 4:29 PM weijie guo 
>> wrote:
>>
>> > Congratulations, Matthias!
>> >
>> > Best regards,
>> >
>> > Weijie
>> >
>> >
>> > Wencong Liu  于2023年8月4日周五 15:50写道:
>> >
>> > > Congratulations, Matthias!
>> > >
>> > > Best,
>> > > Wencong Liu
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > At 2023-08-04 11:18:00, "Xintong Song"  wrote:
>> > > >Hi everyone,
>> > > >
>> > > >On behalf of the PMC, I'm very happy to announce that Matthias Pohl
>> has
>> > > >joined the Flink PMC!
>> > > >
>> > > >Matthias has been consistently contributing to the project since Sep
>> > 2020,
>> > > >and became a committer in Dec 2021. He mainly works in Flink's
>> > distributed
>> > > >coordination and high availability areas. He has worked on many FLIPs
>> > > >including FLIP195/270/285. He helped a lot with the release
>> management,
>> > > >being one of the Flink 1.17 release managers and also very active in
>> > Flink
>> > > >1.18 / 2.0 efforts. He also contributed a lot to improving the build
>> > > >stability.
>> > > >
>> > > >Please join me in congratulating Matthias!
>> > > >
>> > > >Best,
>> > > >
>> > > >Xintong (on behalf of the Apache Flink PMC)
>> > >
>> >
>>



Re: [VOTE] Release flink-connector-opensearch v1.1.0, release candidate #1

2023-11-03 Thread Andriy Redko
+1 (non-binding), thanks a lot Danny!

Best Regards,
Andriy Redko 

> Hi everyone,

> Please review and vote on the release candidate #1 for the version 1.1.0 of
> flink-connector-opensearch, as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)


> The complete staging area is available for your review, which includes:
> * JIRA release notes [1],
> * the official Apache source release to be deployed to dist.apache.org [2],
> which are signed with the key with fingerprint
> 0F79F2AFB2351BC29678544591F9C1EC125FD8DB [3],
> * all artifacts to be deployed to the Maven Central Repository [4],
> * source code tag v1.1.0-rc1 [5],
> * website pull request listing the new release [6].

> The vote will be open for at least 72 hours. It is adopted by majority
> approval, with at least 3 PMC affirmative votes.

> Thanks,
> Danny

> [1]
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12353141
> [2]
> https://dist.apache.org/repos/dist/dev/flink/flink-connector-opensearch-1.1.0-rc1/
> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> [4] https://repository.apache.org/content/repositories/orgapacheflink-1666/
> [5] https://github.com/apache/flink-connector-opensearch/tree/v1.1.0-rc1
> [6] https://github.com/apache/flink-web/pull/694



[DICSUSS] Releasing Flink Opensearch Connector v1.0.0

2022-12-15 Thread Andriy Redko
Hello All,

I would like to start discussing the first release of the new Flink
Opensearch Connector [1] 1.0.0. As per the connector versioning strategy
[2] from the new flink-connector-opensearch repository [3]. As of today,
no outstanding issues are left for the first release to happen.

I propose that we release 1.0.0 with support for Flink 1.16 (1.0.0-1.16). 
Since this is a new connector (has never been part of the Flink distribution),
it probably makes sense to start from version 1.0.0 but suggestions here are 
more than welcome.

I would appreciate if PMC member volunteers to step in as the release manager.

[1] 
https://cwiki.apache.org/confluence/display/FLINK/FLIP-243%3A+Dedicated+Opensearch+connectors
[2] 
https://cwiki.apache.org/confluence/display/FLINK/Externalized+Connector+development
[3] https://github.com/apache/flink-connector-opensearch

Thanks!

Best Regards,
    Andriy Redko
      
 



Re: [DICSUSS] Releasing Flink Opensearch Connector v1.0.0

2022-12-15 Thread Andriy Redko
Thanks a lot Danny. The connector builds against 1.16 and 1.17 (snapshot) 
right now, I think we could explore 1.15 support. The Opensearch is backed 
by AWS as of today, right, but it is a community driven project under 
OpenSearch Project organization, so I would not tight it to AWS anyhow (and 
your are correct, it does not use any AWS SDKs). Hope it makes sense, thank 
you.

Best Regards,
Andriy Redko

DC> Hey,

DC> I will also volunteer to be release manager, since Martijn has been doing
DC> all of them I am happy to take some of the work!

DC> We launched the new flink-connector-dynamodb connector with support for
DC> Flink 1.15. The versioning strategy says to support the latest 2 Flink
DC> versions [1]. If this connector builds against both 1.15/1.16 we should
DC> consider supporting both.

DC> Given that Opensearch is AWS did we consider putting this in
DC> flink-connector-aws? Since it does not use the AWS SDK it makes sense to
DC> split, but wonder if this was already discussed?

DC> Thanks,
DC> Danny

DC> [1]
DC> 
https://cwiki.apache.org/confluence/display/FLINK/Externalized+Connector+development

DC> On Thu, Dec 15, 2022 at 3:19 PM Martijn Visser 
DC> wrote:

>> Hi Andriy,
>>
>> +1 to release the first version. Happy to help with the release (feels like
>> I'm doing nothing else anyway :D)
>>
>> I would indeed go for version 1.0.0 to start since it's a new connector.
>>
>> Cheers,
>>
>> Martijn
>>
>> On Thu, Dec 15, 2022 at 4:16 PM Andriy Redko  wrote:
>>
>> > Hello All,
>> >
>> > I would like to start discussing the first release of the new Flink
>> > Opensearch Connector [1] 1.0.0. As per the connector versioning strategy
>> > [2] from the new flink-connector-opensearch repository [3]. As of today,
>> > no outstanding issues are left for the first release to happen.
>> >
>> > I propose that we release 1.0.0 with support for Flink 1.16 (1.0.0-1.16).
>> > Since this is a new connector (has never been part of the Flink
>> > distribution),
>> > it probably makes sense to start from version 1.0.0 but suggestions here
>> > are
>> > more than welcome.
>> >
>> > I would appreciate if PMC member volunteers to step in as the release
>> > manager.
>> >
>> > [1]
>> >
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-243%3A+Dedicated+Opensearch+connectors
>> > [2]
>> >
>> https://cwiki.apache.org/confluence/display/FLINK/Externalized+Connector+development
>> > [3] https://github.com/apache/flink-connector-opensearch
>> >
>> > Thanks!
>> >
>> > Best Regards,
>> > Andriy Redko
>> >
>> >
>> >
>> >
>>



Re: [VOTE] Release flink-connector-elasticsearch, release candidate #1

2023-04-13 Thread Andriy Redko
+1 (non-binding), thank you Martijn! 

MV> Hi everyone,
MV> Please review and vote on the release candidate #1 for the version 3.0.1,
MV> as follows:
MV> [ ] +1, Approve the release
MV> [ ] -1, Do not approve the release (please provide specific comments)

MV> This version supports both Flink 1.16.x and Flink 1.17.x

MV> The complete staging area is available for your review, which includes:
MV> * JIRA release notes [1],
MV> * the official Apache source release to be deployed to dist.apache.org [2],
MV> which are signed with the key with fingerprint
MV> A5F3BCE4CBE993573EC5966A65321B8382B219AF [3],
MV> * all artifacts to be deployed to the Maven Central Repository [4],
MV> * source code tag v3.0.1-rc1 [5],
MV> * website pull request listing the new release [6].

MV> The vote will be open for at least 72 hours. It is adopted by majority
MV> approval, with at least 3 PMC affirmative votes.

MV> Thanks,
MV> Release Manager

MV> [1]
MV> 
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12352521
MV> [2]
MV> 
https://dist.apache.org/repos/dist/dev/flink/flink-connector-elasticsearch-3.0.1-rc1
MV> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
MV> [4] https://repository.apache.org/content/repositories/orgapacheflink-1609/
MV> [5]
MV> 
https://github.com/apache/flink-connector-elasticsearch/releases/tag/v3.0.1-rc1
MV> [6] https://github.com/apache/flink-web/pull/634



Re: Access to Apache Flink wiki (FLIP submission)

2022-06-16 Thread Andriy Redko
Hi Folks,

Gentle reminder please, :-) I would like to ask for write permissions to
Apache Flink wiki space in order to create a FLIP proposal (username is reta). 
Thank you in advance.

Best Regards,
Andriy Redko

rao> Hi Folks,

rao> Following the FLIP submission process, I would like to ask for write 
permissions to
rao> Apache Flink wiki space in order to create a FLIP proposal (username is 
reta). Thank you
rao> in advance.

rao> Best Regards,
rao> Andriy Redko



[DISCUSS] FLIP-243: Dedicated Opensearch connectors

2022-06-22 Thread Andriy Redko
Hi Folks, 

We would like to start a discussion thread on FLIP-243: Dedicated Opensearch 
connectors [1], [2] where we propose to provide dedicated connectors for 
Opensearch [3], on par with existing Elasticsearch ones. Looking forward to 
comments and feedback. Thank you.

[1] 
https://cwiki.apache.org/confluence/display/FLINK/FLIP-243%3A+Dedicated+Opensearch+connectors
[2] https://github.com/apache/flink/pull/18541 
[3] https://opensearch.org/

Best Regards,
Andriy Redko



Re: [DISCUSS] FLIP-243: Dedicated Opensearch connectors

2022-07-04 Thread Andriy Redko
Hi Martijn,

Thanks a lot supporting the FLIP! Regarding your question, yes, Elasticsearch
has a new client in v8 (deprecating HighLevelRestClient from now on) and, no, 
there won't be any impact on Opensearch (the connectors use Opensearch's
own HighLevelRestClient, no deprecations yet). 

Thank you.

Best Regards,
    Andriy Redko

MV> Hi Andriy,

MV> Thanks for creating the FLIP and opening the discussion. In general +1 for
MV> the FLIP, looking forward!

MV> One question: I recall that in order for Flink to support Elasticsearch 8,
MV> the connector needs to be overhauled to use Elastic's new client because
MV> only the new client has a compatible license. Will that impact the
MV> Opensearch connector in any way?

MV> Best regards,

MV> Martijn

MV> Op wo 22 jun. 2022 om 22:59 schreef Andriy Redko :

>> Hi Folks,
>>
>> We would like to start a discussion thread on FLIP-243: Dedicated
>> Opensearch connectors [1], [2] where we propose to provide dedicated
>> connectors for Opensearch [3], on par with existing Elasticsearch ones.
>> Looking forward to comments and feedback. Thank you.
>>
>> [1]
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-243%3A+Dedicated+Opensearch+connectors
>> [2] https://github.com/apache/flink/pull/18541
>> [3] https://opensearch.org/
>>
>> Best Regards,
>> Andriy Redko
>>
>>



Re: [DISCUSS] FLIP-243: Dedicated Opensearch connectors

2022-07-13 Thread Andriy Redko
Hi Martijn,

Awesome, will do that shortly, thank you!

Best Regards,
Andriy Redko

> Hi Andriy,

> Thanks for that. If there's no more comments I think you can open up a vote 
> thread.

> Best regards,

> Martijn

> Op ma 4 jul. 2022 om 18:32 schreef Andriy Redko :

>> Hi Martijn,

>> Thanks a lot supporting the FLIP! Regarding your question, yes, Elasticsearch
>> has a new client in v8 (deprecating HighLevelRestClient from now on) and, 
>> no, 
>> there won't be any impact on Opensearch (the connectors use Opensearch's
>> own HighLevelRestClient, no deprecations yet). 

>> Thank you.

>> Best Regards,
>>     Andriy Redko

MV>>> Hi Andriy,

MV>>> Thanks for creating the FLIP and opening the discussion. In general +1 for
MV>>> the FLIP, looking forward!

MV>>> One question: I recall that in order for Flink to support Elasticsearch 8,
MV>>> the connector needs to be overhauled to use Elastic's new client because
MV>>> only the new client has a compatible license. Will that impact the
MV>>> Opensearch connector in any way?

MV>>> Best regards,

MV>>> Martijn

MV>>> Op wo 22 jun. 2022 om 22:59 schreef Andriy Redko :

>>>> Hi Folks,
>>>>
>>>> We would like to start a discussion thread on FLIP-243: Dedicated
>>>> Opensearch connectors [1], [2] where we propose to provide dedicated
>>>> connectors for Opensearch [3], on par with existing Elasticsearch ones.
>>>> Looking forward to comments and feedback. Thank you.
>>>>
>>>> [1]
>>>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-243%3A+Dedicated+Opensearch+connectors
>>>> [2] https://github.com/apache/flink/pull/18541
>>>> [3] https://opensearch.org/
>>>>
>>>> Best Regards,
>>>>     Andriy Redko
>>>>
>>>>



[VOTE] FLIP-243: Dedicated Opensearch connectors

2022-07-13 Thread Andriy Redko
Hey Folks,

Thanks a lot for all the feedback and comments so far. Based on the discussion 
[1], 
it seems like there is a genuine interest in supporting OpenSearch [2] 
natively. With 
that being said, I would like to start a vote on FLIP-243 [3].

The vote will last for at least 72 hours unless there is an objection or
insufficient votes.

Thank you!

[1] https://lists.apache.org/thread/jls0vqc7jb84jp14j4jok1pqfgo2cl30
[2] https://opensearch.org/
[3] 
https://cwiki.apache.org/confluence/display/FLINK/FLIP-243%3A+Dedicated+Opensearch+connectors


Best Regards,
Andriy Redko



[RESULT][VOTE] FLIP-243: Dedicated Opensearch connectors

2022-09-02 Thread Andriy Redko
Hello Everybody,

I'm happy to announce that FLIP-243: Dedicated Opensearch connectors [1] 
has been accepted [2] with 5 positive votes (with 3 binding) and 
no negative votes:

- Ryan Skraba (non-binding)
- Hong Teoh (non-binding)
- Matthias Pohl (binding)
- Martijn Visser (binding)
- Danny Cranmer (binding)

Huge thanks to everyone for discussions [3] and contributions to this FLIP. 

Best Regards,
Andriy Redko

[1] 
https://cwiki.apache.org/confluence/display/FLINK/FLIP-243%3A+Dedicated+Opensearch+connectors
[2] https://www.mail-archive.com/dev@flink.apache.org/msg58911.html
[3] https://lists.apache.org/thread/jls0vqc7jb84jp14j4jok1pqfgo2cl30



HELP: New repository 'flink-connector-opensearch' under ASF organization

2022-09-02 Thread Andriy Redko
Hello Everyone,

Kindly asking PMCs for help with creating new 'flink-connector-opensearch' 
repository 
under ASF organization to host Apache Flink OpenSearch connector(s) [1], [2], 
[3], [4], 
following the initiative to move connectors out of core. 

Thank you very much in advance.

Best Regards,
    Andriy Redko


[1] 
https://cwiki.apache.org/confluence/display/FLINK/FLIP-243%3A+Dedicated+Opensearch+connectors
[2] https://www.mail-archive.com/dev@flink.apache.org/msg58911.html
[3] https://lists.apache.org/thread/jls0vqc7jb84jp14j4jok1pqfgo2cl30
[4] https://lists.apache.org/thread/4bms24983g38q956rp8qmm4bpdo4361s



Re: [ANNOUNCE] New Apache Flink Committer - Sergey Nuyanzin

2025-04-03 Thread Andriy Redko
Congrats Sergey!!!
 
  
> Hi everyone,

> On behalf of the PMC, I'm very happy to announce Sergey Nuyanzin as a new 
> Flink Committer.

> Sergey started contributing small improvements to the project in 2018. Over 
> the past 1.5 years, he has become more active and focused on adding and 
> reviewing changes to the Flink SQL ecosystem.

> Currently, he is upgrading Flink's SQL engine to the latest Apache Calcite 
> version [1][2][3] and helps in updating other project-wide dependencies as 
> well.

> Please join me in congratulating Sergey Nuyanzin for becoming a Flink 
> Committer!

> Best,
> Timo Walther (on behalf of the Flink PMC)

> [1] https://issues.apache.org/jira/browse/FLINK-29932
> [2] https://issues.apache.org/jira/browse/FLINK-21239
> [3] https://issues.apache.org/jira/browse/FLINK-20873



[jira] [Created] (FLINK-32209) Opensearch connector should remove the dependency on flink-shaded

2023-05-26 Thread Andriy Redko (Jira)
Andriy Redko created FLINK-32209:


 Summary: Opensearch connector should remove the dependency on 
flink-shaded
 Key: FLINK-32209
 URL: https://issues.apache.org/jira/browse/FLINK-32209
 Project: Flink
  Issue Type: Bug
  Components: Connectors / Opensearch
Affects Versions: opensearch-1.0.1
Reporter: Andriy Redko
 Fix For: opensearch-1.1.0






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (FLINK-30352) [Connectors][Elasticsearch] Document missing configuration properties

2022-12-09 Thread Andriy Redko (Jira)
Andriy Redko created FLINK-30352:


 Summary: [Connectors][Elasticsearch] Document missing 
configuration properties
 Key: FLINK-30352
 URL: https://issues.apache.org/jira/browse/FLINK-30352
 Project: Flink
  Issue Type: Bug
  Components: Connectors / ElasticSearch
Affects Versions: 1.15.3, 1.16.0
Reporter: Andriy Redko


There is a number of configuration properties which are not documented:

- sink.delivery-guarantee
- connection.request-timeout
- connection.timeout
- socket.timeout
 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (FLINK-30488) OpenSearch implementation of Async Sink

2022-12-22 Thread Andriy Redko (Jira)
Andriy Redko created FLINK-30488:


 Summary: OpenSearch implementation of Async Sink
 Key: FLINK-30488
 URL: https://issues.apache.org/jira/browse/FLINK-30488
 Project: Flink
  Issue Type: New Feature
  Components: Connectors / Opensearch
Affects Versions: opensearch-1.0.0
Reporter: Andriy Redko


* Implement an asynchronous sink (Async Sink) support for OpenSearch connector
 * Update documentation
 * Add end to end tests

h2. References

More details to be found 
[https://cwiki.apache.org/confluence/display/FLINK/FLIP-171%3A+Async+Sink]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (FLINK-30718) Cannot assign instance of java.lang.invoke.SerializedLambda to field org.apache.flink.connector.opensearch.sink.OpensearchSink.emitter

2023-01-17 Thread Andriy Redko (Jira)
Andriy Redko created FLINK-30718:


 Summary: Cannot assign instance of 
java.lang.invoke.SerializedLambda to field 
org.apache.flink.connector.opensearch.sink.OpensearchSink.emitter 
 Key: FLINK-30718
 URL: https://issues.apache.org/jira/browse/FLINK-30718
 Project: Flink
  Issue Type: Bug
  Components: Connectors / Opensearch
Affects Versions: 1.16.0
Reporter: Andriy Redko


When using OpenSearchSink programmatically
{noformat}
 
final StreamExecutionEnvironment env = StreamExecutionEnvironment
.createRemoteEnvironment("localhost", 8081);

final Collection> users = new 
ArrayList<>();
users.add(Tuple4.of("u1", "admin", 100L, 200L));

final DataStream> source = 
env.fromCollection(users);
final OpensearchSink> sink =
new OpensearchSinkBuilder>()
.setHosts(new HttpHost("localhost", 9200, "https"))
.setEmitter( (element, ctx, indexer) -> {
indexer.add(
Requests
.indexRequest()
.index("users")
.id(element.f0)
.source(Map.ofEntries(
Map.entry("user_id", element.f0),
Map.entry("user_name", element.f1),
Map.entry("uv", element.f2),
Map.entry("pv", element.f3)
)));
})
.setConnectionUsername("admin")
.setConnectionPassword("admin")
.setAllowInsecure(true)
.setBulkFlushMaxActions(1)
.build();

source.sinkTo(sink);
env.execute("Opensearch end to end sink test example");

{noformat}
the stream processing fails with the exception{color:#00}

{color}
{noformat}
Caused by: org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot 
instantiate user function.
at 
org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperatorFactory(StreamConfig.java:399)
at 
org.apache.flink.streaming.runtime.tasks.OperatorChain.(OperatorChain.java:162)
at 
org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.(RegularOperatorChain.java:60)
at 
org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(StreamTask.java:681)
at 
org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:669)
at 
org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:935)
at 
org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:904)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:728)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:550)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.ClassCastException: cannot assign instance of 
java.lang.invoke.SerializedLambda to field 
org.apache.flink.connector.opensearch.sink.OpensearchSink.emitter of type 
org.apache.flink.connector.opensearch.sink.OpensearchEmitter in instance of 
org.apache.flink.connector.opensearch.sink.OpensearchSink
at 
java.base/java.io.ObjectStreamClass$FieldReflector.setObjFieldValues(ObjectStreamClass.java:2076)
at 
java.base/java.io.ObjectStreamClass$FieldReflector.checkObjectFieldValueTypes(ObjectStreamClass.java:2039)
at 
java.base/java.io.ObjectStreamClass.checkObjFieldValueTypes(ObjectStreamClass.java:1293)
at 
java.base/java.io.ObjectInputStream.defaultCheckFieldValues(ObjectInputStream.java:2512)
at 
java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2419)
at 
java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2228)
at 
java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1687)
at 
java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2496)
at 
java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2390)
at 
java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2228)
at 
java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1687)
at 
java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:489)
at 
java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:447)
at 
org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:617)
at 
org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:602)
at 
org.apache.flink.util.InstantiationUtil.deseri

[jira] [Created] (FLINK-31068) Document how to use Opensearch connector with OpenSearch 1.x / 2.x / ... clusters

2023-02-14 Thread Andriy Redko (Jira)
Andriy Redko created FLINK-31068:


 Summary: Document how to use Opensearch connector with OpenSearch 
1.x / 2.x / ... clusters
 Key: FLINK-31068
 URL: https://issues.apache.org/jira/browse/FLINK-31068
 Project: Flink
  Issue Type: Improvement
  Components: Connectors / Opensearch
Affects Versions: opensearch-1.0.0
Reporter: Andriy Redko


By default, Opensearch connector uses the OpenSearch client from 1.3.x release 
line (due to Apache Flink's JDK-8 baseline requirement). However, Flink 
Opensearch Connector is fully compatible with 1.x / 2.x / 3.x (upcoming) 
release lines. The documentation does not describe the way how to switch off 
the Opensearch client libraries.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (FLINK-25756) Dedicated Opensearch connectors

2022-01-21 Thread Andriy Redko (Jira)
Andriy Redko created FLINK-25756:


 Summary: Dedicated Opensearch connectors
 Key: FLINK-25756
 URL: https://issues.apache.org/jira/browse/FLINK-25756
 Project: Flink
  Issue Type: Improvement
Reporter: Andriy Redko


Since the time Opensearch got forked from Elasticsearch a few things got 
changed. The projects evolve in different directions, the Elasticsearch clients 
up to 7.13.x were able to connect to Opensearch clusters, but since 7.14 - not 
anymore [1] (Elastic continues to harden their clients to connect to 
Elasticsearch clusters only).

For example, running current Flink master against Opensearch clusters using 
Elalsticsearch 7 connectors would fail with:

 

```
{color:#008080}Caused{color}{color:#00} 
{color}{color:#ff}by{color}{color:#00}: 
{color}{color:#008080}ElasticsearchException{color}{color:#00}[{color}{color:#008080}Elasticsearch{color}{color:#00}
 version {color}{color:#098658}6{color}{color:#00} 
{color}{color:#ff}or{color}{color:#00} more is required]{color}
{color:#00} at 
org.apache.flink.elasticsearch7.shaded.org.elasticsearch.client.{color}{color:#008080}RestHighLevelClient$4{color}{color:#00}.onResponse({color}{color:#008080}RestHighLevelClient{color}{color:#00}.java:{color}{color:#098658}2056{color}{color:#00}){color}
{color:#00} at 
org.apache.flink.elasticsearch7.shaded.org.elasticsearch.client.{color}{color:#008080}RestHighLevelClient$4{color}{color:#00}.onResponse({color}{color:#008080}RestHighLevelClient{color}{color:#00}.java:{color}{color:#098658}2043{color}{color:#00}){color}
{color:#00} at 
org.apache.flink.elasticsearch7.shaded.org.elasticsearch.common.util.concurrent.{color}{color:#008080}ListenableFuture{color}{color:#00}.notifyListenerDirectly({color}{color:#008080}ListenableFuture{color}{color:#00}.java:{color}{color:#098658}113{color}{color:#00}){color}
{color:#00} at 
org.apache.flink.elasticsearch7.shaded.org.elasticsearch.common.util.concurrent.{color}{color:#008080}ListenableFuture{color}{color:#00}.done({color}{color:#008080}ListenableFuture{color}{color:#00}.java:{color}{color:#098658}100{color}{color:#00}){color}
{color:#00} at 
org.apache.flink.elasticsearch7.shaded.org.elasticsearch.common.util.concurrent.{color}{color:#008080}BaseFuture{color}{color:#00}.{color}{color:#ff}set{color}{color:#00}({color}{color:#008080}BaseFuture{color}{color:#00}.java:{color}{color:#098658}133{color}{color:#00}){color}
{color:#00} at 
org.apache.flink.elasticsearch7.shaded.org.elasticsearch.common.util.concurrent.{color}{color:#008080}ListenableFuture{color}{color:#00}.onResponse({color}{color:#008080}ListenableFuture{color}{color:#00}.java:{color}{color:#098658}139{color}{color:#00}){color}
{color:#00} at 
org.apache.flink.elasticsearch7.shaded.org.elasticsearch.client.{color}{color:#008080}RestHighLevelClient$5{color}{color:#00}.onSuccess({color}{color:#008080}RestHighLevelClient{color}{color:#00}.java:{color}{color:#098658}2129{color}{color:#00}){color}
{color:#00} at 
org.apache.flink.elasticsearch7.shaded.org.elasticsearch.client.{color}{color:#008080}RestClient$FailureTrackingResponseListener{color}{color:#00}.onSuccess({color}{color:#008080}RestClient{color}{color:#00}.java:{color}{color:#098658}636{color}{color:#00}){color}
{color:#00} at 
org.apache.flink.elasticsearch7.shaded.org.elasticsearch.client.{color}{color:#008080}RestClient$1{color}{color:#00}.completed({color}{color:#008080}RestClient{color}{color:#00}.java:{color}{color:#098658}376{color}{color:#00}){color}
{color:#00} at 
org.apache.flink.elasticsearch7.shaded.org.elasticsearch.client.{color}{color:#008080}RestClient$1{color}{color:#00}.completed({color}{color:#008080}RestClient{color}{color:#00}.java:{color}{color:#098658}370{color}{color:#00}){color}
{color:#00} at 
org.apache.flink.elasticsearch7.shaded.org.apache.http.concurrent.{color}{color:#008080}BasicFuture{color}{color:#00}.completed({color}{color:#008080}BasicFuture{color}{color:#00}.java:{color}{color:#098658}122{color}{color:#00}){color}
{color:#00} at 
org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.client.{color}{color:#008080}DefaultClientExchangeHandlerImpl{color}{color:#00}.responseCompleted({color}{color:#008080}DefaultClientExchangeHandlerImpl{color}{color:#00}.java:{color}{color:#098658}181{color}{color:#00}){color}
{color:#00} at 
org.apache.flink.elasticsearch7.shaded.org.apache.http.nio.protocol.{color}{color:#008080}HttpAsyncRequestExecutor{color}{color:#00}.processResponse({color}{color:#008080}HttpAsyncRequestExecutor{color}{color:#00}.java:{color}{color:#098658}448{color}{color:#00}){color}
{color:#00} at 
org.apache.flink.elasticsearch7

[jira] [Created] (FLINK-25961) Remove transport client from Elasticsearch 6/7 connectors (tests only)

2022-02-04 Thread Andriy Redko (Jira)
Andriy Redko created FLINK-25961:


 Summary: Remove transport client from Elasticsearch 6/7 connectors 
(tests only)
 Key: FLINK-25961
 URL: https://issues.apache.org/jira/browse/FLINK-25961
 Project: Flink
  Issue Type: Improvement
  Components: Connectors / ElasticSearch
Reporter: Andriy Redko


The Elasticsearch 6/7 connectors still use deprecated transport client for 
integration tests. This is not really necessary and brings a lot of 
dependencies, the HighLevelRestClient is fully sufficient and could be used as 
drop-in replacement.

The change affects only tests.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)


[jira] [Created] (FLINK-36633) Add support for Flink 1.20 in Flink OpenSearch connector

2024-10-30 Thread Andriy Redko (Jira)
Andriy Redko created FLINK-36633:


 Summary: Add support for Flink 1.20 in Flink OpenSearch connector
 Key: FLINK-36633
 URL: https://issues.apache.org/jira/browse/FLINK-36633
 Project: Flink
  Issue Type: Bug
  Components: Connectors / Opensearch
Reporter: Andriy Redko






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (FLINK-37652) [Connectors/Opensearch] Move to Flink to 2.0.0

2025-04-10 Thread Andriy Redko (Jira)
Andriy Redko created FLINK-37652:


 Summary: [Connectors/Opensearch] Move to Flink to 2.0.0
 Key: FLINK-37652
 URL: https://issues.apache.org/jira/browse/FLINK-37652
 Project: Flink
  Issue Type: Bug
  Components: Connectors / Opensearch
Reporter: Andriy Redko
 Fix For: 3.0.0






--
This message was sent by Atlassian Jira
(v8.20.10#820010)