elasticsearch-3.1.0

Dear Team, Elasticsearch 8 support has been added for flink but not exposed yet. As per reports, elasticsearch-3.1.0 release has it and we would like to try it out. Should we manually build the code and use the jar? Is there any mvn repo link that can be used? Any sample code( I have gone throug

Re: Elasticsearch 8 - FLINK-26088

Dear Team, I see that the connector version referred to is elasticsearch-3.1.0. But I am not sure from where can I get sample code using this artifact and how to download this artifact. Any help is appreciated. Thanks, Tauseef On Tue, 18 Jun 2024 at 18:55, Tauseef Janvekar wrote: > Dear T

Elasticsearch 8 - FLINK-26088

Dear Team, As per https://issues.apache.org/jira/browse/FLINK-26088, elasticsearch 8 support is already added but I do not see it in any documentation. Also the last version that supports any elasticsearch is 1.17.x. Can I get the steps on how to integrate with elastic 8 and some sample code would

Elasticsearch8 example

Dear Team, Can anyone please share an example for flink-connector-elasticsearch8 I found this connector being added to the github. But no proper documentation is present around it. It will be of great help if a sample code is provided on the above connector. Thanks, Tauseef

Elasticsearch sink suppport

Dear Team, We are using flink 1.17.2 as of now and we use elasticsearch. Any version greater than 1.17 does not support elasticsearch sink connector or as a matter of fact any connector like mongodb, opensearch, cassandra, etc.. Is there any reason for this? What should we do to upgrade to 1.18

Elasticsearch Sink 1.17.2 error message

Hi Team, We get the below error message when we try to add an elastick sink Caused by: org.apache.flink.streaming.runtime.tasks.ExceptionInChainedOperatorException: Could not forward element to next operator at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingCh

Production deployment of Flink

Hi Al, I am using flink in my local setup and it works just fine - I installed it using confluent example training course. Here I had to manually execute start-cluster.sh and othe steps to start task managers. We installed flink on kubernetes using bitnami helm chart and it works just fine. But w

Re: Cast Exception

Dear Team, After changing the code to the below, error got resolved Map rules = alerts.entrySet().stream() .collect(Collectors.toMap(e -> (String) e.getKey(), e -> Double.parseDouble ((String)e.getValue(; Thanks, Tauseef On Tue, 5 Dec 2023 at 14:00, Tauseef Janvekar wrote: >

Cast Exception

Dear Team, I am getting cast exception in flink. Caused by: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: class java.lang.String cannot be cast to class java.lang.Double (java.lang.String and java.lang.Double are in module java.base of loader 'bootstr

Conditional multi collect in flink

Dear Team, I was wondering if there is feature in flink that support conditional multi colect. Conditons 1. A stream is being processed and being converted to another stream/List using flatMap. 2. The place where collector.collect() is being called can we have multiple other collectors also ? 3.

Re: Getting a list of tasks for a running job

Hi Yuxin, Added flink user group Thanks, Tauseef On Tue, 28 Nov 2023 at 11:38, Tauseef Janvekar wrote: > Hi Yuxin, > We have deployed it on kubernetes using helm chart - > https://github.com/bitnami/charts/blob/main/bitnami/flink/values.yaml > We have used ingress and e

Re: Job Manager and Task Manager unable to communicate

Adding flink user group On Tue, 28 Nov 2023 at 13:39, Tauseef Janvekar wrote: > Did you set some specific job manager or task manager deployment > parameters ? - No > > Did you test without the basic ingress auth ? to be sure this is not > related to that. - yes we did. And the p

Getting a list of tasks for a running job

Dear Team, How do we get list of tasks that a particular job executes. If I go toTask Manager then I do not see any tasks. I am also facing the issue where job manager is not able to access task manager but my jobs are completing with no issues. Any help is appreciated. Thanks, Tauseef

Job Manager and Task Manager unable to communicate

Dear Team, We are getting below error messages in our logs. Any help on how to resolve would be greatly appreciated. 2023-11-27 08:14:29,712 INFO org.apache.pekko.remote.transport. ProtocolStateActor [] - No response from remote for outbound association. Associate timed out after [2

Re: Confluent Kafka conection error

On Fri, 24 Nov 2023 at 15:50, Tauseef Janvekar wrote: > Hi Hang, > > I cross checked this issue multiple times. I also upgraded to flink 1.18 > but the issue persists. > > Can you please let me know a few guidelines on how to investigate this and > fix it positively. > &g

Re: Confluent Kafka conection error

Hi Hang, I cross checked this issue multiple times. I also upgraded to flink 1.18 but the issue persists. Can you please let me know a few guidelines on how to investigate this and fix it positively. Thanks, Tauseef On Thu, 23 Nov 2023 at 18:08, Tauseef Janvekar wrote: > Thanks Hang. &g

Re: Confluent Kafka conection error

; Have you ever checked whether the JM is able to access the TM? > > Best, > Hang > > Tauseef Janvekar 于2023年11月23日周四 16:04写道： > >> Dear Team, >> >> We are facing the below issue while connecting to confluent kafka >> Can som

Confluent Kafka conection error

Dear Team, We are facing the below issue while connecting to confluent kafka Can someone please help here. 2023-11-23 06:09:36,989 INFO org.apache.flink.runtime.executiongraph. ExecutionGraph [] - Source: src_source -> Sink: Print to Std. Out (1/1) (496f859d5379cd751a3fc473625125f3_cbc357c

flatmap returns a custom class object

Dear Team, I am getting the following error while using flatMap. Caused by: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: The return type of function 'defineWorkflow(OtelTransformerJob.java:75)' could not be determined automatically, due to type erasu

Java 17 as default

Dear Team, I saw the documentation for 1.18 and Java 17 is not supported and the image is created from Java11. I guess there is separate docker image for java_17. When do we plan to release main image with Java 17. Thanks, Tauseef

Elasticsearch source

Dear Team, We were looking for some elasticsearch source connector for flink and I could not get any example though. Is it possible to read from elasticsearch, technically it is not streaming but somehow it should be supported as we want some transformation logic based on elastic entries. Any he

Re: Error in /jars/upload curl request

ot sure how to proceed on this. > > > > Thanks, > > Tauseef > > > > On Tue, 7 Nov 2023 at 13:36, Yu Chen wrote: > > Hi Tauseef, > > > > The error was caused by the nginx configuration and was not a flink > problem. > > > > You can fi

Re: Error in /jars/upload curl request

t a flink > problem. > > You can find many related solutions on the web [1]. > > Best, > Yu Chen > > [1] > https://stackoverflow.com/questions/24306335/413-request-entity-too-large-file-upload-issue > > 2023年11月7日 15:14，Tauseef Janvekar 写道： > > Hi Chen, > >

Re: Error in /jars/upload curl request

The difference between @ and < is then that @ makes a file > get attached in the post as a file upload, > while the < makes a text field and just get the contents for > that text field from a file. > > > Best, > Yu Chen > -- >

Error in /jars/upload curl request

I am using curl request to upload a jar but it throws the below error [image: image.png] Received unknown attribute jarfile. Not sure what is wrong here. I am following the standard documentation https://nightlies.apache.org/flink/flink-docs-master/docs/ops/rest_api/ Please let me know if I have