Re: Re: Savepoints with bootstraping a datastream function

Thanks for the information Yun! I will go with the workaround then. On Fri, 17 Sept 2021 at 15:22, Yun Gao wrote: > Hi Rakshit, > > I think FLIP-147 might still not be able to support this case, since > for bounded jobs, it supports each task exit after a checkpoint to > commit the remaining dat

Re: hdfs lease issues on flink retry

Hi, AttemptId needs to be an integer (take a look at TaskAttemptID class for more details). As for your prior question, any random id should solve the issue. Best, D. On Mon 20. 9. 2021 at 22:32, Shah, Siddharth wrote: > Hi Matthias, > > > > Thanks for suggesting a workaround, but our jobs fail

RE: hdfs lease issues on flink retry

Hi Matthias, Thanks for suggesting a workaround, but our jobs fail with below error when I apply that change Caused by: java.lang.IllegalArgumentException: TaskAttemptId string : attempt___r_01_1632168799349 is not properly formed at org.apache.hadoop.mapreduce.TaskAttemptID.fo

Re: Exception by flink kafka

Hi Ragini, On Fri, Sep 17, 2021 at 1:40 PM Ragini Manjaiah wrote: > Later I started encountering org.apache.kafka.common.errors.TimeoutException: > Failed to update metadata after 6 ms. > This message can have several causes. There may be network issues, your Kafka configuration might be br

Python statefun - Context update

Hi all ! I am currently developing an environment that uses Flink Stateful functions in Python. The architecture is complex but here are the main points that needs to be understood to frame the problem I am facing. * functions.py contains several methods, one of them is handling protobuf messag

Re: Built-in functions to manipulate MULTISET type

The type strategy can be generic over the input and output types, so you can write something generic that say given a multiset of some type T this function returns an array of some type T. This is the exact same logic built-in functions use and is just as expressive as anything Flink could provide.

ApacheCon starts tomorrow!

ApacheCon @Home starts tomorrow! Details at https://www.apachecon.com/acah2021/index.html (Note: You're receiving this because you are subscribed to one or more user lists for Apache Software Foundation projects.) We've got three days of great content lined up for you, spanning 14 project co

RE: Flink operator stuck on created

Hi Dave, In batch mode an operator/task only starts running once all input tasks are finished. So without further detail this is perfectly in line with what you describe. Thias From: Dave Maughan Sent: Montag, 20. September 2021 13:15 To: user@flink.apache.org Subject: Re: Flink operator stu

Re: Flink operator stuck on created

I should note - this job is being run in batch mode. Could there be a deadlock related to FLINK-16430? On Mon, 20 Sept 2021 at 11:26, Dave Maughan wrote: > Hi, > > I have a Flink job on EMR with an operator stuck on CREATED. The subtasks > are not being assigned to task manager slots. The previo

Flink operator stuck on created

Hi, I have a Flink job on EMR with an operator stuck on CREATED. The subtasks are not being assigned to task manager slots. The previous operator is running and has non-zero Bytes Sent and Records Sent. When the job started the Job manager requested new workers to start a bunch of the operators bu

Re: hdfs lease issues on flink retry

I don't know of any side effects of your approach. But another workaround I saw was replacing the _0 suffix by something like "_" + System.currentMillis() On Fri, Sep 17, 2021 at 8:38 PM Shah, Siddharth wrote: > Hi Matthias, > > > > Thanks for looking into the issue and creating a ticket. I am t