callback by using process function

Hi, According to [1] example section, (i) If we check the stored count of the last modification time against the previous timestamp count, then emit the count if they (count from last modification time) match with the previous timestamp count. Is there refere about checking the previous count? a

savepoint command in code

Hello, I am trying to use the savepoint command (./bin/flink savepoint jobid) in the code instead of doing it manually in the terminal. The jobid can get using getjobid(). The problem is to define the path ./bin/flink — it can not be shown as a directory (probably because of a unix executable fi

Re: Question about snapshot file

) I am asking: is it possible to extract all savepoint files data? Thank you again On Fri, Apr 30, 2021 at 12:42 PM Abdullah bin Omar < abdullahbinoma...@gmail.com> wrote: > Thank you so much for your reply. > > I apologise I did not mention multiple savepoint files in my last &g

Re: Question about snapshot file

gain more > understanding of these aspects of flink. > > [1] > https://ci.apache.org/projects/flink/flink-docs-stable/try-flink/flink-operations-playground.html > > Best regards, > David > > On Fri, Apr 30, 2021 at 1:56 PM Abdullah bin Omar < > abdullahbinoma...@gmail.com> wr

"myuid" in snapshot.readingstate

Hi, when we readstate of of savepooint, we use, "myuid" as a argument of the function. For example, DataSet keyedState = savepoint.readKeyedState("my-uid", new ReaderFunction()); *Question 1:* In [1] (line no 79), we get the "uid" with datastream. Then in [2] (line no 45), *how can we use the

Re: Question about snapshot file

c/main/java/com/ververica/flink/training/exercises/ReadRidesAndFaresSnapshot.java > [2] > https://github.com/ververica/flink-training/blob/master/rides-and-fares/src/solution/java/org/apache/flink/training/solutions/ridesandfares/RidesAndFaresSolution.java > > Best regards, > David > &g

Re: Question about snapshot file

s another way to > accomplish what you're trying to do. > > Matthias > > [1] > https://github.com/apache/flink/blob/adaaed426c2e637b8e5ffa3f0d051326038d30aa/flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/metadata/MetadataV2V3SerializerBase.java#L83 > > On Thu, Apr 22

Re: Question about snapshot file

1326038d30aa/flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/metadata/MetadataV2V3SerializerBase.java#L83 > [2] > https://ci.apache.org/projects/flink/flink-docs-stable/dev/libs/state_processor_api.html > > On Thu, Apr 22, 2021 at 4:57 PM Abdullah bin Omar < > abdul

Question about snapshot file

Hi, (1) what 's the snapshot metadata file (binary) contains ? is it possible to read the snapshot metadata file by using Flink Deserialization? (2) is there any function that can be used to see the previous states on time of operation? Thank you

Read the metadata files (got from savepoints)

Hi, (My work is to see the state. So, I have got the save points metadata file where the state is saved) *What is the way to read the metadata files that are saved from savepoints in the local machine?* I guess the file is in binary. Thank you Regards, Abdullah

Re: Saved State in FSstate Backend

checkpoint. *Question:* *Why is discarded shown true? and why still there is no file appeared in the location of the FsStateBackend?* Thank you! Regards, Abdullah On Fri, Mar 19, 2021 at 8:26 AM Abdullah bin Omar < abdullahbinoma...@gmail.com> wrote: > Hi Yun, > > I use the

Saved State in FSstate Backend

Hi, I used the FSstate backend to save the state. I just got a folder named similar to JobID (attached image). Inside the folder, there are two more folders named by shared and task owned. However, there is nothing in those folders. How can I see the saved state? or, where is the state saved? Th

Re: Is there any complete code available for checkpointing

on, you can check out the > AsyncIOExample [1]. > > [1] > https://github.com/apache/flink/blob/328e51b44db76cdd71dfa533c96967cb742b8d53/flink-examples/flink-examples-streaming/src/main/java/org/apache/flink/streaming/examples/async/AsyncIOExample.java#L52 > > On Wed, Mar 10, 2021 at 8

Is there any complete code available for checkpointing

Hi, [1] speaks about enable checkpointing, and [2] speaks about checkpointed function. Do we need both to see a triggered and completed checkpoint (according to the pic) in the apache flink UI? Is there any complete code available for checkpointing so that we can run and see the triggered and com

Re: Trigger and completed Checkpointing do not appeared

Hi, CheckpointedFunction is needed to trigger and complete the checkpoint, is it right? What is the work for the code in here [1]? it [1] said to enable checkpointing or some other function. What is the difference between [1] and [2]? If we use ExampleCountSource[2] example, what will be the im

Re: Trigger and completed Checkpointing do not appeared

understand everything. In this case, could you please inform me where I will have to change in my code (it is in the first email) and which one to replace? If you can answer, it will be really helpful for me as a newbie. Thank you in advance! On Tue, Mar 9, 2021 at 4:07 AM Abdullah bin Omar

failure checkpoint counts

Hi, I faced this exception at the time of checkpoint counts. Could you please inform me what the problem is here? the exception: org.apache.flink.runtime.JobException: Recovery is suppressed by FixedDelayRestartBackoffTimeStrategy(maxNumberRestartAttempts=3, backoffTimeMS=100) at org.apache

Re: Compile Error

> use mvn install then, it is the closest thing to package. > > On 3/4/2021 5:12 PM, Abdullah bin Omar wrote: > > I found this one (attached pic 1) when I run the pom.xml from eclipse IDE. > I select maven clean. > > > > > > On Thu, Mar 4, 2021 at 10:00 AM Chesnay Sche

Re: Compile Error

ing output. > > Additionally, I would recommend to base your project on the quickstarts > <https://ci.apache.org/projects/flink/flink-docs-stable/dev/project-configuration.html#maven-quickstart> > . > > On 3/3/2021 4:55 PM, Abdullah bin Omar wrote: > > Hi, > > I am

Compile Error

Hi, I am running a code (Example Program) from [1], and followed the [2] for the dependencies. I used this in the pom.xml: http://maven.apache.org/POM/4.0.0 "* xmlns:xsi=*"http://www.w3.org/2001/XMLSchema-instance "* xs