Re: Savepoint a failing job

Hi Tim. > Is the only solution to just use the DataStream API? Just as Martijn mentioned, if the execution plan has been changed, it's difficult to reuse the original state to restore. Only if you are dropping some operators, then you could use -- allowNonRestoredState to restore withouting droppin

Re: Parse checkpoint _metadata file

Hi, > Is there some way to deserialize the checkpoint _metadata file? You could use some methods like SavepointLoader#loadSavepointMetadata in the State processor api to load it. > If i try to process the file with regular expressions, then approximately 90% of S3 paths of objects are actually mis

Re: Savepoint a failing job

Hi Martijn, Sorry I didn't see your response! Basically we had a bad event that was blowing up our python UDF, so we wanted to change the SQL to add a where clause that filters out the event to mitigate the issue. Our job happens to be stateless, so we're okay this time, but if we had used state (

Using TumblingEventTimeWindows on low traffic kafka topic

(Adding subject) On Wed, Dec 21, 2022 at 5:41 PM deepthi s wrote: > Hello, I am new to even-time processing and need some help. > > > > We have a kafka source with very low qps and multiple topic partitions > have no data for long periods of time. Additionally, data from the source > can come o

[no subject]

Hello, I am new to even-time processing and need some help. We have a kafka source with very low qps and multiple topic partitions have no data for long periods of time. Additionally, data from the source can come out of order (within bounds) and the application needs to process the events in o

[ANNOUNCE] Apache flink-connector-rabbitmq 3.0.0 released

The Apache Flink community is very happy to announce the release of Apache flink-connector-rabbitmq 3.0.0 This release marks the first time we have released this connector separately from the main Flink release. Over time more connectors will be migrated to this release model. This release is equ

[ANNOUNCE] Apache flink-connector-jdbc 3.0.0 released

The Apache Flink community is very happy to announce the release of Apache flink-connector-jdbc 3.0.0 This release marks the first time we have released this connector separately from the main Flink release. Over time more connectors will be migrated to this release model. This release is equival

[ANNOUNCE] Apache flink-connector-pulsar 3.0.0 released

The Apache Flink community is very happy to announce the release of Apache flink-connector-pulsar 3.0.0 This release marks the first time we have released this connector separately from the main Flink release. Over time more connectors will be migrated to this release model. This release is equiv

Parse checkpoint _metadata file

Hello All Is there some way to deserialize the checkpint _metadata file? I want to understand what the checkpoint saves and how the occupied space is distributed. If i try to process the file with regular expressions, then approximately 90% of S3 paths of objects are actually missing in the buc

Re: AsyncDataStream: Retries keep executing after timeout

Hi Yoni, Sorry for the late response! I checked the issue and it is indeed a bug, I have created a ticket(https://issues.apache.org/jira/browse/FLINK-30477) and open a pr to fix it, the reproduced case was added in AsyncWaitOperatorTest#testProcessingTimeWithTimeoutFunctionUnorderedWithRetry & te

Re: Understanding pipelined regions

Hello Gen, I do have a basic understanding of fault tolerance and how checkpointing is used to achieve it. What I'm confused about from reading the blog post [1] is what are considered blocking data exchanges. Is windowing in a streaming job considered a blocking data exchange? In a job with a DAG