Hi Kartik,
It should be the other way around: the connector should use the proper
Source and Sink interfaces, and therefore get the right guarantees and
integration with mechanisms like checkpoints and savepoints. I would
say there's no other way to achieve your desired result, because of
all the
Thank you Martijn, the article you provided had detailed explanation on the
exactly once two phase commit.
Returning to the best way to handle commits/acknowledgments on sources like
JMS Queues or Solace topics to support guaranteed delivery, when they are
not supported out of the box by Flink (
Hi Flink Kubernetes Operator Community,
I hope this is the right way to report an issue with the Apache Flink
Kubernetes Operator. We are experiencing problems with some streaming job
clusters which end up in a terminated state, because of the operator not
behaving as expected. The problem is t
Sources don't need to support two phase commits, that's something for
sinks. I think the example of exactly-once-processing (although the
interfaces have changed since) at
https://flink.apache.org/2018/02/28/an-overview-of-end-to-end-exactly-once-processing-in-apache-flink-with-apache-kafka-too/
is
Let me put the question in other words.
What happens if a source does not support two phase commit and the Flink
job has to guarantee exactly once delivery to downstream? Checkpointing as
I understand, works on interval basis. New events for which the checkpoint
barrier has not yet reached will ge
Hi Kartik,
I don't think there's much that the Flink community can do here to
help you. The Solace source and sink aren't owned by the Flink
project, and based on the source code they haven't been touched for
the last 7 years [1] and I'm actually not aware of anyone who uses
Solace at all.
Best r
Any help here please.
Regards,
Kartik
On Fri, Feb 9, 2024, 8:33 AM Kartik Kushwaha
wrote:
> I am using flink checkpointing to restore states of my job. I am using
> unaligned checkpointing with 100 ms as the checkpointing interval. I see
> few events getting dropped that were sucessfully proce
Hello
Request you to please update.
Thanks !!
From: Praveen Chandna via user
Sent: Friday, February 9, 2024 2:46 PM
To: Praveen Chandna via user
Subject: Flink connection with AWS OpenSearch Service
Hello
As per the Flink OpenSearch connector documentation, it specify how to connect
to Open
