Hi Desheng,
Welcome to the community!
What you’re asking alludes the question: How does Flink support end-to-end
(from external source to external sink, e.g. Kafka to database) exactly-once
delivery?
Whether or not that is supported depends on the guarantees of the source and
sink and how they
Hi Flink Community,
I am new to Flink and now looking at checkpoint of Flink.
After reading the document, I am still confused. Here is scene:
I have a datastream finally flow to a database sink. I will update one of the
field in database based on the incomming stream. I have now complete a
sna
Thank you. That clears it up.
I meant SavePoints. Sorry I used the term Snapshots in its place :-).
Thanks,
Sameer
On Tue, Jul 26, 2016 at 8:33 AM, Ufuk Celebi wrote:
> On Tue, Jul 26, 2016 at 2:15 PM, Sameer W wrote:
> > 1. Calling clear() on the KV state is only possible for snapshots right
On Tue, Jul 26, 2016 at 2:15 PM, Sameer W wrote:
> 1. Calling clear() on the KV state is only possible for snapshots right? Do
> you control that for checkpoints too.
What do you mean with snapshots vs. checkpoints exactly?
> 2. Assuming that the user has no control over the checkpoint process o
Thanks Ufuk,
That was very helpful. But that raised a few more questions :-):
1. Calling clear() on the KV state is only possible for snapshots right? Do
you control that for checkpoints too.
2. Assuming that the user has no control over the checkpoint process
outside of controlling the checkpoi
On Mon, Jul 25, 2016 at 8:50 PM, Sameer W wrote:
> The question is, if using really long windows (in hours) if the state of the
> window gets very large over time, would size of the RocksDB get larger?
> Would replication to HDFS start causing performance bottlenecks? Also would
> this need a cons
Hi,
My understanding about the RocksDB state backend is as follows:
When using a RocksDB state backend, it the checkpoints are backed up
locally (to the TaskManager) using the backup feature of RocksDB by taking
snapshots from RocksDB which are consistent read-only views on the RockDB
database. E
