Hello Milind,
I'm not entirely sure i fully understood your question, but I'll try
anyway :)
There is now way to provide exactly-once semantics for Cassandra's
counters. As such we (will) only provide exactly-once semantics for a
subset of Cassandra operations; idempotent inserts/updates.
There are several things that would allow exactly-once semantics:
* transactions
o rather obvious i think
* replaying/rollback to a given state
o replay for sources/rollback for sinks upon failure
* an atomic idempotent update across 2 tables.
o allows tracking every read/write made; selectively re-read/write
upon failure
One of the key requisites is proper failure reporting though; if an
update fails we /need to know/. As far as i know Cassandra doesn't make
this guarantee.
Regards,
Chesnay Schepler
On 10.05.2016 07:48, milind parikh wrote:
Given FLINK 3311 & 3332, I am wondering it would be possible, without
idempotent counters in Cassandra, to deliver on an exactly once sink
into Cassandra. I do note that the verbiage/disc2 in 3332 does warn
the user that this is not exactly "exactly once" sink.
However my question has to do with whether having idempotent counters
and a Data model that enables all other idempotent operations are a
necessary prerequisite to exactly once semantics in flink.
Asked a different way, what source and sink would enable a end-to-end
exactly - once semantics, in the current state-of-the-art, with Flink
in the middle.
Thanks
Milind
