"counter" variable, that is held in a "ValueState", this counter will always be in sync with the number of elements in the kafka topic (because the counter
is reset on a failure).
On Tue, Feb 21, 2017 at 4:04 PM, Y. Sakamoto mailto:phonypian...@gmail.com>> wrote:
es to Flink's state to Elastic. That way the writes to ElasticSearch
become "idempotent", which means duplicates simple make no additional changes.
Hope that helps!
Stephan
On Mon, Feb 20, 2017 at 5:53 PM, Y. Sakamoto mailto:phonypian...@gmail.com>> wrote:
Hi
Hi,
I'm using Flink 1.2.0 and try to do "exactly once" data transfer
from Kafka to Elasticsearch, but I cannot.
(Scala 2.11, Kafka 0.10, without YARN)
There are 2 Flink TaskManager nodes, and when processing
with 2 parallelism, shutdown one of them (simulating node failure).
Using flink-connecto
