Elias Levy created FLINK-3692:
---------------------------------
Summary: Develop a Kafka state backend
Key: FLINK-3692
URL: https://issues.apache.org/jira/browse/FLINK-3692
Project: Flink
Issue Type: New Feature
Components: Core
Reporter: Elias Levy
Flink clusters usually consume of a Kafka cluster. It simplify operations if
Flink could store its state checkpoints in Kafka. This should be possibly
using different topics to write to, partitioning appropriately, and using
compacted topics. This would avoid the need to run an HDFS cluster just to
store Flink checkpoints.
For inspiration you may want to take a look at how Samza checkpoints a task's
local state to a Kafka topic, and how the newer Kafka consumers checkpoint
their offsets to Kafka.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
