user
Subject: Re: Storage options for RocksDBStateBackend
Till and Stephan, thanks for your clarification.
@Till One more question, from what I have read about the checkpointing [1], the
list operations don't seem likely to be performed frequently, so storing state
backend on s3 shouldn't h
Till and Stephan, thanks for your clarification.
@Till One more question, from what I have read about the checkpointing [1],
the list operations don't seem likely to be performed frequently, so
storing state backend on s3 shouldn't have any severe impact on flink
performance. Is this assumption ri
Small addition to Till's comment:
In the case where file:// points to a mounted distributed file system (NFS,
MapRFs, ...), then it actually works. The important thing is that the
filesystem where the checkpoints go is replicated (fault tolerant) and
accessible from all nodes.
On Thu, May 11, 201
Hi Ayush,
you’re right that RocksDB is the recommend state backend because of the
above-mentioned reasons. In order to make the recovery properly work, you
have to configure a shared directory for the checkpoint data via
state.backend.fs.checkpointdir. You can basically configure any file system
w
Hello,
I had a few questions regarding checkpoint storage options using
RocksDBStateBackend. In the flink 1.2 documentation, it is the recommended
state
backend due to it's ability to store large states and asynchronous
snapshotting.
For high availabilty it seems HDFS is the recommended store for
use a NFS volume for state backend, with
"file://" urls? Will there be any performance impact?
-- Ayush
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Storage-options-for-RocksDBStateBackend-tp13102.html
Sent from the Apache Fli
