Hi Ufuk,
Yes, it does help with Rocksdb backend!
After tune checkpoint frequency align with network throughput, task manager
released and job get cancelled are gone.
Chen
> On May 10, 2016, at 10:33 AM, Ufuk Celebi wrote:
>
>> On Tue, May 10, 2016 at 5:07 PM, Chen Qin wrote:
>> Future, to k
On Tue, May 10, 2016 at 5:07 PM, Chen Qin wrote:
> Future, to keep large key/value space, wiki point out using rocksdb as
> backend. My understanding is using rocksdb will write to local file systems
> instead of sync to s3. Does flink support memory->rocksdb(local disk)->s3
> checkpoint state spl
Hi there,
With S3 as state backend, as well as keeping a large chunk of user state on
heap. I can see task manager starts to fail without showing OOM exception.
Instead, it shows a generic error message (below) when checkpoint triggered. I
assume this has something to do with how state were kep
