Re: Rocksdb state directory path in EMR

2020-06-22 Thread Dawid Wysakowicz
Hi, If I understand you correctly, you want to check the local RocksDB files, right? They are stored locally on each TaskManager in a temporary directory. This can be configured via "state.backend.rocksdb.localdir"[1]. If not specified it will use the globally defined temporary directory set via

Rocksdb state directory path in EMR

2020-06-22 Thread Sudan S
Hi, I have enabled rocksdb(State store) with s3 (external checkpoint) on EMR . I am using rocksdb as state store with ValueState and checkpoints are stored in s3. I am able to see checkpoints in s3 and functionality with respect to state store is working fine. But i am trying to dissect rocksdb