Hi, If I understand you correctly, you want to check the local RocksDB files, right? They are stored locally on each TaskManager in a temporary directory.
This can be configured via "state.backend.rocksdb.localdir"[1]. If not
specified it will use the globally defined temporary directory set via
"io.tmp.dirs"[2], which defaults to : "'LOCAL_DIRS' on Yarn.
'_FLINK_TMP_DIR' on Mesos. System.getProperty("java.io.tmpdir") in
standalone."
Best,
Dawid
[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/config.html#state-backend-rocksdb-localdir
[2]
https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/config.html#io-tmp-dirs
On 22/06/2020 15:22, Sudan S wrote:
> Hi,
>
> I have enabled rocksdb(State store) with s3 (external checkpoint) on EMR .
>
> I am using rocksdb as state store with ValueState and checkpoints are
> stored in s3.
>
> I am able to see checkpoints in s3 and functionality with respect to
> state store is working fine.
>
> But i am trying to dissect rocksdb folder structure but i am not able
> to find them on EMR. I am checking /mnt/yarn folder.
>
> Cn u plz help on where flink persists store on EMR when rocksdb is
> used as state backend?
>
>
>
>
>
> ------------------------------------------------------------------------
> /"The information contained in this e-mail and any accompanying
> documents may contain information that is confidential or otherwise
> protected from disclosure. If you are not the intended recipient of
> this message, or if this message has been addressed to you in error,
> please immediately alert the sender by replying to this e-mail and
> then delete this message, including any attachments. Any
> dissemination, distribution or other use of the contents of this
> message by anyone other than the intended recipient is strictly
> prohibited. All messages sent to and from this e-mail address may be
> monitored as permitted by applicable law and regulations to ensure
> compliance with our internal policies and to protect our business."/
> ------------------------------------------------------------------------
signature.asc
Description: OpenPGP digital signature
