Hi all, I was wondering if anyone has any ideas or advice when it comes to being efficient about the use of disks with the RocksDB StateBackend.
In general not all operators in a Flink Job will be stateful and require persistent disks to use with RocksDB, and so any stateless operators running on TaskManagers with disks will not use the disk at all, which is wasteful. I found the External Resources plugin documentation <https://nightlies.apache.org/flink/flink-docs-release-1.18/docs/deployment/advanced/external_resources/>, which seems interesting, but we are using the Flink SQL API, and only the DataStream API is supported there. Thanks in advance, Kevin