[DISCUSS] FLIP-185: Shorter heartbeat timeout and interval default values

Hi everyone, Since Flink 1.5 we have the same heartbeat timeout and interval default values that are defined as heartbeat.timeout: 50s and heartbeat.interval: 10s. These values were mainly chosen to compensate for lengthy GC pauses and blocking operations that were executed in the main threads of

Re: Flink RocksDB Performance

Yes absolutely. Unless we need a very large state order of GB rocks DB is not required. RocksDB is good only because the Filesystem is very bad at LargeState. In other words FileSystem performs much better than RocksDB upto GB's. After that the file system degrades compared to RocksDB. Its not that

Re: Flink RocksDB Performance

Hi Li Jim, Filesystem performs much better than rocksdb (by multiple times), but it is only suitable for small states. Rocksdb will consume more CPU on background tasks, cache management, serialization/deserialization and compression/decompression. In most cases, performance of the Rocksdb will mee

Flink 1.13.1 - Vulnerabilities CVE-2019-12900 for librocksdbjni

Hi I am observing flink-1.13.1 is being flagged for CVE-2019-12900 for librocksdbjni The issue seems to have been fixed in RocksDB in https://github.com/facebook/rocksdb/issues/6703. Can someone let me know if flink is affected by this? I see this has been discussed a bit on https://issues.apa

Re: OOM Metaspace after multiple jobs

Since I'm running in a container, I was able to copy some of the jars to Flink's lib folder. When it comes to gRPC, I don't know if there's any other good option due to possible issues with ThreadLocals: https://github.com/grpc/grpc-java/issues/8309 Even then, I'm not sure that's a complete sol

Flink RocksDB Performance

Hello everyone, I am using Flink 1.13.1 CEP Library and doing some pressure test. My message rate is about 16000 records per second. I find that it cant process more than 16000 records per second because the CPU cost is up to 100%(say 800% because I allocated 8 vcores to a taskmanager). I tried sw

Re: key_by problem in Pyflink

Hi again, It seems the temporary solution I mentioned in last mail doesn't work steadily. I am wondering that whether all states should be accessed in `process_element1`, `process_element2` and `on_timer` (if a timer is registered) or not ? Or is there any other suggested workaround? Thanks. Fe