Re: General guidance

2021-03-25 Thread Kenneth Knowles
This is a Beam issue indeed, though it is an issue with the FlinkRunner. So I think I will BCC the Flink list. You may be in one of the following situations: - These timers should not be viewed as distinct by the runner, but deduped, per https://issues.apache.org/jira/browse/BEAM-8212#comment-169

General guidance

2021-03-25 Thread Almeida, Julius
Hi Team, My streaming pipeline is based on beam & running using flink runner with rocksdb as state backend. Over time I am seeing memory spike & after giving a look at heap dump, I am seeing records in ‘__StatefulParDoGcTimerId’ which seems to be never cleaned. Found this jira https://issues