Re: How does Flink handle shorted lived keyed streams

2021-02-10 Thread narasimha
Thanks Matthias, I think it will help to find out what all live keys are present. Let me check and revert back on the thread. On Wed, Feb 10, 2021 at 10:46 PM Matthias Pohl wrote: > Hi narashima, > not sure whether this fits your use case, but have you considered creating > a savepoint and anal

Re: How does Flink handle shorted lived keyed streams

2021-02-10 Thread Matthias Pohl
Hi narashima, not sure whether this fits your use case, but have you considered creating a savepoint and analyzing it using the State Processor API [1]? Best, Matthias [1] https://ci.apache.org/projects/flink/flink-docs-stable/dev/libs/state_processor_api.html#state-processor-api On Wed, Feb 10,

Re: How does Flink handle shorted lived keyed streams

2021-02-10 Thread narasimha
It is not solving the problem. I could see the memory keep increasing, resulting in a lot of high GCs. There could be a memory leak, just want to know how to know if older keps are skill alive, even after the pattern has been satisfied or within range of the pattern has expired. Can someone sugg

Re: How does Flink handle shorted lived keyed streams

2020-12-24 Thread narasimha
Thanks Xintong. I'll check it out and get back to you. On Thu, Dec 24, 2020 at 1:30 PM Xintong Song wrote: > I believe what you are looking for is the State TTL [1][2]. > > > Thank you~ > > Xintong Song > > > [1] > https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/state.ht

Re: How does Flink handle shorted lived keyed streams

2020-12-24 Thread Xintong Song
I believe what you are looking for is the State TTL [1][2]. Thank you~ Xintong Song [1] https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/state.html#state-time-to-live-ttl [2] https://ci.apache.org/projects/flink/flink-docs-stabledev/table/config.html#table-exec-state-tt