Re: Flink cep checkpoint size

2021-07-08 Thread Li Jim
Hi, Dawid. Thanks for replying, happy to know you are working on this. On 2021/07/08 12:14:21, Dawid Wysakowicz wrote: > Hi, > > Sorry for the late reply. > > Indeed I found a couple of problems with clearing the state for short > lived keys. I created a JIRA[1] issue to track it and opened a

Re: Flink cep checkpoint size

2021-07-08 Thread Dawid Wysakowicz
Hi, Sorry for the late reply. Indeed I found a couple of problems with clearing the state for short lived keys. I created a JIRA[1] issue to track it and opened a PR (which needs test coverage before it can be merged) with fixes for those. Best, Dawid [1] https://issues.apache.org/jira/browse/

Re: Flink CEP checkpoint size

2021-07-07 Thread Aeden Jameson
We did look into fixing it ourselves, but decided that migrating to the datastream api, not using CEP, was more fruitful overall for us. Unfortunately, I don't have a good answer for you. The bug from a non-contributors stand point appears pretty deep in the codebase, but the authors are best ones

Re: Flink CEP checkpoint size

2021-07-06 Thread Li Jim
Hi, Jameson Thanks very much for replying , I am really struggling on this. I am using flowId as my keys, which means they will be matched and never use again. This seems like the scenario 2. I didn't know it is not fixed yet. thank you again and do you have any solutions ? On 2021/07/07 01:47:00

Re: Flink CEP checkpoint size

2021-07-06 Thread Aeden Jameson
Hi Li, How big is your keyspace? Had a similar problem which turns out to be scenario 2 in this issue https://issues.apache.org/jira/browse/FLINK-19970. Looks like the bug in scenario 1 got fixed by scenario 2 did not. There's more detail in this thread, http://deprecated-apache-flink-user-ma

Re: Flink cep checkpoint size

2021-07-06 Thread Li Jim
Hi, Mohit, Have you figured out any solusions on this problem ? I am now facing the exactly same problem , I was using Flink of version 1.12.0 and I also upgrated it to 1.13.1 but the checkpoint size is still growing. On 2021/06/02 15:45:59, "Singh, Mohit" wrote: > Hi, > > I am facing an i