Re: Issues with state corruption in RocksDB

2025-02-14 Thread Peter Westermann
r Somogyi Date: Friday, February 14, 2025 at 12:25 PM To: Peter Westermann Cc: user@flink.apache.org Subject: Re: Issues with state corruption in RocksDB EXTERNAL EMAIL - Please use caution with links and attachments Is it related to state size or can happen

Re: Issues with state corruption in RocksDB

2025-02-14 Thread Gabor Somogyi
> signature_561365254] <https://www.facebook.com/Genesys/>[image: > signature_262758434] <https://www.youtube.com/Genesys>[image: > signature_2090310858] <http://blog.genesys.com/> > > > > > > > > > > *From: *Gabor Somogyi > *Date: *Friday, F

Re: Issues with state corruption in RocksDB

2025-02-14 Thread Peter Westermann
Friday, February 14, 2025 at 11:54 AM To: Peter Westermann Cc: user@flink.apache.org Subject: Re: Issues with state corruption in RocksDB EXTERNAL EMAIL - Please use caution with links and attachments BTW, do you have a bare minimum application which can reproduce t

Re: Issues with state corruption in RocksDB

2025-02-14 Thread Gabor Somogyi
BTW, do you have a bare minimum application which can reproduce the issue? On Fri, Feb 14, 2025 at 5:07 PM Gabor Somogyi wrote: > Hi Peter, > > I've had a look on this and seems like this problem or a similar one still > exists in 1.20. > Since I've not done in-depth analysis I can only say this

Re: Issues with state corruption in RocksDB

2025-02-14 Thread Gabor Somogyi
Hi Peter, I've had a look on this and seems like this problem or a similar one still exists in 1.20. Since I've not done in-depth analysis I can only say this by taking a look at the last comments, like: > Faced an issue on Flink 1.20.0 but there are more of such... So all in all I've the feelin

Issues with state corruption in RocksDB

2025-01-07 Thread Peter Westermann
In the past, we have infrequently had issues with corrupted timer state (see https://issues.apache.org/jira/browse/FLINK-23886), which resolve, when we go back to a previous savepoint and restart the job from that. Recently (since we upgraded to Flink 1.19.1), we have seen more state corruption