Re: NegativeArraySizeException trying to take a savepoint

2022-06-14 Thread Mike Barborak
, 2022 at 9:15 AM To: Martijn Visser , Mike Barborak Cc: user@flink.apache.org Subject: Re: NegativeArraySizeException trying to take a savepoint Hi Mike, I think the root cause is that the size of java bytes array still exceed VM limit. The exception message is not friendly and not covered by

NegativeArraySizeException trying to take a savepoint

2022-06-13 Thread Mike Barborak
When trying to savepoint our job, we are getting the stack trace below. Is there a way to know more about this failure? Like which function in the job graph is associated with the problematic state and which key (assuming it is keyed state)? Or is there a fix for this exception? The only mentio

savepoint.readKeyedState hangs on org.rocksdb.RocksDB.disposeInternal

2021-10-31 Thread Mike Barborak
Hi, I am using the state processing API to examine a savepoint. My code works fine when I use a HashMapStateBackend but for larger savepoints, I don’t have enough memory so need to use a EmbeddedRocksDBStateBackend. Even then, I am able to process some smaller states but this one: operatorID,p

Write savepoint from test harness

2021-10-19 Thread Mike Barborak
Hi, I am using the KeyedOneInputStreamOperatorTestHarness. With that, I can take a snapshot and then use OperatorSnapshotUtil to write and read it. I am wondering if I can take a savepoint using the test harness or write the snapshot as a savepoint in order to use the ExistingSavepoint API to e