Re: No effect from --allowNonRestoredState or "execution.savepoint.ignore-unclaimed-state" in K8S application mode

2022-02-22 Thread Andrey Bulgakov
lementation[1]. > > [1]. > https://github.com/apache/flink/blob/master/flink-clients/src/main/java/org/apache/flink/client/cli/ProgramOptions.java#L181 > > > Best, > Yang > > Andrey Bulgakov 于2022年2月19日周六 08:30写道: > >> Hi Austin, >>

Re: No effect from --allowNonRestoredState or "execution.savepoint.ignore-unclaimed-state" in K8S application mode

2022-02-18 Thread Andrey Bulgakov
> Best, > Austin > > [1]: > https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/resource-providers/native_kubernetes/#configuring-flink-on-kubernetes > > On Fri, Feb 18, 2022 at 5:14 PM Andrey Bulgakov wrote: > >> Hey all, >> >> I&#x

No effect from --allowNonRestoredState or "execution.savepoint.ignore-unclaimed-state" in K8S application mode

2022-02-18 Thread Andrey Bulgakov
r am I doing something wrong? For context, the savepoint is produced by Flink 1.8.2 and the version I'm trying to run on K8S is 1.14.3. -- With regards, Andrey Bulgakov

Re: Extracting state keys for a very large RocksDB savepoint

2021-03-17 Thread Andrey Bulgakov
I guess there's no point in making it a KeyedProcessFunction since it's not going to have access to context, timers or anything like that. So it can be a simple InputFormat returning a DataSet of key and value tuples. On Wed, Mar 17, 2021 at 8:37 AM Andrey Bulgakov wrote: > Hi

Re: Extracting state keys for a very large RocksDB savepoint

2021-03-17 Thread Andrey Bulgakov
is correct; basically > it > tries to restore the state backends as is. > > In you current implementation, do you see it worthwhile to try to add this? > > Cheers, > Gordon > > > > -- > Sent from: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/ > -- With regards, Andrey Bulgakov

Re: Extracting state keys for a very large RocksDB savepoint

2021-03-14 Thread Andrey Bulgakov
KeyGroupsStateHandle objects and built an InputFormat heavily based on the code I found in RocksDBFullRestoreOperation.java. It ended up working extremely quickly while keeping memory and CPU usage at the minimum. On Tue, Mar 9, 2021 at 1:51 PM Andrey Bulgakov wrote: > Hi all, > > I'm

Extracting state keys for a very large RocksDB savepoint

2021-03-09 Thread Andrey Bulgakov
es available to each task manager - enabling object reuse and modifying the tuple mapper to avoid extra tuple allocations - manipulating memory ratios to allocate more memory to be used as heap, managed - allocating 20% of memory for JVM overhead - switching to G1GC garbage collector Again, would appreciate any help with this. -- With regards, Andrey Bulgakov