Hi Dongwon,

With State Processor API you should be able to create a new snapshot that
doesn't reference the unused classes.

Regards,
Roman


On Tue, Feb 9, 2021 at 3:39 AM Dongwon Kim <[email protected]> wrote:

> Hi Khachatryan,
>
> Thanks for the explanation and the input!
>
> 1. Use the State Processor API to create a new snapshot [1]
>
> I haven't used it. but does the API prevent the class of a specific
> serializer from being loaded?
>
> 2. If the operator has only this state then changing uid (together with
>> allowNonRestoredState) should help
>
> Very unfortunately, I have another per-key state defined on the operator
> which is very important and cannot be abandoned T.T
>
> 3. Probably just changing POJO to an empty class will suffice in your case?
>
> Yeah, I might be bringing the class definition for a while.
>
> Best,
>
> Dongwon
>
>
> On Tue, Feb 9, 2021 at 2:35 AM Khachatryan Roman <
> [email protected]> wrote:
>
>> Hi,
>>
>> I'm pulling Yun Tang who is familiar with StateBackends and RocksDB in
>> particular.
>>
>> From what I see, the 2nd snapshot (sp2) is built using the same set of
>> states obtained from the starting savepoint/checkpoint (sp1) to write its
>> metadata. This metadata includes serializers snapshots, including
>> PojoSerializer for your custom type. On restore, this metadata is read, and
>> POJO class itself is loaded.
>>
>> I see the following ways to overcome this issue:
>> 1. Use the State Processor API to create a new snapshot [1]
>> 2. If the operator has only this state then changing uid (together with
>> allowNonRestoredState) should help
>> 3. Probably just changing POJO to an empty class will suffice in your
>> case?
>>
>> [1]
>> https://ci.apache.org/projects/flink/flink-docs-stable/dev/libs/state_processor_api.html
>>
>> Regards,
>> Roman
>>
>>
>> On Mon, Feb 8, 2021 at 4:31 PM Dongwon Kim <[email protected]> wrote:
>>
>>> Hi 张静,
>>>
>>>     Q1: By default, a savepoint restore will try to match all state
>>>> back to the restored job. `AllowNonRestoredState` cannot avoid
>>>> recovery all state from savepoint, but only skip match all of the
>>>> restore state back to the restored job. So `ClassNotFoundException `
>>>> could not be avoid.
>>>
>>> okay
>>>
>>>    Q2: Not really. After you recover new job from the savepoint
>>>> (savepoint1)based on (1), you could do a new savepoint (savepoint2),
>>>> then remove the definition of the POJO type. then you can restore from
>>>> savepoint2.
>>>>
>>> I did it but it ends up with the same ClassNotFoundException :-(
>>>
>>> What I did exactly are
>>> (1) Trigger sp1 from v1
>>> (2) Start v2-1 (w/ the definition of the POJO but do not use it at all)
>>> from sp1
>>> (3) Trigger sp2 from v2-1
>>> (4) Start v2-2 (w/o the definition of the POJO)  from sp2
>>> (5) v2-2 failed with the same ClassNotFoundException regarding the POJO
>>> type
>>>
>>> Should v2-2 successfully start from sp2?
>>>
>>> Best,
>>>
>>> Dongwon
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Mon, Feb 8, 2021 at 11:48 PM 张静 <[email protected]> wrote:
>>>
>>>> Hi, Dongwon,
>>>>      Q1: By default, a savepoint restore will try to match all state
>>>> back to the restored job. `AllowNonRestoredState` cannot avoid
>>>> recovery all state from savepoint, but only skip match all of the
>>>> restore state back to the restored job. So `ClassNotFoundException `
>>>> could not be avoid.
>>>>      Q2: Not really. After you recover new job from the savepoint
>>>> (savepoint1)based on (1), you could do a new savepoint (savepoint2),
>>>> then remove the definition of the POJO type. then you can restore from
>>>> savepoint2.
>>>> Correct me please if I'm wrong. Thanks.
>>>>
>>>> Best,
>>>> Beyond1920
>>>>
>>>> Dongwon Kim <[email protected]> 于2021年2月8日周一 下午9:43写道:
>>>> >
>>>> > Hi,
>>>> >
>>>> > I have an original job (say v1) and I want to start a new job (say
>>>> v2) from a savepoint of v1.
>>>> >
>>>> > An operator of v1 used to have per-key states of a POJO type, but I
>>>> want to remove the states together with the definition of the POJO type.
>>>> >
>>>> > When I start v2 from a savepoint of v1, I specified
>>>> "--allowNonRestoredState" but  I got the following exception:
>>>> >
>>>> > 2021-02-08 22:07:28,324 INFO
>>>> org.apache.flink.runtime.executiongraph.ExecutionGraph       [] -
>>>> input-to-idata (5/8) (168520e96e73dbf6c2e3c5342b324764) switched from
>>>> RUNNING to FAILED on container_e02_1607261469522_0242_01_000008 @
>>>> mobdata-flink-dn29.dakao.io (dataPort=45505).
>>>> > java.lang.Exception: Exception while creating
>>>> StreamOperatorStateContext.
>>>> > at
>>>> org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:254)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:272)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:425)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$2(StreamTask.java:535)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:525)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:565)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:755)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:570)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_222]
>>>> > Caused by: org.apache.flink.util.FlinkException: Could not restore
>>>> keyed state backend for
>>>> CoBroadcastWithKeyedOperator_692eb9021f6319ecb59ea7c8901de92a_(5/8) from
>>>> any of the 1 provided restore options.
>>>> > at
>>>> org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:160)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.keyedStatedBackend(StreamTaskStateInitializerImpl.java:345)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:163)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > ... 9 more
>>>> > Caused by: org.apache.flink.runtime.state.BackendBuildingException:
>>>> Caught unexpected exception.
>>>> > at
>>>> org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackendBuilder.build(RocksDBKeyedStateBackendBuilder.java:361)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.contrib.streaming.state.RocksDBStateBackend.createKeyedStateBackend(RocksDBStateBackend.java:587)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.contrib.streaming.state.RocksDBStateBackend.createKeyedStateBackend(RocksDBStateBackend.java:93)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.lambda$keyedStatedBackend$1(StreamTaskStateInitializerImpl.java:328)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.streaming.api.operators.BackendRestorerProcedure.attemptCreateAndRestore(BackendRestorerProcedure.java:168)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:135)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.keyedStatedBackend(StreamTaskStateInitializerImpl.java:345)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:163)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > ... 9 more
>>>> > Caused by: java.io.IOException: Could not find class
>>>> 'com.kakaomobility.drivinghabit.stream.IDataConverter$SpeedAndLoc' in
>>>> classpath.
>>>> > at
>>>> org.apache.flink.util.InstantiationUtil.resolveClassByName(InstantiationUtil.java:756)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.util.InstantiationUtil.resolveClassByName(InstantiationUtil.java:731)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.api.java.typeutils.runtime.PojoSerializerSnapshotData.readSnapshotData(PojoSerializerSnapshotData.java:214)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.api.java.typeutils.runtime.PojoSerializerSnapshotData.createFrom(PojoSerializerSnapshotData.java:135)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.api.java.typeutils.runtime.PojoSerializerSnapshot.readSnapshot(PojoSerializerSnapshot.java:129)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.api.common.typeutils.TypeSerializerSnapshot.readVersionedSnapshot(TypeSerializerSnapshot.java:175)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.api.common.typeutils.NestedSerializersSnapshotDelegate.readNestedSerializerSnapshots(NestedSerializersSnapshotDelegate.java:178)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.api.common.typeutils.CompositeTypeSerializerSnapshot.readSnapshot(CompositeTypeSerializerSnapshot.java:171)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.api.common.typeutils.TypeSerializerSnapshot.readVersionedSnapshot(TypeSerializerSnapshot.java:175)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.api.common.typeutils.TypeSerializerSnapshotSerializationUtil$TypeSerializerSnapshotSerializationProxy.deserializeV2(TypeSerializerSnapshotSerializationUtil.java:174)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.api.common.typeutils.TypeSerializerSnapshotSerializationUtil$TypeSerializerSnapshotSerializationProxy.read(TypeSerializerSnapshotSerializationUtil.java:145)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.api.common.typeutils.TypeSerializerSnapshotSerializationUtil.readSerializerSnapshot(TypeSerializerSnapshotSerializationUtil.java:77)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.runtime.state.metainfo.StateMetaInfoSnapshotReadersWriters$CurrentReaderImpl.readStateMetaInfoSnapshot(StateMetaInfoSnapshotReadersWriters.java:237)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.runtime.state.KeyedBackendSerializationProxy.read(KeyedBackendSerializationProxy.java:184)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.contrib.streaming.state.restore.AbstractRocksDBRestoreOperation.readMetaData(AbstractRocksDBRestoreOperation.java:211)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.contrib.streaming.state.restore.RocksDBFullRestoreOperation.restoreKVStateMetaData(RocksDBFullRestoreOperation.java:173)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.contrib.streaming.state.restore.RocksDBFullRestoreOperation.restoreKeyGroupsInStateHandle(RocksDBFullRestoreOperation.java:157)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.contrib.streaming.state.restore.RocksDBFullRestoreOperation.restore(RocksDBFullRestoreOperation.java:142)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackendBuilder.build(RocksDBKeyedStateBackendBuilder.java:284)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.contrib.streaming.state.RocksDBStateBackend.createKeyedStateBackend(RocksDBStateBackend.java:587)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.contrib.streaming.state.RocksDBStateBackend.createKeyedStateBackend(RocksDBStateBackend.java:93)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.lambda$keyedStatedBackend$1(StreamTaskStateInitializerImpl.java:328)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.streaming.api.operators.BackendRestorerProcedure.attemptCreateAndRestore(BackendRestorerProcedure.java:168)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:135)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.keyedStatedBackend(StreamTaskStateInitializerImpl.java:345)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:163)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > ... 9 more
>>>> > Caused by: java.lang.ClassNotFoundException:
>>>> com.kakaomobility.drivinghabit.stream.IDataConverter$SpeedAndLoc
>>>> > at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
>>>> ~[?:1.8.0_222]
>>>> > at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>> ~[?:1.8.0_222]
>>>> > at
>>>> org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:64)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:74)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>> ~[?:1.8.0_222]
>>>> > at
>>>> org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.loadClass(FlinkUserCodeClassLoaders.java:172)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at java.lang.Class.forName0(Native Method) ~[?:1.8.0_222]
>>>> > at java.lang.Class.forName(Class.java:348) ~[?:1.8.0_222]
>>>> > at
>>>> org.apache.flink.util.InstantiationUtil.resolveClassByName(InstantiationUtil.java:754)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.util.InstantiationUtil.resolveClassByName(InstantiationUtil.java:731)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.api.java.typeutils.runtime.PojoSerializerSnapshotData.readSnapshotData(PojoSerializerSnapshotData.java:214)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.api.java.typeutils.runtime.PojoSerializerSnapshotData.createFrom(PojoSerializerSnapshotData.java:135)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.api.java.typeutils.runtime.PojoSerializerSnapshot.readSnapshot(PojoSerializerSnapshot.java:129)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.api.common.typeutils.TypeSerializerSnapshot.readVersionedSnapshot(TypeSerializerSnapshot.java:175)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.api.common.typeutils.NestedSerializersSnapshotDelegate.readNestedSerializerSnapshots(NestedSerializersSnapshotDelegate.java:178)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.api.common.typeutils.CompositeTypeSerializerSnapshot.readSnapshot(CompositeTypeSerializerSnapshot.java:171)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.api.common.typeutils.TypeSerializerSnapshot.readVersionedSnapshot(TypeSerializerSnapshot.java:175)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.api.common.typeutils.TypeSerializerSnapshotSerializationUtil$TypeSerializerSnapshotSerializationProxy.deserializeV2(TypeSerializerSnapshotSerializationUtil.java:174)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.api.common.typeutils.TypeSerializerSnapshotSerializationUtil$TypeSerializerSnapshotSerializationProxy.read(TypeSerializerSnapshotSerializationUtil.java:145)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.api.common.typeutils.TypeSerializerSnapshotSerializationUtil.readSerializerSnapshot(TypeSerializerSnapshotSerializationUtil.java:77)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.runtime.state.metainfo.StateMetaInfoSnapshotReadersWriters$CurrentReaderImpl.readStateMetaInfoSnapshot(StateMetaInfoSnapshotReadersWriters.java:237)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.runtime.state.KeyedBackendSerializationProxy.read(KeyedBackendSerializationProxy.java:184)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.contrib.streaming.state.restore.AbstractRocksDBRestoreOperation.readMetaData(AbstractRocksDBRestoreOperation.java:211)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.contrib.streaming.state.restore.RocksDBFullRestoreOperation.restoreKVStateMetaData(RocksDBFullRestoreOperation.java:173)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.contrib.streaming.state.restore.RocksDBFullRestoreOperation.restoreKeyGroupsInStateHandle(RocksDBFullRestoreOperation.java:157)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.contrib.streaming.state.restore.RocksDBFullRestoreOperation.restore(RocksDBFullRestoreOperation.java:142)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackendBuilder.build(RocksDBKeyedStateBackendBuilder.java:284)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.contrib.streaming.state.RocksDBStateBackend.createKeyedStateBackend(RocksDBStateBackend.java:587)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.contrib.streaming.state.RocksDBStateBackend.createKeyedStateBackend(RocksDBStateBackend.java:93)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.lambda$keyedStatedBackend$1(StreamTaskStateInitializerImpl.java:328)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.streaming.api.operators.BackendRestorerProcedure.attemptCreateAndRestore(BackendRestorerProcedure.java:168)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:135)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.keyedStatedBackend(StreamTaskStateInitializerImpl.java:345)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> > at
>>>> org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:163)
>>>> ~[flink-dist_2.11-1.12.1.jar:1.12.1]
>>>> >
>>>> >  ... 9 more
>>>> >
>>>> >
>>>> > So I make two versions of v2:
>>>> > 1) v2_1 : remove only key states w/o removing the definition of the
>>>> POJO type. I can manage to resume from a savepoint of v1.
>>>> > 2) v2_2 : remove both key states and the definition of the POJO type.
>>>> I hope resuming from a savepoint of v2_1 could succeed but it fails with
>>>> the same exception as above.
>>>> >
>>>> > Q1) Why doesn't the "--allowNonRestoredState" option suppress
>>>> ClassNotFoundException?
>>>> > Q2) Do I have to live forever with the definition of the POJO type
>>>> which is no longer necessary?
>>>> >
>>>> > Best,
>>>> >
>>>> > Dongwon
>>>> >
>>>>
>>>

Reply via email to