>From these snippets it is hard to tell what's going wrong. Could you maybe give us a minimal example with which to reproduce the problem? Alternatively, have you read through Flink's serializer documentation [1]? Have you tried to use a simple POJO instead of inheriting from a HashMap?
The stack trace looks as if the job fails deserializing some key of your MapRecord map. [1] https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/types_serialization.html#most-frequent-issues Cheers, Till On Wed, Feb 3, 2021 at 11:49 AM 赵一旦 <hinobl...@gmail.com> wrote: > Some facts are possibly related with these, since another job do not meet > these expectations. > The problem job use a class which contains a field of Class MapRecord, and > MapRecord is defined to extend HashMap so as to accept variable json data. > > Class MapRecord: > > @NoArgsConstructor > @Slf4j > public class MapRecord extends HashMap<Object, Object> implements > Serializable { > @Override > public void setTimestamp(Long timestamp) { > put("timestamp", timestamp); > put("server_time", timestamp); > } > > @Override > public Long getTimestamp() { > try { > Object ts = getOrDefault("timestamp", getOrDefault("server_time", > 0L)); > return ((Number) Optional.ofNullable(ts).orElse(0L)).longValue(); > } catch (Exception e) { > log.error("Error, MapRecord's timestamp invalid.", e); > return 0L; > } > } > } > > Class UserAccessLog: > > public class UserAccessLog extends AbstractRecord<UserAccessLog> { > private MapRecord d; // I think this is related to the problem... > > ... ... > > } > > > 赵一旦 <hinobl...@gmail.com> 于2021年2月3日周三 下午6:43写道： > >> Actually the exception is different every time I stop the job. >> Such as: >> (1) com.esotericsoftware.kryo.KryoException: Unable to find class: g^XT >> The stack as I given above. >> >> (2) java.lang.IndexOutOfBoundsException: Index: 46, Size: 17 >> 2021-02-03 18:37:24 >> java.lang.IndexOutOfBoundsException: Index: 46, Size: 17 >> at java.util.ArrayList.rangeCheck(ArrayList.java:657) >> at java.util.ArrayList.get(ArrayList.java:433) >> at com.esotericsoftware.kryo.util.MapReferenceResolver.getReadObject( >> MapReferenceResolver.java:42) >> at com.esotericsoftware.kryo.Kryo.readReferenceOrNull(Kryo.java:805) >> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:759) >> at com.esotericsoftware.kryo.serializers.MapSerializer.read( >> MapSerializer.java:135) >> at com.esotericsoftware.kryo.serializers.MapSerializer.read( >> MapSerializer.java:21) >> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) >> at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer >> .deserialize(KryoSerializer.java:346) >> at org.apache.flink.api.java.typeutils.runtime.PojoSerializer >> .deserialize(PojoSerializer.java:411) >> at org.apache.flink.streaming.runtime.streamrecord. >> StreamElementSerializer.deserialize(StreamElementSerializer.java:202) >> at org.apache.flink.streaming.runtime.streamrecord. >> StreamElementSerializer.deserialize(StreamElementSerializer.java:46) >> at org.apache.flink.runtime.plugable. >> NonReusingDeserializationDelegate.read(NonReusingDeserializationDelegate >> .java:55) >> at org.apache.flink.runtime.io.network.api.serialization. >> SpillingAdaptiveSpanningRecordDeserializer.getNextRecord( >> SpillingAdaptiveSpanningRecordDeserializer.java:92) >> at org.apache.flink.streaming.runtime.io.StreamTaskNetworkInput >> .emitNext(StreamTaskNetworkInput.java:145) >> at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor >> .processInput(StreamOneInputProcessor.java:67) >> at org.apache.flink.streaming.runtime.io.StreamTwoInputProcessor >> .processInput(StreamTwoInputProcessor.java:92) >> at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput( >> StreamTask.java:372) >> at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor >> .runMailboxLoop(MailboxProcessor.java:186) >> at org.apache.flink.streaming.runtime.tasks.StreamTask >> .runMailboxLoop(StreamTask.java:575) >> at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke( >> StreamTask.java:539) >> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:722) >> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:547) >> at java.lang.Thread.run(Thread.java:748) >> >> (3) com.esotericsoftware.kryo.KryoException: Encountered unregistered >> class ID: 96 >> com.esotericsoftware.kryo.KryoException: Encountered unregistered class >> ID: 96 >> at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass( >> DefaultClassResolver.java:119) >> at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:641) >> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:752) >> at com.esotericsoftware.kryo.serializers.MapSerializer.read( >> MapSerializer.java:135) >> at com.esotericsoftware.kryo.serializers.MapSerializer.read( >> MapSerializer.java:21) >> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) >> at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer >> .deserialize(KryoSerializer.java:346) >> at org.apache.flink.api.java.typeutils.runtime.PojoSerializer >> .deserialize(PojoSerializer.java:411) >> at org.apache.flink.streaming.runtime.streamrecord. >> StreamElementSerializer.deserialize(StreamElementSerializer.java:202) >> at org.apache.flink.streaming.runtime.streamrecord. >> StreamElementSerializer.deserialize(StreamElementSerializer.java:46) >> at org.apache.flink.runtime.plugable. >> NonReusingDeserializationDelegate.read(NonReusingDeserializationDelegate >> .java:55) >> at org.apache.flink.runtime.io.network.api.serialization. >> SpillingAdaptiveSpanningRecordDeserializer.getNextRecord( >> SpillingAdaptiveSpanningRecordDeserializer.java:92) >> at org.apache.flink.streaming.runtime.io.StreamTaskNetworkInput >> .emitNext(StreamTaskNetworkInput.java:145) >> at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor >> .processInput(StreamOneInputProcessor.java:67) >> at org.apache.flink.streaming.runtime.io.StreamTwoInputProcessor >> .processInput(StreamTwoInputProcessor.java:92) >> at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput( >> StreamTask.java:372) >> at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor >> .runMailboxLoop(MailboxProcessor.java:186) >> at org.apache.flink.streaming.runtime.tasks.StreamTask >> .runMailboxLoop(StreamTask.java:575) >> at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke( >> StreamTask.java:539) >> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:722) >> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:547) >> at java.lang.Thread.run(Thread.java:748) >> >> ... >> >> Till Rohrmann <trohrm...@apache.org> 于2021年2月3日周三 下午6:28写道： >> >>> Hi, >>> >>> could you show us the job you are trying to resume? Is it a SQL job or a >>> DataStream job, for example? >>> >>> From the stack trace, it looks as if the class g^XT is not on the class >>> path. >>> >>> Cheers, >>> Till >>> >>> On Wed, Feb 3, 2021 at 10:30 AM 赵一旦 <hinobl...@gmail.com> wrote: >>> >>>> I have a job, the checkpoint and savepoint all right. >>>> But, if I stop the job using 'stop -p', after the savepoint generated, >>>> then the job goes to fail. Here is the log: >>>> >>>> 2021-02-03 16:53:55,179 WARN org.apache.flink.runtime.taskmanager.Task >>>> [] - ual_ft_uid_subid_SidIncludeFilter -> >>>> ual_ft_uid_subid_Default >>>> PassThroughFilter[null, null) -> ual_ft_uid_subid_UalUidFtExtractor -> >>>> ual_ft_uid_subid_EmptyUidFilter (17/30)#0 >>>> (46abce5d1148b56094726d442df2fd9c) switched >>>> from RUNNING to FAILED. >>>> >>>> com.esotericsoftware.kryo.KryoException: Unable to find class: g^XT >>>> at >>>> com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:138) >>>> ~[flink-dist_2.11-1.12.0.jar:1.12.0] >>>> at >>>> com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:115) >>>> ~[flink-dist_2.11-1.12.0.jar:1.12.0] >>>> at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:641) >>>> ~[flink-dist_2.11-1.12.0.jar:1.12.0] >>>> at >>>> com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:752) >>>> ~[flink-dist_2.11-1.12.0.jar:1.12.0] >>>> at >>>> com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:143) >>>> ~[flink-dist_2.11-1.12.0.jar:1.12.0] >>>> at >>>> com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:21) >>>> ~[flink-dist_2.11-1.12.0.jar:1.12.0] >>>> at >>>> com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) >>>> ~[flink-dist_2.11-1.12.0.jar:1.12.0] >>>> at >>>> org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:346) >>>> ~[flink-dist_2.11-1.12.0.jar:1.12.0] >>>> at >>>> org.apache.flink.api.java.typeutils.runtime.PojoSerializer.deserialize(PojoSerializer.java:411) >>>> ~[flink-dist_2.11-1.12.0.jar:1.12.0] >>>> at >>>> org.apache.flink.streaming.runtime.streamrecord.StreamElementSerializer.deserialize(StreamElementSerializer.java:202) >>>> ~[flink-dist_2.11-1.12.0.jar:1.12.0] >>>> at >>>> org.apache.flink.streaming.runtime.streamrecord.StreamElementSerializer.deserialize(StreamElementSerializer.java:46) >>>> ~[flink-dist_2.11-1.12.0.jar:1.12.0] >>>> at >>>> org.apache.flink.runtime.plugable.NonReusingDeserializationDelegate.read(NonReusingDeserializationDelegate.java:55) >>>> ~[flink-dist_2.11-1.12.0.jar:1.12.0] >>>> at >>>> org.apache.flink.runtime.io.network.api.serialization.SpillingAdaptiveSpanningRecordDeserializer.getNextRecord(SpillingAdaptiveSpanningRecordDeserializer.java:92) >>>> ~[flink-dist_2.11-1.12.0.jar:1.12.0] >>>> at >>>> org.apache.flink.streaming.runtime.io.StreamTaskNetworkInput.emitNext(StreamTaskNetworkInput.java:145) >>>> ~[flink-dist_2.11-1.12.0.jar:1.12.0] >>>> at >>>> org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:67) >>>> ~[flink-dist_2.11-1.12.0.jar:1.12.0] >>>> at >>>> org.apache.flink.streaming.runtime.io.StreamTwoInputProcessor.processInput(StreamTwoInputProcessor.java:92) >>>> ~[flink-dist_2.11-1.12.0.jar:1.12.0] >>>> at >>>> org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:372) >>>> ~[flink-dist_2.11-1.12.0.jar:1.12.0] >>>> at >>>> org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:186) >>>> ~[flink-dist_2.11-1.12.0.jar:1.12.0] >>>> at >>>> org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:575) >>>> ~[flink-dist_2.11-1.12.0.jar:1.12.0] >>>> at >>>> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:539) >>>> ~[flink-dist_2.11-1.12.0.jar:1.12.0] >>>> at >>>> org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:722) >>>> [flink-dist_2.11-1.12.0.jar:1.12.0] >>>> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:547) >>>> [flink-dist_2.11-1.12.0.jar:1.12.0] >>>> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_251] >>>> Caused by: java.lang.ClassNotFoundException: g^XT >>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:382) >>>> ~[?:1.8.0_251] >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:418) >>>> ~[?:1.8.0_251] >>>> at >>>> org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:63) >>>> ~[flink-dist_2.11-1.12.0.jar:1.12.0] >>>> at >>>> org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:72) >>>> ~[flink-dist_2.11-1.12.0.jar:1.12.0] >>>> at >>>> org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:49) >>>> ~[flink-dist_2.11-1.12.0.jar:1.12.0] >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:351) >>>> ~[?:1.8.0_251] >>>> at >>>> org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.loadClass(FlinkUserCodeClassLoaders.java:168) >>>> ~[flink-dist_2.11-1.12.0.jar:1.12.0] >>>> at java.lang.Class.forName0(Native Method) ~[?:1.8.0_251] >>>> at java.lang.Class.forName(Class.java:348) ~[?:1.8.0_251] >>>> at >>>> com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:136) >>>> ~[flink-dist_2.11-1.12.0.jar:1.12.0] >>>> ... 22 more >>>> >>>>
