Re: 想知道state写到checkpoint文件没有

Tue, 21 Jul 2020 22:19:15 -0700 

Hi
    Checkpoint 包括两部分：1）meta 文件；2）具体的数据。如果是 Meta 部分可以参考
CheckpointMetadataLoadingTest[1] 自己写一个测试，如果你知道具体的内容，或许也可以看一下
StatePorcessAPI[2]

[1]
https://github.com/apache/flink/blob/master/flink-runtime/src/test/java/org/apache/flink/runtime/checkpoint/CheckpointMetadataLoadingTest.java
[2]
https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/libs/state_processor_api.html

Best,
Congxian


sun <[email protected]> 于2020年7月21日周二 下午12:02写道：

> &nbsp; 请问怎么反编译checkpoint文件啊，我想知道state写到checkpoint文件没有
>
>
>
>
> &nbsp;          _default_&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
> OPERATOR_STATE_DISTRIBUTION_MODE  SPLIT_DISTRIBUTE&nbsp; &nbsp;
>  VALUE_SERIALIZER&nbsp; &nbsp;
> Gorg.apache.flink.api.common.typeutils.ParameterlessTypeSerializerConfigzS酿&nbsp;
> &nbsp; &nbsp; &nbsp;脂? sr
> -org.apache.flink.runtime.state.JavaSerializerFSX韦4
> ?&nbsp; xr
> Borg.apache.flink.api.common.typeutils.base.TypeSerializerSingletony﹪.wE
> &nbsp; xr 4org.apache.flink.api.common.typeutils.TypeSerializer&nbsp;
> &nbsp; &nbsp; &nbsp;  &nbsp; xp&nbsp; &nbsp;
> -org.apache.flink.runtime.state.JavaSerializer
> topic-partition-offset-states&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
> OPERATOR_STATE_DISTRIBUTION_MODE  UNION&nbsp; &nbsp;
>  VALUE_SERIALIZER&nbsp; &nbsp;
> Iorg.apache.flink.api.java.typeutils.runtime.TupleSerializerConfigSnapshotzS酿&nbsp;
> &nbsp; &nbsp;  矛? sr
> ;org.apache.flink.api.java.typeutils.runtime.TupleSerializer&nbsp; &nbsp;
> &nbsp; &nbsp;  &nbsp; xr
> ?org.apache.flink.api.java.typeutils.runtime.TupleSerializerBase&nbsp;
> &nbsp; &nbsp; &nbsp;    I  arityI  length[  fieldSerializerst
> 7[Lorg/apache/flink/api/common/typeutils/TypeSerializer;L&nbsp;
> tupleClasst  Ljava/lang/Class;xr
> 4org.apache.flink.api.common.typeutils.TypeSerializer&nbsp; &nbsp; &nbsp;
> &nbsp;  &nbsp; xp&nbsp; &nbsp; r
> 7[Lorg.apache.flink.api.common.typeutils.TypeSerializer;9?Ч 麡 &nbsp;
> xp&nbsp; &nbsp; sr
> ?org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer&nbsp;
> &nbsp; &nbsp; &nbsp;    L  defaultSerializerClassest
> Ljava/util/LinkedHashMap;L  defaultSerializersq ~  L  kryoRegistrationsq ~
>        L  registeredTypest  Ljava/util/LinkedHashSet;L
> $registeredTypesWithSerializerClassesq ~        L
> registeredTypesWithSerializersq ~    L  typeq ~  xq ~  sr
> java.util.LinkedHashMap4繬\ l利   Z  accessOrderxr  java.util.HashMap  诹?`?
> F&nbsp;
> loadFactorI     thresholdxp?@&nbsp; &nbsp; &nbsp; w &nbsp; &nbsp; &nbsp;
> &nbsp; x sq ~ ?@&nbsp; &nbsp; &nbsp; w &nbsp; &nbsp; &nbsp; &nbsp; x sq ~
> ?@&nbsp; &nbsp; &nbsp; w &nbsp; &nbsp; &nbsp; &nbsp; t
> Iorg.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartitionsr
> <org.apache.flink.api.java.typeutils.runtime.KryoRegistrationJ?坣厏o   L
> registeredClassq ~  L  serializableSerializerInstancet
> DLorg/apache/flink/api/common/ExecutionConfig$SerializableSerializer;L
> serializerClassq ~  L  serializerDefinitionTypet
> WLorg/apache/flink/api/java/typeutils/runtime/KryoRegistration$SerializerDefinitionType;xpvr
> Iorg.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition
>  [/剀X5   I&nbsp;
> cachedHashI     partitionL  topict  Ljava/lang/String;xppp~r
> Uorg.apache.flink.api.java.typeutils.runtime.KryoRegistration$SerializerDefinitionType&nbsp;
> &nbsp; &nbsp; &nbsp;  &nbsp; xr  java.lang.Enum&nbsp; &nbsp; &nbsp; &nbsp;
> &nbsp; xpt  UNSPECIFIEDt )org.apache.avro.generic.GenericData$Arraysq ~  vr
> Uorg.apache.flink.api.java.typeutils.runtime.kryo.Serializers$DummyAvroRegisteredClass&nbsp;
> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;xppvr
> Yorg.apache.flink.api.java.typeutils.runtime.kryo.Serializers$DummyAvroKryoSerializerClass&nbsp;
> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;xp~q ~  t  CLASSx pppq ~  sr
> 9org.apache.flink.api.common.typeutils.base.LongSerializer&nbsp; &nbsp;
> &nbsp; &nbsp;  &nbsp; xr
> Borg.apache.flink.api.common.typeutils.base.TypeSerializerSingletony﹪.wE
> &nbsp; xq ~  vr &amp;org.apache.flink.api.java.tuple.Tuple2&nbsp; &nbsp;
> &nbsp; &nbsp;    L  f0t  Ljava/lang/Object;L  f1q ~ )xr
> %org.apache.flink.api.java.tuple.Tuple&nbsp; &nbsp; &nbsp; &nbsp;  &nbsp;
> xp&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;  ;&nbsp;  ?  ?  )&nbsp;
> &nbsp; &nbsp;  3  sr
> ?org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer&nbsp;
> &nbsp; &nbsp; &nbsp;    L  defaultSerializerClassest
> Ljava/util/LinkedHashMap;L  defaultSerializersq ~  L  kryoRegistrationsq ~
> L  registeredTypest  Ljava/util/LinkedHashSet;L
> $registeredTypesWithSerializerClassesq ~  L
> registeredTypesWithSerializersq ~  L  typet  Ljava/lang/Class;xr
> 4org.apache.flink.api.common.typeutils.TypeSerializer&nbsp; &nbsp; &nbsp;
> &nbsp;  &nbsp; xpsr  java.util.LinkedHashMap4繬\ l利   Z  accessOrderxr
> java.util.HashMap  诹?`?  F&nbsp;
> loadFactorI     thresholdxp?@&nbsp; &nbsp; &nbsp; w &nbsp; &nbsp; &nbsp;
> &nbsp; x sq ~  ?@&nbsp; &nbsp; &nbsp; w &nbsp; &nbsp; &nbsp; &nbsp; x sq ~
> ?@&nbsp; &nbsp; &nbsp; w &nbsp; &nbsp; &nbsp; &nbsp; t
> Iorg.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartitionsr
> <org.apache.flink.api.java.typeutils.runtime.KryoRegistrationJ?坣厏o   L
> registeredClassq ~  L  serializableSerializerInstancet
> DLorg/apache/flink/api/common/ExecutionConfig$SerializableSerializer;L
> serializerClassq ~  L  serializerDefinitionTypet
> WLorg/apache/flink/api/java/typeutils/runtime/KryoRegistration$SerializerDefinitionType;xpvr
> Iorg.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition
>  [/剀X5   I&nbsp;
> cachedHashI     partitionL  topict  Ljava/lang/String;xppp~r
> Uorg.apache.flink.api.java.typeutils.runtime.KryoRegistration$SerializerDefinitionType&nbsp;
> &nbsp; &nbsp; &nbsp;  &nbsp; xr  java.lang.Enum&nbsp; &nbsp; &nbsp; &nbsp;
> &nbsp; xpt  UNSPECIFIEDt )org.apache.avro.generic.GenericData$Arraysq ~ vr
> Uorg.apache.flink.api.java.typeutils.runtime.kryo.Serializers$DummyAvroRegisteredClass&nbsp;
> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;xppvr
> Yorg.apache.flink.api.java.typeutils.runtime.kryo.Serializers$DummyAvroKryoSerializerClass&nbsp;
> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;xp~q ~  t  CLASSx pppq ~  &nbsp; &nbsp;
> \org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer$KryoSerializerConfigSnapshotzS酿&nbsp;
> &nbsp; &nbsp;  3  sr
> ?org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer&nbsp;
> &nbsp; &nbsp; &nbsp;    L  defaultSerializerClassest
> Ljava/util/LinkedHashMap;L  defaultSerializersq ~  L  kryoRegistrationsq ~
> L  registeredTypest  Ljava/util/LinkedHashSet;L
> $registeredTypesWithSerializerClassesq ~  L
> registeredTypesWithSerializersq ~  L  typet  Ljava/lang/Class;xr
> 4org.apache.flink.api.common.typeutils.TypeSerializer&nbsp; &nbsp; &nbsp;
> &nbsp;  &nbsp; xpsr  java.util.LinkedHashMap4繬\ l利   Z  accessOrderxr
> java.util.HashMap  诹?`?  F&nbsp;
> loadFactorI     thresholdxp?@&nbsp; &nbsp; &nbsp; w &nbsp; &nbsp; &nbsp;
> &nbsp; x sq ~  ?@&nbsp; &nbsp; &nbsp; w &nbsp; &nbsp; &nbsp; &nbsp; x sq ~
> ?@&nbsp; &nbsp; &nbsp; w &nbsp; &nbsp; &nbsp; &nbsp; t
> Iorg.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartitionsr
> <org.apache.flink.api.java.typeutils.runtime.KryoRegistrationJ?坣厏o   L
> registeredClassq ~  L  serializableSerializerInstancet
> DLorg/apache/flink/api/common/ExecutionConfig$SerializableSerializer;L
> serializerClassq ~  L  serializerDefinitionTypet
> WLorg/apache/flink/api/java/typeutils/runtime/KryoRegistration$SerializerDefinitionType;xpvr
> Iorg.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition
>  [/剀X5   I&nbsp;
> cach发送edHashI   partitionL  topict  Ljava/lang/String;xppp~r
> Uorg.apache.flink.api.java.typeutils.runtime.KryoRegistration$SerializerDefinitionType&nbsp;
> &nbsp; &nbsp; &nbsp;  &nbsp; xr  java.lang.Enum&nbsp; &nbsp; &nbsp; &nbsp;
> &nbsp; xpt  UNSPECIFIEDt )org.apache.avro.generic.GenericData$Arraysq ~ vr
> Uorg.apache.flink.api.java.typeutils.runtime.kryo.Serializers$DummyAvroRegisteredClass&nbsp;
> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;xppvr
> Yorg.apache.flink.api.java.typeutils.runtime.kryo.Serializers$DummyAvroKryoSerializerClass&nbsp;
> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;xp~q ~  t  CLASSx pppq ~  &nbsp; &nbsp;
> Iorg.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition&nbsp;
> &nbsp;
> Iorg.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition
> Iorg.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition&nbsp;
> &nbsp; &nbsp;)org.apache.avro.generic.GenericData$Array
> Uorg.apache.flink.api.java.typeutils.runtime.kryo.Serializers$DummyAvroRegisteredClass&nbsp;
> &nbsp;
> Yorg.apache.flink.api.java.typeutils.runtime.kryo.Serializers$DummyAvroKryoSerializerClass&nbsp;
> &nbsp; &nbsp; &nbsp;猬? sr
> 9org.apache.flink.api.common.typeutils.base.LongSerializer&nbsp; &nbsp;
> &nbsp; &nbsp;  &nbsp; xr
> Borg.apache.flink.api.common.typeutils.base.TypeSerializerSingletony﹪.wE
> &nbsp; xr 4org.apache.flink.api.common.typeutils.TypeSerializer&nbsp;
> &nbsp; &nbsp; &nbsp;  &nbsp; xp&nbsp; &nbsp;
> Porg.apache.flink.api.common.typeutils.base.LongSerializer$LongSerializerSnapshot&nbsp;
> &nbsp;  9org.apache.flink.api.common.typeutils.base.LongSerializer  vr
> &amp;org.apache.flink.api.java.tuple.Tuple2&nbsp; &nbsp; &nbsp; &nbsp;
> L  f0t  Ljava/lang/Object;L  f1q ~  xr
> %org.apache.flink.api.java.tuple.Tuple&nbsp; &nbsp; &nbsp; &nbsp;  &nbsp;
> xp&nbsp;&nbsp;

回复