I also found these pages:
https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/schema_evolution.html
https://flink.apache.org/news/2020/04/15/flink-serialization-tuning-vol-1.html#avro
I hope this helps.
Regards,
Timo
On 30.04.21 13:20, Sandeep khanzode wrote:
Hi Timo,
Thanks! I will take a look at the links.
Can you please share if you have any simple (or complex) example of Avro state
data structures?
Thanks,
Sandeep
On 30-Apr-2021, at 4:46 PM, Timo Walther <twal...@apache.org> wrote:
Hi Sandeep,
did you have a chance to look at this documentation page?
https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/custom_serialization.html
The interfaces might not be easy to implement but are very powerful to address
compatibility issues. You can also look into Flink serializers for some
examples:
https://github.com/apache/flink/tree/master/flink-core/src/main/java/org/apache/flink/api/java/typeutils/runtime
Esp:
https://github.com/apache/flink/blob/89c6c03660a88a648bbd13b4e6696124fe46d013/flink-core/src/main/java/org/apache/flink/api/java/typeutils/runtime/PojoSerializer.java#L599
For the POJO logic.
By the way, usually we recommend Avro for state data structures if schema
evolution is a topic.
Regards,
Timo
On 29.04.21 18:10, Sandeep khanzode wrote:
Hello,
Is there a working example of a TypeSerializer for a Java type stored in the
State?
My requirement is that I should be able to store the Java POJO entity in the
MapState. The state is backed by RocksDBBackend.
If I update the entity with a new member variable, I am unable to deserialise
the state into the new entity.
I checked this link.
https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/types_serialization.html
<https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/types_serialization.html>
It does mention that the POJO type is special based on the rules. Does that
mean that I can add or remove member variables for the POJO? I have been unable
to get it to work.
Thanks,
Sandeep
