Hi all, I’d like to initiate a discussion about enhancing state schema evolution support for RowData in Flink.
*Motivation* Flink applications frequently need to evolve their state schema as business requirements change. Currently, when users update a Table API or SQL job with schema changes involving RowData types (particularly nested structures), they encounter serialization compatibility errors during state restoration, causing job failures.The issue occurs because existing state migration mechanisms don't properly handle RowData types during schema evolution, preventing users from making backward-compatible changes like: - Adding nullable fields to existing structures - Reordering fields within a row while preserving field names - Evolving nested row structures This limitation impacts production applications using Flink's Table API, as the RowData type is central to this interface. Users are forced to choose between maintaining outdated schemas or reprocessing all state data when schema changes are required. Here’s the proposal document: Link <https://docs.google.com/document/d/1WtAxp-jAVTLMOfWNldLCAoK137P0ZCMxR8hOZGcMxuc/edit?tab=t.0> Your feedback and ideas are welcome to refine this feature. Thanks, Weiqing
