Hello everyone, This question is related to schema evolution support in Iceberg.
We have data coming in with fields out-of-order wrt to the schema in Iceberg (e.g. inbound struct(a,b,c) vs. iceberg struct(c,b,a)) As a result we are hitting the following error in Iceberg when saving the data -> "Cannot write incompatible dataset to table with schema", generated within the IcebergeSource -> https://github.com/apache/incubator-iceberg/blob/d1f0b540f5f14f002be86133ef9f66445f7e0926/spark/src/main/java/org/apache/iceberg/spark/source/IcebergSource.java#L157 I also noted in the documentation that re-ordering was allowed -> https://iceberg.apache.org/evolution/ , which led me to believe that we could update the schema prior to writing the data, However, I see no means of re-ordering fields on the current UpdateSchema API. How are people handling out-of-order fields today? Our data is deeply nested, as a result I am hoping not to have to transform/prep on ingest and looking for alternatives. Any thoughts appreciated! Regards, Shone Sadler
