Hi team, Happy New Year! I have a question regarding the "org.apache.beam:beam-sdks-java-io-iceberg:2.61.0".
Does the org.apache.beam.sdk.schemas.Schema getDataSchema(String destination) of class DynamicDestinations already exist? Our team tried to use DynamicDestinations to decide which iceberg table to write in the runtime but failed. Attached is my code class DynamicIcebergDestinations. Our problem is that we don't have a universal schema that can match all events to be implemented for getDataSchema(). We need to use a parameter to get the correct schema for each event, so we want to use: getDataSchema(String destination). But seems getDataSchema(String destination) is not implemented for org.apache.beam.sdk.io.iceberg.DynamicDestinations. Since getDataSchema() will be automatically called and it cannot return a universal schema, there is no way for us to use IcebergIO.writeRows(icebergCatalogConfig).to(dynamicDestinations). I also tried to use managed I/O connector, but I don't think Managed.write support with DynamicDestinations. Could you give some urgent guidance? Thank you so much. Sincerely, Luyao
DynamicIcebergDestinations.java
Description: Binary data
