On 14.09.20 02:20, Steven Wu wrote:
Right now, we use UnionState to store the `nextCheckpointId` in the Iceberg sink use case, because we can't retrieve the checkpointId from the FunctionInitializationContext during the restore case. But we can move away from it if the restore context provides the checkpointId.
Is the code for this available in the open source? I checked the Iceberg sink that's available in Iceberg proper and the one in Netflix Skunkworks: https://github.com/Netflix-Skunkworks/nfflink-connector-iceberg/blob/master/nfflink-connector-iceberg/src/main/java/com/netflix/spaas/nfflink/connector/iceberg/sink/IcebergCommitter.java#L228
Both of them are only using operator state, not the union variant. Best, Aljoscha
