Hey all, A user recently reported issues reading Iceberg timestamps with the Python SDK. As I investigated, I noticed some gaps in our timestamp story for IcebergIO (and potentially other IOs).
I've written a design doc to address these challenges specifically for IcebergIO. The goal is to establish a more consistent and robust timestamp strategy that also supports the upcoming nanosecond-precision timestamps in the Iceberg v3 spec [1]. The doc outlines current gaps and proposes a few approaches, including a preferred one that uses new logical types to ensure accuracy and flexibility. It also details potential breaking changes and our plan for managing them. Please take a look and share your feedback: https://docs.google.com/document/d/19wwp9-4WyE8Ctao0tb1kKCppR4NtvscZ2P2yjokALfQ/edit?usp=sharing [1] https://iceberg.apache.org/spec/#version-3-extended-types-and-capabilities
