+1 thanks for the writeup, it is helpful and I think I'm aligned with the direction (pending some of the threads).
Thanks, Danny On Tue, Aug 12, 2025 at 9:03 AM Kenneth Knowles <k...@apache.org> wrote: > Thanks for writing this up. In addition to being a clear proposal, this is > a great explanation of some of the intricacies of timestamps in Beam > schemas. > > Kenn > > On Mon, Aug 11, 2025 at 4:34 PM Ahmed Abualsaud via dev < > dev@beam.apache.org> wrote: > >> Hey all, >> >> A user recently reported issues reading Iceberg timestamps with the >> Python SDK. As I investigated, I noticed some gaps in our timestamp story >> for IcebergIO (and potentially other IOs). >> >> I've written a design doc to address these challenges specifically for >> IcebergIO. The goal is to establish a more consistent and robust timestamp >> strategy that also supports the upcoming nanosecond-precision timestamps in >> the Iceberg v3 spec [1]. >> >> The doc outlines current gaps and proposes a few approaches, including a >> preferred one that uses new logical types to ensure accuracy and >> flexibility. It also details potential breaking changes and our plan for >> managing them. >> >> Please take a look and share your feedback: >> >> https://docs.google.com/document/d/19wwp9-4WyE8Ctao0tb1kKCppR4NtvscZ2P2yjokALfQ/edit?usp=sharing >> >> [1] >> https://iceberg.apache.org/spec/#version-3-extended-types-and-capabilities >> >> >
