Hey everyone, Thank you all for the valuable feedback!
I'm writing to follow up because the scope of the proposal has expanded quite a bit. It started as a strategy for the IcebergIO connector but has evolved into a general approach for handling timestamps and datetimes across all of Beam. The updated document now introduces a new *Proposal 2: introducing flexible, parameterized logical types like Timestamp(precision) and DateTime(precision)*. This creates a more consistent and scalable way to manage multiple precisions across SDKs. Could you please take another look? I’m especially seeking feedback for the *“Python Nanosecond Support”* section to help decide the best path forward for representing nanosecond precision in the Python SDK. Link: https://s.apache.org/beam-timestamp-strategy Thank you, Ahmed On Tue, Aug 12, 2025 at 3:16 PM Yi Hu via dev <dev@beam.apache.org> wrote: > +1 Thank you Ahmed! This has been a long standing issue. Hopefully we can > finally resolve it! > > On Tue, Aug 12, 2025 at 1:10 PM Chamikara Jayalath via dev < > dev@beam.apache.org> wrote: > >> Thanks Ahmed. We have run into portable timestamp/instant related issues >> several times before (Iceberg, JDBCIO etc.) and it's good to see a proposal >> that takes a detailed look at this. >> Added some comments. >> >> - Cham >> >> On Tue, Aug 12, 2025 at 8:53 AM Tarun Annapareddy via dev < >> dev@beam.apache.org> wrote: >> >>> +1 good design doc. Thanks Ahmed ! >>> >>> Thank you, >>> Tarun. >>> >>> >>> >>> On Mon, Aug 11, 2025 at 1:34 PM Ahmed Abualsaud via dev < >>> dev@beam.apache.org> wrote: >>> >>>> Hey all, >>>> >>>> A user recently reported issues reading Iceberg timestamps with the >>>> Python SDK. As I investigated, I noticed some gaps in our timestamp story >>>> for IcebergIO (and potentially other IOs). >>>> >>>> I've written a design doc to address these challenges specifically for >>>> IcebergIO. The goal is to establish a more consistent and robust timestamp >>>> strategy that also supports the upcoming nanosecond-precision timestamps in >>>> the Iceberg v3 spec [1]. >>>> >>>> The doc outlines current gaps and proposes a few approaches, including >>>> a preferred one that uses new logical types to ensure accuracy and >>>> flexibility. It also details potential breaking changes and our plan for >>>> managing them. >>>> >>>> Please take a look and share your feedback: >>>> >>>> https://docs.google.com/document/d/19wwp9-4WyE8Ctao0tb1kKCppR4NtvscZ2P2yjokALfQ/edit?usp=sharing >>>> >>>> [1] >>>> https://iceberg.apache.org/spec/#version-3-extended-types-and-capabilities >>>> >>>> >>>
