Thanks Weston for the insight - for the short term we are going to try to
unify the time unit to "microseconds" to be compatible with substrait and
pay the cost of converting to nanoseconds (e.g., when passing to pandas)
when needed.
Longer term I think option (3) is probably the most practical (a
The Substrait decision for microseconds was made because, at the time, the
goal was to keep the type system simple and universal, and there were
systems that didn't support ns (e.g. Iceberg, postgres, duckdb, velox).
A few options (off the top of my head):
1. Attempt to get a nanoseconds timesta
Hi,
I recently came across some limitations in expressing timestamp type with
Substrait in the Acero substrait consumer and am curious to hear what
people's thoughts are.
The particular issue that I have is when specifying timestamp type in
substrait, the unit is "microseconds" and there is no wa
