Hi Soojin, > Why have the timezone info in the TimeStampMilliTZHolder if it's never set? My guess is this is an oversight (it should probably be removed).
> - What if each of the rows have different TZ info? Are we to shift to the tz on the TimeStampMilliTZVector on the write? Currently there is no data type in the Arrow spec that allows for a timezone per row. How you handle this would be application specific. Shifting to a standard timezone is one solution, using a custom encoding with extension types [1] is another. > How do we know what the original TZ info was if we enforce the column wide set tz? Again this is application specific. There was an old thread discussing this limitation [2] and it doesn't seem like anyone invested the effort to address the limitation. The options would be to add another column, use an extension type as mentioned above, or propose a new type. The last option is non-trivial, it would require an addition to the specification to be approved and reference implementations provided. Hope this helps, Micah [1] https://arrow.apache.org/docs/format/Columnar.html#extension-types [2] https://lists.apache.org/thread.html/5feaadc74e6f284f55e79ae2104621af519e6d44607162fec8f72ef7%40%3Cdev.arrow.apache.org%3E On Thu, Mar 12, 2020 at 4:11 PM Soojin Jeong <[email protected]> wrote: > Cc Anthony > > On Thu, Mar 12, 2020 at 5:27 PM Soojin Jeong <[email protected]> > wrote: > > > Hi team, > > > > I would like to know what the recommended way of using TimeStampMilliTZ > > is. > > > > I see that > > - TimeStampMilliTZVector contains timestamp arrow type with timezone > > - TimeStampMilliTZVector's set doesn't actually pass in the timezone info > > in TimeStampMilliTZHolder > > > > This raises questions on > > - Why have the timezone info in the TimeStampMilliTZHolder if it's never > > set? > > - What if each of the rows have different TZ info? Are we to shift to the > > tz on the TimeStampMilliTZVector on the write? > > - How do we know what the original TZ info was if we enforce the column > > wide set tz? > > > > I couldnt find any usages or unit tests, and seems like the holder code > > was generated. Any guidance is much appreciated. > > > > Thanks > > > > -- > > Soojin Jeong > > > -- > Soojin Jeong >
