Re: [Discuss] Handling timezones in (C++) compute kernels for timestamp data

As there was a lot of discussion around timestamp localization I'd like to point out there is an open PR for it now [1]. [1] https://github.com/apache/arrow/pull/10610 Rok On Thu, Jun 10, 2021 at 11:11 PM Wes McKinney wrote: > > I agree that we need to implement the equivalent of pandas's > "tz

Re: [Discuss] Handling timezones in (C++) compute kernels for timestamp data

I agree that we need to implement the equivalent of pandas's "tz_localize" method which performs UTC normalization on tz-naive data and sets the timezone field. Here's a demo of this functionality (I originally implemented this years ago by porting pytz's logic to run against NumPy arrays in Cython

Re: [Discuss] Handling timezones in (C++) compute kernels for timestamp data

On Thu, 10 Jun 2021 at 18:06, Antoine Pitrou wrote: > > On Thu, 10 Jun 2021 17:33:23 +0200 > Joris Van den Bossche wrote: > > > > We just merged a PR to add some kernels to extract fields from timestamps > > (year, month, day, hour, etc -> ARROW-11759 > >

Re: [Discuss] Handling timezones in (C++) compute kernels for timestamp data

On Thu, 10 Jun 2021 17:33:23 +0200 Joris Van den Bossche wrote: > > We just merged a PR to add some kernels to extract fields from timestamps > (year, month, day, hour, etc -> ARROW-11759 > ). But once you start with > kernels for timestamp data, you qu

[Discuss] Handling timezones in (C++) compute kernels for timestamp data

Hi all, There was recently a discussion on the interpretation of the spec about the "timezone" field of timestamp type (and different timestamp-related types that Arrow should have). See https://lists.apache.org/thread.html/r017084eed74edbc95810fc049056570f45b0bb034d6eeadd647e8621%40%3Cdev.arrow.a