Re: [C++] Implement function for time zone conversion

2022-02-07 Thread David Li
Ah - the Result is for error handling. So either: 1) Return arrow::Status from this function, and use the ARROW_ASSIGN_OR_RAISE macro: ARROW_ASSIGN_OR_RAISE(auto scalar, arrow::MakeScalar(...)); ..., cp::literal(std::move(scalar)) ... 2) For experimentation purposes, just assume it'll succeed:

Re: [C++] Implement function for time zone conversion

2022-02-07 Thread Li Jin
So I think I am getting close... now I have cp::Expression predicate = cp::call( "equal", {...} cp::literal(arrow::MakeScalar(arrow::time64(arrow::TimeUnit::NANO), time))} ); And

Re: [C++] Implement function for time zone conversion

2022-02-07 Thread Li Jin
Another question - what is the correct way of creating a constant Scalar that represents a time64 from a string in C++ (e.g. "10:00:00") ? I am trying to use cp::literal(arrow::MakeScalar(...)) but not sure what to fill in the body of MakeScalar On Mon, Feb 7, 2022 at 12:05 PM Li Jin wrote: > H

Re: [C++] Implement function for time zone conversion

2022-02-07 Thread Li Jin
Hello, Did some more work on this. Follow up question on this: I am doing a {cp::call("cast", {cp::field_ref("time_ns")}, cp::CastOptions::Safe(arrow::timestamp(arrow::TimeUnit::NANO, timezone)))}, And got: NotImplemented: Unsupported cast from

Re: [C++] Implement function for time zone conversion

2022-02-03 Thread Rok Mihevc
By the way - if you're extracting components to segment time you might prefer temporal rounding https://arrow.apache.org/docs/dev/cpp/compute.html?highlight=floor_temporal#conversions. On Thu, Feb 3, 2022 at 5:19 PM Li Jin wrote: > > Gotcha. Thanks for the pointer! > > On Thu, Feb 3, 2022 at 11:0

Re: [C++] Implement function for time zone conversion

2022-02-03 Thread Li Jin
Gotcha. Thanks for the pointer! On Thu, Feb 3, 2022 at 11:06 AM Rok Mihevc wrote: > > Gotcha. "Assume timezone" is like "tz_localize" in pandas. > > Indeed! > > > I didn't see any target timezone information being passed in the "cast" > > function - how would I do that? > > Just cast with: > aut

Re: [C++] Implement function for time zone conversion

2022-02-03 Thread Rok Mihevc
> Gotcha. "Assume timezone" is like "tz_localize" in pandas. Indeed! > I didn't see any target timezone information being passed in the "cast" > function - how would I do that? Just cast with: auto options = CastOptions::Safe(timestamp(TimeUnit::NANO, "America/New_York")); See examples in: https

Re: [C++] Implement function for time zone conversion

2022-02-03 Thread Li Jin
Gotcha. "Assume timezone" is like "tz_localize" in pandas. > If you are starting from a UTC-zoned timestamp you can just cast it, as David suggests, to the desired timezone (this is a metadata only change) I didn't see any target timezone information being passed in the "cast" function - how woul

Re: [C++] Implement function for time zone conversion

2022-02-03 Thread Rok Mihevc
Assume_timezone will "assume timezone" of the local time you pass it and give you a zoned timestamp. This means your timestamps will be interpreted as local times and converted to UTC in the background. Resulting array will have timezone metadata of your assumed timezone. If you are starting from

Re: [C++] Implement function for time zone conversion

2022-02-03 Thread Li Jin
David - Thanks for the pointers. I didn't know you could cast a timestamp to time type to extract the hour/minute information. Nice! Rok - Not sure I understand what you mean... My input is UTC but I want to extract the time information local to New York Timezone (e.g. filter time to 10 AM New Yo

Re: [C++] Implement function for time zone conversion

2022-02-03 Thread Rok Mihevc
Hey Li, If your input data is in UTC you don't need assume_timezone [1]. You would need it if your input was America/New_York local time and you wanted to convert to a zoned timestamp array where underlying data is in UTC and timezone is metadata only. Perhaps python tests are interesting for refe

Re: [C++] Implement function for time zone conversion

2022-02-03 Thread David Li
We just added some documentation and examples around the C++ ExecPlan: - https://arrow.apache.org/docs/dev/cpp/streaming_execution.html - https://github.com/apache/arrow/blob/master/cpp/examples/arrow/execution_plan_documentation_examples.cc For this, "cast" should work. Here's an example in Pyt

[C++] Implement function for time zone conversion

2022-02-03 Thread Li Jin
Hello! I am new to the Arrow C++ compute engine and trying to figure out this time zone conversion and time extraction: t.dt.tz_convert('America/New_York').dt.time == datetime.time(11, 30, 0) So I started looking at: https://github.com/apache/arrow/blob/master/cpp/src/arrow/compute/kernels/scala