We just added some documentation and examples around the C++ ExecPlan:
- https://arrow.apache.org/docs/dev/cpp/streaming_execution.html
-
https://github.com/apache/arrow/blob/master/cpp/examples/arrow/execution_plan_documentation_examples.cc
For this, "cast" should work. Here's an example in Python (not using ExecPlan):
>>> import pyarrow as pa
>>> import pyarrow.compute as pc
>>> from datetime import datetime
>>> arr = pa.array([datetime(2022, 2, 1, 5)])
>>> arr
<pyarrow.lib.TimestampArray object at 0x7f00805d21c0>
[
2022-02-01 05:00:00.000000
]
>>> pc.assume_timezone(arr, 'America/New_York')
<pyarrow.lib.TimestampArray object at 0x7f00805f1880>
[
2022-02-01 10:00:00.000000
]
>>> pc.assume_timezone(arr, 'America/New_York').to_pylist()
[datetime.datetime(2022, 2, 1, 5, 0, tzinfo=<DstTzInfo 'America/New_York' EST-1
day, 19:00:00 STD>)]
>>> pc.assume_timezone(arr, 'America/New_York').cast(pa.time64('ns'))
<pyarrow.lib.Time64Array object at 0x7f00827a2e20>
[
05:00:00.000000000
]
>>> arr2 = pa.array([time(5,0)], pa.time64('ns'))
>>> pc.equal(pc.assume_timezone(arr, 'America/New_York').cast(pa.time64('ns')),
>>> arr2)
<pyarrow.lib.BooleanArray object at 0x7f00805f1880>
[
true
]
In C++, a ProjectNode and an Expression could be used to compute this,
something like
using cp = arrow::compute;
auto expr = cp::call("equal", {cp::call("cast", {cp::call("assume_timezone",
{cp::field_ref("input")}, cp::AssumeTimezoneOptions(...))},
CastOptions::Safe(arrow::time64(TimeUnit::NANOSECOND))),
cp::literal(arrow::MakeScalar(...))});
-David
On Thu, Feb 3, 2022, at 09:54, Li Jin wrote:
> Hello!
>
> I am new to the Arrow C++ compute engine and trying to figure out this time
> zone conversion and time extraction:
>
> t.dt.tz_convert('America/New_York').dt.time == datetime.time(11, 30, 0)
>
> So I started looking at:
> https://github.com/apache/arrow/blob/master/cpp/src/arrow/compute/kernels/scalar_temporal_unary.cc
>
> and found these these functions seem relevant:
> assume_timezone
> hour
> minute
>
> So my thinking is trying to figure out a way to build plan that basically
> does these steps:
> (1) Assume timezone to New_York (input data is UTC)
> (2) Extract hour value
> (3) Extract minute value
> (4) Filter on hour and minute value
>
> I wonder what is a good way to map these functions in
> scalar_temporal_unary to an ExecPlan? (Looked under
> https://github.com/apache/arrow/tree/master/cpp/src/arrow/compute/exec but
> didn't see anything obvious)
>
> Thanks!
> Li
