Yes.  Something like:

if (ErrorIfNotOk(flight_writer->WriteRecordBatch(...))) return;

Today this method calls `output->ErrorReceived(...)`.  The original
idea (I think) is that, possibly, a downstream node could "handle" the
error.  However, in practice, nothing does that, and all errors
propagate downstream to the sink node.  The sink node will then abort
the plan.

We could potentially add an ACERO_RETURN_NOT_OK(...) macro for this
but Sasha has a proposal in [1] which simplifies error handling.  With
this proposal InputReceived returns a status.  If that status is not
valid then the plan is aborted.  So you can just use
ARROW_RETURN_NOT_OK as you do elsewhere.  I believe [1] is currently
blocked slightly by [2].

[1] https://github.com/apache/arrow/pull/13848
[2] https://issues.apache.org/jira/browse/ARROW-17509

On Tue, Oct 18, 2022 at 2:37 PM Yaron Gvili <rt...@hotmail.com> wrote:
>
> Hi Li,
>
> One way I've seen (which hopefully is the right way) is invoking 
> `ExecNode::ErrorIfNotOk(Status)`. If `WriteRecordBatch` returns a `Status` 
> then just pass it; if it returns a `Result` then you can pass its `.status()`.
>
>
> Yaron.
> ________________________________
> From: Li Jin <ice.xell...@gmail.com>
> Sent: Tuesday, October 18, 2022 5:19 PM
> To: dev@arrow.apache.org <dev@arrow.apache.org>
> Subject: [Acero] Error handling in ExecNode
>
> Hello!
>
> I am trying to implement an ExecNode in Acero that receives the input
> batch, writes the batch to the FlightStreamWriter and then passes the batch
> to the downstream node.
>
> Looking at the API, I am thinking of doing sth like :
>
> void InputReceived(ExecNode* input, ExecBatch batch) {
>     # turn batch to RecordBatch
>     # call flight_writer->WriteRecordBatch
>     # call output_.inputReceived(this, batch);
> }
>
> My question is, how do I handle the error in WriteRecordBatch properly with
> ExecNode API?
>
> Thanks,
> Li

Reply via email to