Re: [C++] Clarifying the behavior of source node and executor

2022-07-25 Thread Li Jin
27;ll > look into adding this sequential-option to source-node and report back. > > > Yaron. > > From: Li Jin > Sent: Monday, July 25, 2022 11:39 AM > To: dev@arrow.apache.org > Subject: Re: [C++] Clarifying the behavior of source node and ex

Re: [C++] Clarifying the behavior of source node and executor

2022-07-25 Thread Yaron Gvili
AM To: dev@arrow.apache.org Subject: Re: [C++] Clarifying the behavior of source node and executor Now I think about it more. Weston has probably answered this in another mailing thread that this is not guaranteed and the observation of batches becoming out of file reader + source node happened by chance.

Re: [C++] Clarifying the behavior of source node and executor

2022-07-25 Thread Li Jin
n how to > obtain such a guarantee. > > > Yaron. > > From: Li Jin > Sent: Monday, July 25, 2022 11:10 AM > To: dev@arrow.apache.org > Subject: Re: [C++] Clarifying the behavior of source node and executor > > Sorry the link to the gen

Re: [C++] Clarifying the behavior of source node and executor

2022-07-25 Thread Yaron Gvili
__ From: Li Jin Sent: Monday, July 25, 2022 11:10 AM To: dev@arrow.apache.org Subject: Re: [C++] Clarifying the behavior of source node and executor Sorry the link to the generator above is wrong - We traced into the code and found it uses BackgroundGenerator: https://github.com/ap

Re: [C++] Clarifying the behavior of source node and executor

2022-07-25 Thread Li Jin
Sorry the link to the generator above is wrong - We traced into the code and found it uses BackgroundGenerator: https://github.com/apache/arrow/blob/78fb2edd30b602bd54702896fa78d36ec6fefc8c/cpp/src/arrow/util/async_generator.h#L1581 On Mon, Jul 25, 2022 at 11:07 AM Li Jin wrote: > Hi, > > Ivan a