Re: [C++] MakeReaderGenerator Behavior using GetCPUThreadPool

2022-07-25 Thread Weston Pace
1) Yes, that sounds correct. The file readers will read from files in parallel (even if there is one file it can read from row groups in parallel). There is no guarantee these reads will finish sequentially. 2) Hmm, this one will work for now, because the executor==nullptr behavior is to borrow

[C++] MakeReaderGenerator Behavior using GetCPUThreadPool

2022-07-25 Thread Ivan Chau
Hey all, While investigating the in-order behavior of the SourceNode, we found some interesting observations: 1) The ExecContext should use nullptr for its executor to guarantee any sequential behavior (as discussed previously). We found cases where our File BatchReader was reading out of order w