Got it! Thank you both! On Thu, Jun 3, 2021 at 17:46 Ryan Blue <b...@apache.org> wrote:
> Jack is right. We evaluate the residual for each task lazily, so it > happens on the node that is reading data for the task. The residual is the > part of the original filter expression that needs to be run in the task to > produce the correct filtered rows. Since that is specific to the task, we > don't want to build it during job planning. > > On Thu, Jun 3, 2021 at 2:39 PM Jack Ye <yezhao...@gmail.com> wrote: > >> Not sure if it is the only case, but the top thing that comes to my mind >> is that: BaseFileScanTask has ResidualEvaluator which has Expression. >> Because ScanTask is serializable and passed around workers, expressions >> have to be serializable. >> -Jack >> >> On Thu, Jun 3, 2021 at 2:33 PM QH Yan <mailto...@gmail.com> wrote: >> >>> Curious about the use-case where Expression get serialized and passed >>> around. >>> >>> Thank you! >>> Qinhua >>> -- >>> *Qinhua* >>> >>> > > -- > Ryan Blue > -- *Qinhua*
