Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/22662 )
Change subject: IMPALA-3841: Enable late materialization for collections ...................................................................... Patch Set 7: (2 comments) http://gerrit.cloudera.org:8080/#/c/22662/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/22662/2//COMMIT_MSG@13 PS2, Line 13: For a collection column, late materialization takes effect only when the : collection > Agree, no need to do it here. This can add some miner perf regression, right? I mean that in case there are many rows where the collection column is empty without the !empty() predicate the scanner won't drop the row and pass it to the next node, and only unnest node will drop it. I think that this should be noticeable only in very extreme cases but it could be noted in the commit message. An idea on how to avoid this in some cases is to keep the !empty() pred in case there is not other top level predicate (not sure how hard it is to add this in the planner). http://gerrit.cloudera.org:8080/#/c/22662/7/be/src/exec/parquet/hdfs-parquet-scanner.cc File be/src/exec/parquet/hdfs-parquet-scanner.cc: http://gerrit.cloudera.org:8080/#/c/22662/7/be/src/exec/parquet/hdfs-parquet-scanner.cc@2385 PS7, Line 2385: late_materialization_threshold_ I think that this is not the right place to calculate this, as this will be called many times - place called only during initialization would be better. -- To view, visit http://gerrit.cloudera.org:8080/22662 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia21bdfa6811408d66d74367e0a9520e20951105f Gerrit-Change-Number: 22662 Gerrit-PatchSet: 7 Gerrit-Owner: Xuebin Su <x...@cloudera.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Xuebin Su <x...@cloudera.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Comment-Date: Wed, 30 Apr 2025 14:42:55 +0000 Gerrit-HasComments: Yes
