Zoltan Borok-Nagy has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/22141 )
Change subject: IMPALA-13589: SELECT INPUT__FILE__NAME can crash Impala ...................................................................... IMPALA-13589: SELECT INPUT__FILE__NAME can crash Impala If the user only queries virtual column INPUT__FILE__NAME from a table backed by text files, and the last row doesn't end with the row delimiter (e.g. '\n') then Impala crashes. In HdfsTextScanner::FinishScanRange() there is specific code to deal with the last row if it doesn't end with the row delimiter, and we fill the last tuple here. This code wasn't active when we only read INPUT__FILE__NAME, which means the last tuple contained garbage which caused a segfault later. The fix is to always fill the last tuple if we have a template tuple as it means we either have partition expressions, or file-level virtual columns like INPUT__FILE__NAME. Other file-level virtual columns only apply to Iceberg tables which don't support text data files, so those are not affected by this bug. Testing * added e2e tests Change-Id: I0ea8e7fed77cbc9ae90a858eafeee9dcfd73d143 Reviewed-on: http://gerrit.cloudera.org:8080/22141 Tested-by: Impala Public Jenkins <[email protected]> Reviewed-by: Gabor Kaszab <[email protected]> --- M be/src/exec/text/hdfs-text-scanner.cc M testdata/workloads/functional-query/queries/QueryTest/virtual-column-input-file-name-in-table.test M testdata/workloads/functional-query/queries/QueryTest/virtual-column-input-file-name.test 3 files changed, 46 insertions(+), 1 deletion(-) Approvals: Impala Public Jenkins: Verified Gabor Kaszab: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/22141 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I0ea8e7fed77cbc9ae90a858eafeee9dcfd73d143 Gerrit-Change-Number: 22141 Gerrit-PatchSet: 3 Gerrit-Owner: Zoltan Borok-Nagy <[email protected]> Gerrit-Reviewer: Gabor Kaszab <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
