Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/21680 )
Change subject: IMPALA-13303: FileSystemUtil.listFiles() should handle non-recursive case ...................................................................... IMPALA-13303: FileSystemUtil.listFiles() should handle non-recursive case FileSystemUtil.listFiles() is used in FileMetadataLoader#loadInternal() to list the files with block locations. When table property "impala.disable.recursive.listing" is set to true, it's supposed to skip files in the sub dirs. However, for FileSystems that don't support recursive listFiles(), we always create a RecursingIterator and don't respect the 'recursive' argument. This patch fixes the issue by adding the check for the 'recursive' argument and use the non-recursive iterator when it's false. Tests - Add test in test_recursive_listing.py to reveal the issue Change-Id: Ia930e6071963d53561ce79896bff9d19720468a4 Reviewed-on: http://gerrit.cloudera.org:8080/21680 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java M tests/metadata/test_recursive_listing.py 2 files changed, 8 insertions(+), 1 deletion(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/21680 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ia930e6071963d53561ce79896bff9d19720468a4 Gerrit-Change-Number: 21680 Gerrit-PatchSet: 4 Gerrit-Owner: Quanlong Huang <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Peter Rozsa <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
