Yida Wu has uploaded this change for review. ( http://gerrit.cloudera.org:8080/22549
Change subject: IMPALA-13798: Clear of host-level remote scratch dir on shutdown ...................................................................... IMPALA-13798: Clear of host-level remote scratch dir on shutdown IMPALA-13677 introduces cleanup remote scratch files on startup, and also introduces a host-level directory to do easier cleanup. An empty host-level directory in remote storage does not use resources, but it can stay there forever if a host goes offline permanently. To improve this, this patch removed the remote host-level temporary directory on shutdown. This helps prevent too many empty directories left in the remote scratch path, especially for hosts that never restart. Changed the flag remote_scratch_cleanup_on_startup to remote_scratch_cleanup_on_startstop, so that this flag also controls to cleanup on shutdown as well. Tests: Added an e2e testcase test_scratch_dirs_remote_dir_removal_on_shutdown. Change-Id: Ic8f446894afdf975630aef80a9d964a9a78d3b46 --- M be/src/runtime/tmp-file-mgr-test.cc M be/src/runtime/tmp-file-mgr.cc M be/src/runtime/tmp-file-mgr.h M be/src/service/impala-server.cc M tests/custom_cluster/test_scratch_disk.py 5 files changed, 57 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/49/22549/1 -- To view, visit http://gerrit.cloudera.org:8080/22549 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ic8f446894afdf975630aef80a9d964a9a78d3b46 Gerrit-Change-Number: 22549 Gerrit-PatchSet: 1 Gerrit-Owner: Yida Wu <[email protected]>
