Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/20754 )
Change subject: IMPALA-12595: Allow automatic removal of old logs from previous PID ...................................................................... IMPALA-12595: Allow automatic removal of old logs from previous PID IMPALA-11184 add code to target specific PID for log rotation. This align with glog behavior and grant safety. That is, it is strictly limit log rotation to only consider log files made by the currently running Impalad and exclude logs made by previous PID or other living-colocated Impalads. The downside of this limit is that logs can start accumulate in a node when impalad is frequently restarted and is only resolvable by admin doing manual log removal. To help avoid this manual removal, this patch adds a backend flag 'log_rotation_match_pid' that relax the limit by dropping the PID in glob pattern. Default value for this new flag is False. However, for testing purpose, start-impala-cluster.py will override it to True since test minicluster logs to a common log directory. Setting 'log_rotation_match_pid' to True will prevent one impalad from interfering with log rotation of other impalad in minicluster. As a minimum exercise for this new log rotation behavior, test_breakpad.py::TestLogging is modified to invoke start-impala-cluster.py with 'log_rotation_match_pid' set to False. Testing: - Add test_excessive_cerr_ignore_pid and test_excessive_cerr_match_pid. - Split TestLogging into two. One run test_excessive_cerr_ignore_pid in core exploration, while the other run the rest of logging tests in exhaustive exploration. - Pass exhaustive tests. Change-Id: I599799e73f27f941a1d7f3dec0f40b4f05ea5ceb Reviewed-on: http://gerrit.cloudera.org:8080/20754 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/common/global-flags.cc M be/src/common/logging.cc M bin/start-impala-cluster.py M tests/common/custom_cluster_test_suite.py M tests/custom_cluster/test_breakpad.py 5 files changed, 112 insertions(+), 23 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/20754 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I599799e73f27f941a1d7f3dec0f40b4f05ea5ceb Gerrit-Change-Number: 20754 Gerrit-PatchSet: 9 Gerrit-Owner: Riza Suminto <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Kurt Deschler <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Wenzhe Zhou <[email protected]>
