Joe McDonnell has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/23796 )
Change subject: IMPALA-14634: Organize custom cluster logs into subdirectories by test name ...................................................................... IMPALA-14634: Organize custom cluster logs into subdirectories by test name This changes the logic for setting the log directory for custom cluster tests. It uses the test class name and (optionally) the test method name to put log files into subdirectories. For a test that restarts the cluster for each test method, this would have a directory structure of base_dir / test class / test method name / For a test that starts the cluster at the class level and reuses it across tests methods, this would be: base_dir / test class / This directory structure can still have logs from multiple tests going to a single directory. For example, some tests are parameterized so that a test method runs multiple times with different parameter values. Those will share a directory. This should be rare and most directories will have logs from a single test. Some test methods restart the cluster within the test itself rather than just at the beginning. This stores the test method so these restarts continue to use the same directory. The code is often in @classmethod functions, so this stores the test method name at the class level and manipulates it through @classmethod functions. This is awkward, but it works because the custom cluster tests are single threaded. This also adjusts the logic for determining the base directory for custom cluster logs. tests/run-custom-cluster-tests.sh sets LOG_DIR to $IMPALA_CUSTOM_CLUSTER_TEST_LOGS_DIR (i.e. logs/custom_cluster_tests) and this continues to respect LOG_DIR. However, in the dev environment using impala-py.test, LOG_DIR is not set and the code currently defaults to /tmp. This changes it to use $IMPALA_CUSTOM_CLUSTER_TEST_LOGS_DIR so that logs go to logs/custom_cluster_tests rather than /tmp. Testing: - Ran locally and verified the output goes into directories - Ran exhaustive custom cluster tests Change-Id: I8a8402fed1584a99f91451a3976e7026d0deb834 Reviewed-on: http://gerrit.cloudera.org:8080/23796 Reviewed-by: Michael Smith <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> Reviewed-by: Quanlong Huang <[email protected]> --- M tests/common/custom_cluster_test_suite.py M tests/custom_cluster/test_breakpad.py 2 files changed, 52 insertions(+), 4 deletions(-) Approvals: Michael Smith: Looks good to me, but someone else must approve Impala Public Jenkins: Verified Quanlong Huang: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/23796 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I8a8402fed1584a99f91451a3976e7026d0deb834 Gerrit-Change-Number: 23796 Gerrit-PatchSet: 6 Gerrit-Owner: Joe McDonnell <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]>
