Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/21071 )
Change subject: IMPALA-12827: Fix failures in processing AbortTxnEvent due to aborted write id is cleaned up ...................................................................... IMPALA-12827: Fix failures in processing AbortTxnEvent due to aborted write id is cleaned up HdfsTable tracks the ValidWriteIdList from HMS. When the table is reloaded, the ValidWriteIdList is updated to the latest state. An ABORT_TXN event that is lagging behind could match to aborted write ids that have already been cleaned up by the HMS housekeeping thread. Such write ids can't be found in the cached ValidWriteIdList as opened or aborted write ids. This hits a Precondition check and fails the event processing. This patch fixes the check to allow this case. Also adds more logs for dealing with write ids. Tests - Add custom-cluster test to start Hive with the housekeeping thread turned on and verified that such ABORT_TXN event is processed correctly. Change-Id: I93b6f684d6e4b94961d804a0c022029249873681 Reviewed-on: http://gerrit.cloudera.org:8080/21071 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M bin/create-test-configuration.sh M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/hive/common/MutableValidReaderWriteIdList.java M fe/src/test/resources/hive-site.xml.py M tests/common/impala_test_suite.py M tests/custom_cluster/test_events_custom_configs.py 8 files changed, 84 insertions(+), 11 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/21071 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I93b6f684d6e4b94961d804a0c022029249873681 Gerrit-Change-Number: 21071 Gerrit-PatchSet: 6 Gerrit-Owner: Quanlong Huang <[email protected]> Gerrit-Reviewer: Anonymous Coward <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Kurt Deschler <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]>
