[
https://issues.apache.org/jira/browse/HIVE-22275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16941646#comment-16941646
]
Hive QA commented on HIVE-22275:
--------------------------------
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981839/HIVE-22275.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17016 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.llap.cache.TestBuddyAllocator.testMTT[2] (batchId=363)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18806/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18806/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18806/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981839 - PreCommit-HIVE-Build
> OperationManager.queryIdOperation does not properly clean up multiple queryIds
> ------------------------------------------------------------------------------
>
> Key: HIVE-22275
> URL: https://issues.apache.org/jira/browse/HIVE-22275
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
> Reporter: Jason Dere
> Assignee: Jason Dere
> Priority: Major
> Attachments: HIVE-22275.1.patch
>
>
> In the case that multiple statements are run by a single Session before being
> cleaned up, it appears that OperationManager.queryIdOperation is not cleaned
> up properly.
> See the log statements below - with the exception of the first "Removed
> queryId:" log line, the queryId listed during cleanup is the same, when each
> of these handles should have their own queryId. Looks like only the last
> queryId executed is being cleaned up.
> As a result, HS2 can run out of memory as OperationManager.queryIdOperation
> grows and never cleans these queryIds/Operations up.
> {noformat}
> 2019-09-13T08:37:36,785 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a
> HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) -
> Adding operation: OperationHandle [opType=EXECUTE_STATEMENT,
> getHandleIdentifier()=dfed4c18-a284-4640-9f4a-1a20527105f9]
> 2019-09-13T08:37:38,432 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a
> HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) -
> Removed queryId: hive_20190913083736_c49cf3cc-cfe8-48a1-bd22-8b924dfb0396
> corresponding to operation: OperationHandle [opType=EXECUTE_STATEMENT,
> getHandleIdentifier()=dfed4c18-a284-4640-9f4a-1a20527105f9] with tag: null
> 2019-09-13T08:37:38,469 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a
> HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) -
> Adding operation: OperationHandle [opType=EXECUTE_STATEMENT,
> getHandleIdentifier()=24d0030c-0e49-45fb-a918-2276f0941cfb]
> 2019-09-13T08:37:52,662 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a
> HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) -
> Adding operation: OperationHandle [opType=EXECUTE_STATEMENT,
> getHandleIdentifier()=b983802c-1dec-4fa0-8680-d05ab555321b]
> 2019-09-13T08:37:56,239 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a
> HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) -
> Adding operation: OperationHandle [opType=EXECUTE_STATEMENT,
> getHandleIdentifier()=75dbc531-2964-47b2-84d7-85b59f88999c]
> 2019-09-13T08:38:02,551 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a
> HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) -
> Adding operation: OperationHandle [opType=EXECUTE_STATEMENT,
> getHandleIdentifier()=72c79076-9d67-4894-a526-c233fa5450b2]
> 2019-09-13T08:38:10,558 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a
> HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) -
> Adding operation: OperationHandle [opType=EXECUTE_STATEMENT,
> getHandleIdentifier()=17b30a62-612d-4b70-9ba7-4287d2d9229b]
> 2019-09-13T08:38:16,930 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a
> HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) -
> Adding operation: OperationHandle [opType=EXECUTE_STATEMENT,
> getHandleIdentifier()=ea97e99d-cc77-470b-b49a-b869c73a4615]
> 2019-09-13T08:38:20,440 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a
> HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) -
> Adding operation: OperationHandle [opType=EXECUTE_STATEMENT,
> getHandleIdentifier()=a277b789-ebb8-4925-878f-6728d3e8c5fb]
> 2019-09-13T08:38:26,303 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a
> HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) -
> Adding operation: OperationHandle [opType=EXECUTE_STATEMENT,
> getHandleIdentifier()=9a023ab8-aa80-45db-af88-94790cc83033]
> 2019-09-13T08:38:30,791 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a
> HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) -
> Adding operation: OperationHandle [opType=EXECUTE_STATEMENT,
> getHandleIdentifier()=b697c801-7da0-4544-bcfa-442eb1d3bd77]
> 2019-09-13T08:39:10,187 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a
> HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) -
> Adding operation: OperationHandle [opType=EXECUTE_STATEMENT,
> getHandleIdentifier()=bda93c8f-0822-4592-a61c-4701720a1a5c]
> 2019-09-13T08:39:15,471 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a
> HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) -
> Removed queryId: hive_20190913083910_c4809ca8-d8db-423c-8b6d-fbe3eee89971
> corresponding to operation: OperationHandle [opType=EXECUTE_STATEMENT,
> getHandleIdentifier()=24d0030c-0e49-45fb-a918-2276f0941cfb] with tag: null
> 2019-09-13T08:39:15,507 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a
> HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) -
> Removed queryId: hive_20190913083910_c4809ca8-d8db-423c-8b6d-fbe3eee89971
> corresponding to operation: OperationHandle [opType=EXECUTE_STATEMENT,
> getHandleIdentifier()=b983802c-1dec-4fa0-8680-d05ab555321b] with tag: null
> 2019-09-13T08:39:15,538 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a
> HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) -
> Removed queryId: hive_20190913083910_c4809ca8-d8db-423c-8b6d-fbe3eee89971
> corresponding to operation: OperationHandle [opType=EXECUTE_STATEMENT,
> getHandleIdentifier()=75dbc531-2964-47b2-84d7-85b59f88999c] with tag: null
> 2019-09-13T08:39:15,572 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a
> HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) -
> Removed queryId: hive_20190913083910_c4809ca8-d8db-423c-8b6d-fbe3eee89971
> corresponding to operation: OperationHandle [opType=EXECUTE_STATEMENT,
> getHandleIdentifier()=72c79076-9d67-4894-a526-c233fa5450b2] with tag: null
> 2019-09-13T08:39:15,600 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a
> HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) -
> Removed queryId: hive_20190913083910_c4809ca8-d8db-423c-8b6d-fbe3eee89971
> corresponding to operation: OperationHandle [opType=EXECUTE_STATEMENT,
> getHandleIdentifier()=17b30a62-612d-4b70-9ba7-4287d2d9229b] with tag: null
> 2019-09-13T08:39:15,629 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a
> HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) -
> Removed queryId: hive_20190913083910_c4809ca8-d8db-423c-8b6d-fbe3eee89971
> corresponding to operation: OperationHandle [opType=EXECUTE_STATEMENT,
> getHandleIdentifier()=ea97e99d-cc77-470b-b49a-b869c73a4615] with tag: null
> 2019-09-13T08:39:15,657 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a
> HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) -
> Removed queryId: hive_20190913083910_c4809ca8-d8db-423c-8b6d-fbe3eee89971
> corresponding to operation: OperationHandle [opType=EXECUTE_STATEMENT,
> getHandleIdentifier()=a277b789-ebb8-4925-878f-6728d3e8c5fb] with tag: null
> 2019-09-13T08:39:15,692 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a
> HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) -
> Removed queryId: hive_20190913083910_c4809ca8-d8db-423c-8b6d-fbe3eee89971
> corresponding to operation: OperationHandle [opType=EXECUTE_STATEMENT,
> getHandleIdentifier()=9a023ab8-aa80-45db-af88-94790cc83033] with tag: null
> 2019-09-13T08:39:15,723 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a
> HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) -
> Removed queryId: hive_20190913083910_c4809ca8-d8db-423c-8b6d-fbe3eee89971
> corresponding to operation: OperationHandle [opType=EXECUTE_STATEMENT,
> getHandleIdentifier()=b697c801-7da0-4544-bcfa-442eb1d3bd77] with tag: null
> 2019-09-13T08:39:15,754 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a
> HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) -
> Removed queryId: hive_20190913083910_c4809ca8-d8db-423c-8b6d-fbe3eee89971
> corresponding to operation: OperationHandle [opType=EXECUTE_STATEMENT,
> getHandleIdentifier()=bda93c8f-0822-4592-a61c-4701720a1a5c] with tag: null
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
