[ https://issues.apache.org/jira/browse/HIVE-22275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16942500#comment-16942500 ]
Hive QA commented on HIVE-22275: -------------------------------- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m 35s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 24s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 12s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 41s{color} | {color:blue} service in master has 49 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 25s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 23s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 23s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 12s{color} | {color:red} service: The patch generated 3 new + 25 unchanged - 0 fixed = 28 total (was 25) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 47s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 17s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 14s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 13m 55s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-18822/dev-support/hive-personality.sh | | git revision | master / 5717cde | | Default Java | 1.8.0_111 | | findbugs | v3.0.1 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-18822/yetus/diff-checkstyle-service.txt | | modules | C: service U: service | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-18822/yetus.txt | | Powered by | Apache Yetus http://yetus.apache.org | This message was automatically generated. > OperationManager.queryIdOperation does not properly clean up multiple queryIds > ------------------------------------------------------------------------------ > > Key: HIVE-22275 > URL: https://issues.apache.org/jira/browse/HIVE-22275 > Project: Hive > Issue Type: Bug > Components: HiveServer2 > Reporter: Jason Dere > Assignee: Jason Dere > Priority: Major > Attachments: HIVE-22275.1.patch, HIVE-22275.2.patch > > > In the case that multiple statements are run by a single Session before being > cleaned up, it appears that OperationManager.queryIdOperation is not cleaned > up properly. > See the log statements below - with the exception of the first "Removed > queryId:" log line, the queryId listed during cleanup is the same, when each > of these handles should have their own queryId. Looks like only the last > queryId executed is being cleaned up. > As a result, HS2 can run out of memory as OperationManager.queryIdOperation > grows and never cleans these queryIds/Operations up. > {noformat} > 2019-09-13T08:37:36,785 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a > HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) - > Adding operation: OperationHandle [opType=EXECUTE_STATEMENT, > getHandleIdentifier()=dfed4c18-a284-4640-9f4a-1a20527105f9] > 2019-09-13T08:37:38,432 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a > HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) - > Removed queryId: hive_20190913083736_c49cf3cc-cfe8-48a1-bd22-8b924dfb0396 > corresponding to operation: OperationHandle [opType=EXECUTE_STATEMENT, > getHandleIdentifier()=dfed4c18-a284-4640-9f4a-1a20527105f9] with tag: null > 2019-09-13T08:37:38,469 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a > HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) - > Adding operation: OperationHandle [opType=EXECUTE_STATEMENT, > getHandleIdentifier()=24d0030c-0e49-45fb-a918-2276f0941cfb] > 2019-09-13T08:37:52,662 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a > HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) - > Adding operation: OperationHandle [opType=EXECUTE_STATEMENT, > getHandleIdentifier()=b983802c-1dec-4fa0-8680-d05ab555321b] > 2019-09-13T08:37:56,239 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a > HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) - > Adding operation: OperationHandle [opType=EXECUTE_STATEMENT, > getHandleIdentifier()=75dbc531-2964-47b2-84d7-85b59f88999c] > 2019-09-13T08:38:02,551 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a > HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) - > Adding operation: OperationHandle [opType=EXECUTE_STATEMENT, > getHandleIdentifier()=72c79076-9d67-4894-a526-c233fa5450b2] > 2019-09-13T08:38:10,558 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a > HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) - > Adding operation: OperationHandle [opType=EXECUTE_STATEMENT, > getHandleIdentifier()=17b30a62-612d-4b70-9ba7-4287d2d9229b] > 2019-09-13T08:38:16,930 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a > HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) - > Adding operation: OperationHandle [opType=EXECUTE_STATEMENT, > getHandleIdentifier()=ea97e99d-cc77-470b-b49a-b869c73a4615] > 2019-09-13T08:38:20,440 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a > HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) - > Adding operation: OperationHandle [opType=EXECUTE_STATEMENT, > getHandleIdentifier()=a277b789-ebb8-4925-878f-6728d3e8c5fb] > 2019-09-13T08:38:26,303 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a > HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) - > Adding operation: OperationHandle [opType=EXECUTE_STATEMENT, > getHandleIdentifier()=9a023ab8-aa80-45db-af88-94790cc83033] > 2019-09-13T08:38:30,791 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a > HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) - > Adding operation: OperationHandle [opType=EXECUTE_STATEMENT, > getHandleIdentifier()=b697c801-7da0-4544-bcfa-442eb1d3bd77] > 2019-09-13T08:39:10,187 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a > HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) - > Adding operation: OperationHandle [opType=EXECUTE_STATEMENT, > getHandleIdentifier()=bda93c8f-0822-4592-a61c-4701720a1a5c] > 2019-09-13T08:39:15,471 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a > HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) - > Removed queryId: hive_20190913083910_c4809ca8-d8db-423c-8b6d-fbe3eee89971 > corresponding to operation: OperationHandle [opType=EXECUTE_STATEMENT, > getHandleIdentifier()=24d0030c-0e49-45fb-a918-2276f0941cfb] with tag: null > 2019-09-13T08:39:15,507 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a > HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) - > Removed queryId: hive_20190913083910_c4809ca8-d8db-423c-8b6d-fbe3eee89971 > corresponding to operation: OperationHandle [opType=EXECUTE_STATEMENT, > getHandleIdentifier()=b983802c-1dec-4fa0-8680-d05ab555321b] with tag: null > 2019-09-13T08:39:15,538 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a > HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) - > Removed queryId: hive_20190913083910_c4809ca8-d8db-423c-8b6d-fbe3eee89971 > corresponding to operation: OperationHandle [opType=EXECUTE_STATEMENT, > getHandleIdentifier()=75dbc531-2964-47b2-84d7-85b59f88999c] with tag: null > 2019-09-13T08:39:15,572 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a > HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) - > Removed queryId: hive_20190913083910_c4809ca8-d8db-423c-8b6d-fbe3eee89971 > corresponding to operation: OperationHandle [opType=EXECUTE_STATEMENT, > getHandleIdentifier()=72c79076-9d67-4894-a526-c233fa5450b2] with tag: null > 2019-09-13T08:39:15,600 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a > HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) - > Removed queryId: hive_20190913083910_c4809ca8-d8db-423c-8b6d-fbe3eee89971 > corresponding to operation: OperationHandle [opType=EXECUTE_STATEMENT, > getHandleIdentifier()=17b30a62-612d-4b70-9ba7-4287d2d9229b] with tag: null > 2019-09-13T08:39:15,629 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a > HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) - > Removed queryId: hive_20190913083910_c4809ca8-d8db-423c-8b6d-fbe3eee89971 > corresponding to operation: OperationHandle [opType=EXECUTE_STATEMENT, > getHandleIdentifier()=ea97e99d-cc77-470b-b49a-b869c73a4615] with tag: null > 2019-09-13T08:39:15,657 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a > HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) - > Removed queryId: hive_20190913083910_c4809ca8-d8db-423c-8b6d-fbe3eee89971 > corresponding to operation: OperationHandle [opType=EXECUTE_STATEMENT, > getHandleIdentifier()=a277b789-ebb8-4925-878f-6728d3e8c5fb] with tag: null > 2019-09-13T08:39:15,692 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a > HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) - > Removed queryId: hive_20190913083910_c4809ca8-d8db-423c-8b6d-fbe3eee89971 > corresponding to operation: OperationHandle [opType=EXECUTE_STATEMENT, > getHandleIdentifier()=9a023ab8-aa80-45db-af88-94790cc83033] with tag: null > 2019-09-13T08:39:15,723 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a > HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) - > Removed queryId: hive_20190913083910_c4809ca8-d8db-423c-8b6d-fbe3eee89971 > corresponding to operation: OperationHandle [opType=EXECUTE_STATEMENT, > getHandleIdentifier()=b697c801-7da0-4544-bcfa-442eb1d3bd77] with tag: null > 2019-09-13T08:39:15,754 INFO [8eaa1601-f045-4ad5-9c2e-1e5944b75f6a > HiveServer2-Handler-Pool: Thread-202]: operation.OperationManager (:()) - > Removed queryId: hive_20190913083910_c4809ca8-d8db-423c-8b6d-fbe3eee89971 > corresponding to operation: OperationHandle [opType=EXECUTE_STATEMENT, > getHandleIdentifier()=bda93c8f-0822-4592-a61c-4701720a1a5c] with tag: null > {noformat} -- This message was sent by Atlassian Jira (v8.3.4#803005)