[
https://issues.apache.org/jira/browse/HIVE-23633?focusedWorklogId=646821&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-646821
]
ASF GitHub Bot logged work on HIVE-23633:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 06/Sep/21 09:41
Start Date: 06/Sep/21 09:41
Worklog Time Spent: 10m
Work Description: dengzhhu653 commented on a change in pull request #2344:
URL: https://github.com/apache/hive/pull/2344#discussion_r702712424
##########
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java
##########
@@ -2107,15 +2115,17 @@ private ColumnStatisticsObj
prepareCSObjWithAdjustedNDV(Object[] row, int i,
makeParams(inputColNames.size()),
makeParams(inputPartNames.size()));
long start = doTrace ? System.nanoTime() : 0;
Query query = pm.newQuery("javax.jdo.query.SQL", queryText);
- Object qResult = executeWithArray(query, prepareParams(
- catName, dbName, tableName, inputPartNames, inputColNames,
engine), queryText);
- MetastoreDirectSqlUtils.timingTrace(doTrace, queryText0, start,
(doTrace ? System.nanoTime() : 0));
- if (qResult == null) {
- query.closeAll();
- return Collections.emptyList();
+ try {
Review comment:
Maybe we should need this, `Query#execute` may return a lazy result,
e.g, the `org.datanucleus.store.rdbms.query.ForwardQueryResult`, closing the
query ealier may result to the exception `Query result has been closed` when
iterating/accessing the result later on.
For example:
MetaException(message:Query result has been closed)
at
org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.handleDirectSqlError(ObjectStore.java:4298)
at
org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:4255)
at
org.apache.hadoop.hive.metastore.ObjectStore.getPartitionColumnStatisticsInternal(ObjectStore.java:10079)
at
org.apache.hadoop.hive.metastore.ObjectStore.getPartitionColumnStatistics(ObjectStore.java:9995)
at
org.apache.hadoop.hive.metastore.cache.TestCachedStore.testCacheUpdatePartitionColStats(TestCachedStore.java:365)
on `list.size()`
https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java#L2135
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 646821)
Time Spent: 11h 20m (was: 11h 10m)
> Metastore some JDO query objects do not close properly
> ------------------------------------------------------
>
> Key: HIVE-23633
> URL: https://issues.apache.org/jira/browse/HIVE-23633
> Project: Hive
> Issue Type: Bug
> Reporter: Zhihua Deng
> Assignee: Zhihua Deng
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-23633.01.patch
>
> Time Spent: 11h 20m
> Remaining Estimate: 0h
>
> After patched [HIVE-10895|https://issues.apache.org/jira/browse/HIVE-10895],
> The metastore still has seen a memory leak on db resources: many
> StatementImpls left unclosed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
