[
https://issues.apache.org/jira/browse/HIVE-2609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165909#comment-13165909
]
Ashutosh Chauhan commented on HIVE-2609:
----------------------------------------
@Mithun,
Good finding. We can definitely upgrade the jdo libs. If all our tests pass,
then certainly it makes sense to do so, given that there is atleast one bug
(this jira) which will be fixed by it. There is some work for it already going
on at HIVE-2084 for this. But, looks like Carl hit failures there. Also, if we
are upgrading we should try to go to 3.0.1 instead of 2.x series.
> NPE when pruning partitions by thrift method get_partitions_by_filter
> ---------------------------------------------------------------------
>
> Key: HIVE-2609
> URL: https://issues.apache.org/jira/browse/HIVE-2609
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.1
> Reporter: Min Zhou
>
> It's a datanucleus bug indeed.
> try this code:
> {code}
> boolean open = false;
> for (int i = 0; i < 5 && !open; ++i) {
> try {
> transport.open();
> open = true;
> } catch (TTransportException e) {
> System.out.println("failed to connect to MetaStore, re-trying...");
> try {
> Thread.sleep(1000);
> } catch (InterruptedException ignore) {}
> }
> }
> try {
> List<Partition> parts =
> client.get_partitions_by_filter("default", "partitioned_nation",
> "pt < '2'", (short) -1);
> for (Partition part : parts) {
> System.out.println(part.getSd().getLocation());
> }
> } catch (Exception te) {
> te.printStackTrace();
> }
> {code}
> A NPEexception would be thrown on the thrift server side
> {noformat}
> 11/11/25 13:11:55 ERROR api.ThriftHiveMetastore$Processor: Internal error
> processing get_partitions_by_filter
> java.lang.NullPointerException
> at
> org.datanucleus.store.mapped.mapping.MappingHelper.getMappingIndices(MappingHelper.java:35)
> at
> org.datanucleus.store.mapped.expression.StatementText.applyParametersToStatement(StatementText.java:194)
> at
> org.datanucleus.store.rdbms.query.RDBMSQueryUtils.getPreparedStatementForQuery(RDBMSQueryUtils.java:233)
> at
> org.datanucleus.store.rdbms.query.legacy.SQLEvaluator.evaluate(SQLEvaluator.java:115)
> at
> org.datanucleus.store.rdbms.query.legacy.JDOQLQuery.performExecute(JDOQLQuery.java:288)
> at org.datanucleus.store.query.Query.executeQuery(Query.java:1657)
> at
> org.datanucleus.store.rdbms.query.legacy.JDOQLQuery.executeQuery(JDOQLQuery.java:245)
> at org.datanucleus.store.query.Query.executeWithMap(Query.java:1526)
> at org.datanucleus.jdo.JDOQuery.executeWithMap(JDOQuery.java:334)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.listMPartitionsByFilter(ObjectStore.java:1329)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getPartitionsByFilter(ObjectStore.java:1241)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$40.run(HiveMetaStore.java:2369)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$40.run(HiveMetaStore.java:2366)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.executeWithRetry(HiveMetaStore.java:307)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_partitions_by_filter(HiveMetaStore.java:2366)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_partitions_by_filter.process(ThriftHiveMetastore.j
> ava:6099)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor.process(ThriftHiveMetastore.java:4789)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$TLoggingProcessor.process(HiveMetaStore.java:3167)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:253)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> {noformat}
> A null JavaTypeMapping was passed into
> org.datanucleus.store.mapped.mapping.MappingHelper.(int initialPosition,
> JavaTypeMapping mapping), that caused NPE.
> After digged into the datanucleus source, I found that the null value was
> born in the constructor of
> org.datanucleus.store.mapped.expression.SubstringExpression. see
> {code}
> /**
> * Constructs the substring
> * @param str the String Expression
> * @param begin The start position
> * @param end The end position expression
> **/
> public SubstringExpression(StringExpression str, NumericExpression begin,
> NumericExpression end)
> {
> super(str.getQueryExpression());
> st.append("SUBSTRING(").append(str).append(" FROM ")
> .append(begin.add(new IntegerLiteral(qs, mapping,
> BigInteger.ONE)))
> .append(" FOR ").append(end.sub(begin)).append(')');
> }
> {code}
> The field mapping hasn't been instanced at that moment.
> How do you deal with such a external bug?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
