[
https://issues.apache.org/jira/browse/HIVE-3620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13627148#comment-13627148
]
Arup Malakar commented on HIVE-3620:
------------------------------------
Error log I see in the server is:
{code}
2013-04-09 19:47:41,955 ERROR thrift.ProcessFunction
(ProcessFunction.java:process(41)) - Internal error processing
get_databasejava.lang.OutOfMemoryError: Java heap space at
java.lang.AbstractStringBuilder.<init>(AbstractStringBuilder.java:45) at
java.lang.StringBuilder.<init>(StringBuilder.java:80) at
oracle.net.ns.Packet.<init>(Packet.java:513) at
oracle.net.ns.Packet.<init>(Packet.java:142)
at oracle.net.ns.NSProtocol.connect(NSProtocol.java:279)
at oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:1042)
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:301)
at
oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:531)
at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:221) at
oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:503)
at
yjava.database.jdbc.oracle.KeyDbDriverWrapper.connect(KeyDbDriverWrapper.java:81)
at java.sql.DriverManager.getConnection(DriverManager.java:582)
at java.sql.DriverManager.getConnection(DriverManager.java:185) at
org.apache.commons.dbcp.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:75)
at
org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582)
at
org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1148)
at
org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106)
at
org.datanucleus.store.rdbms.ConnectionProviderPriorityList.getConnection(ConnectionProviderPriorityList.java:57)
at
org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:363)
at
org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getXAResource(ConnectionFactoryImpl.java:322)
at
org.datanucleus.store.connection.ConnectionManagerImpl.enlistResource(ConnectionManagerImpl.java:388)
at
org.datanucleus.store.connection.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:253)
at
org.datanucleus.store.connection.AbstractConnectionFactory.getConnection(AbstractConnectionFactory.java:60)
at
org.datanucleus.store.AbstractStoreManager.getConnection(AbstractStoreManager.java:338)
at
org.datanucleus.store.AbstractStoreManager.getConnection(AbstractStoreManager.java:307)
at
org.datanucleus.store.rdbms.query.JDOQLQuery.performExecute(JDOQLQuery.java:582)
at org.datanucleus.store.query.Query.executeQuery(Query.java:1692)
at org.datanucleus.store.query.Query.executeWithArray(Query.java:1527)
at org.datanucleus.jdo.JDOQuery.execute(JDOQuery.java:243)
at
org.apache.hadoop.hive.metastore.ObjectStore.getMDatabase(ObjectStore.java:405)
at
org.apache.hadoop.hive.metastore.ObjectStore.getDatabase(ObjectStore.java:424)
{code}
Show table takes time too:
{code}
hive> show tables;
OK
load_test_table_20000_0
testTime taken: 285.705 seconds
Log in server:
2013-04-09 19:53:52,783 INFO metastore.HiveMetaStore
(HiveMetaStore.java:logInfo(434)) - 5: get_database: default
2013-04-09 19:54:09,143 INFO metastore.HiveMetaStore
(HiveMetaStore.java:newRawStore(391)) - 5: Opening raw store with implemenation
class:org.apache.hadoop.hive.metastore.ObjectStore
2013-04-09 19:57:44,812 INFO metastore.ObjectStore
(ObjectStore.java:initialize(222)) - ObjectStore, initialize called
2013-04-09 19:57:44,816 INFO metastore.ObjectStore
(ObjectStore.java:setConf(205)) - Initialized ObjectStore
2013-04-09 19:57:51,700 INFO metastore.HiveMetaStore
(HiveMetaStore.java:logInfo(434)) - 6: get_database: default
2013-04-09 19:57:51,706 INFO metastore.HiveMetaStore
(HiveMetaStore.java:newRawStore(391)) - 6: Opening raw store with implemenation
class:org.apache.hadoop.hive.metastore.ObjectStore
2013-04-09 19:57:51,712 INFO metastore.ObjectStore
(ObjectStore.java:initialize(222)) - ObjectStore, initialize called
2013-04-09 19:57:51,714 INFO metastore.ObjectStore
(ObjectStore.java:setConf(205)) - Initialized ObjectStore
2013-04-09 19:57:52,048 INFO metastore.HiveMetaStore
(HiveMetaStore.java:logInfo(434)) - 6: get_tables: db=default pat=.*
2013-04-09 19:57:52,262 ERROR DataNucleus.Transaction
(Log4JLogger.java:error(115)) - Operation rollback failed on resource:
org.datanucleus.store.rdbms.ConnectionFactoryImpl$EmulatedXAResource@18d3a2f,
error code UNKNOWN and transaction: [DataNucleus Transaction, ID=Xid=�,
enlisted
resources=[org.datanucleus.store.rdbms.ConnectionFactoryImpl$EmulatedXAResource@18d3a2f]]
2013-04-09 19:57:52,262 ERROR DataNucleus.Transaction
(Log4JLogger.java:error(115)) - Operation rollback failed on resource:
org.datanucleus.store.rdbms.ConnectionFactoryImpl$EmulatedXAResource@18d3a2f,
error code UNKNOWN and transaction: [DataNucleus Transaction, ID=Xid=�,
enlisted
resources=[org.datanucleus.store.rdbms.ConnectionFactoryImpl$EmulatedXAResource@18d3a2f]]
{code}
> Drop table using hive CLI throws error when the total number of partition in
> the table is around 50K.
> -----------------------------------------------------------------------------------------------------
>
> Key: HIVE-3620
> URL: https://issues.apache.org/jira/browse/HIVE-3620
> Project: Hive
> Issue Type: Bug
> Reporter: Arup Malakar
>
> hive> drop table load_test_table_20000_0;
>
> FAILED: Error in metadata: org.apache.thrift.transport.TTransportException:
> java.net.SocketTimeoutException: Read timedout
>
>
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask
> The DB used is Oracle and hive had only one table:
> select COUNT(*) from PARTITIONS;
> 54839
> I can try and play around with the parameter
> hive.metastore.client.socket.timeout if that is what is being used. But it is
> 200 seconds as of now, and 200 seconds for a drop table calls seems high
> already.
> Thanks,
> Arup
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
