https://community.hortonworks.com/questions/59681/puthivestreaming-nifi-processor-various-errors.html
has
2016-10-03 23:40:24,322 ERROR [pool-5-thread-114]: metastore.RetryingHMSHandler
(RetryingHMSHandler.java:invokeInternal(195)) -
java.lang.IllegalStateException: Unexpected DataOperationType: UNSET
agentInfo=Unknown txnid:98201
I don’t see this in the stack trace below but if you are seeing this, I think
you need to recompile Storm bolt since it is an uber jar that includes some
Hive classes.
Based on the error above it is using old classes (from before HDP 2.5).
Eugene
On 3/26/17, 1:20 PM, "Marcin Kasiński" <marcin.kasin...@gmail.com> wrote:
Hello.
I have problem with Storm hive bolt.
When I try to save data to hiev I get error Unable to acquire lock
(storm logs below)
I have very simple application (save simple data to hive ).
It works with HDP 2.4 (Apache Hive 1.2.1 and Apache Storm 0.10.0)
I switched to HDP 2.5 (Apache Hive 1.2.1 and Apache Storm 1.0.1)
It stopped working.
I saw simillar error here:
https://community.hortonworks.com/questions/59681/puthivestreaming-nifi-processor-various-errors.html
Ther are saying that there is a issue for Hive Streaming between HDF
2.0 and HDP 2.5
I Ilike HDP 2.5 .
My question is:
Do you know how I can solve this problem ?
... or the only way is to switch back to HDP 2.4 ?
Storm logs below:
org.apache.storm.hive.common.HiveWriter$ConnectFailure: Failed
connecting to EndPoint {metaStoreUri='thrift://ambari.local:9083',
database='default', table='stock_prices', partitionVals=[Marcin] }
at org.apache.storm.hive.common.HiveWriter.<init>(HiveWriter.java:80)
~[stormjar.jar:?]
at org.apache.storm.hive.common.HiveUtils.makeHiveWriter(HiveUtils.java:50)
~[stormjar.jar:?]
at org.apache.storm.hive.bolt.HiveBolt.getOrCreateWriter(HiveBolt.java:259)
~[stormjar.jar:?]
at org.apache.storm.hive.bolt.HiveBolt.execute(HiveBolt.java:112)
[stormjar.jar:?]
at
org.apache.storm.daemon.executor$fn__9362$tuple_action_fn__9364.invoke(executor.clj:734)
[storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
at
org.apache.storm.daemon.executor$mk_task_receiver$fn__9283.invoke(executor.clj:466)
[storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
at
org.apache.storm.disruptor$clojure_handler$reify__8796.onEvent(disruptor.clj:40)
[storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
at
org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:451)
[storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
at
org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:430)
[storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
at
org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
[storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
at
org.apache.storm.daemon.executor$fn__9362$fn__9375$fn__9428.invoke(executor.clj:853)
[storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
at org.apache.storm.util$async_loop$fn__656.invoke(util.clj:484)
[storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245]
at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_77]
Caused by: org.apache.storm.hive.common.HiveWriter$TxnBatchFailure:
Failed acquiring Transaction Batch from EndPoint:
{metaStoreUri='thrift://ambari.local:9083', database='default',
table='stock_prices', partitionVals=[Marcin] }
at org.apache.storm.hive.common.HiveWriter.nextTxnBatch(HiveWriter.java:264)
~[stormjar.jar:?]
at org.apache.storm.hive.common.HiveWriter.<init>(HiveWriter.java:72)
~[stormjar.jar:?]
... 13 more
Caused by: org.apache.hive.hcatalog.streaming.TransactionError: Unable
to acquire lock on {metaStoreUri='thrift://ambari.local:9083',
database='default', table='stock_prices', partitionVals=[Marcin] }
at
org.apache.hive.hcatalog.streaming.HiveEndPoint$TransactionBatchImpl.beginNextTransactionImpl(HiveEndPoint.java:575)
~[stormjar.jar:?]
at
org.apache.hive.hcatalog.streaming.HiveEndPoint$TransactionBatchImpl.beginNextTransaction(HiveEndPoint.java:544)
~[stormjar.jar:?]
at org.apache.storm.hive.common.HiveWriter.nextTxnBatch(HiveWriter.java:259)
~[stormjar.jar:?]
at org.apache.storm.hive.common.HiveWriter.<init>(HiveWriter.java:72)
~[stormjar.jar:?]
... 13 more
Caused by: org.apache.thrift.transport.TTransportException
at
org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
~[stormjar.jar:?]
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
~[stormjar.jar:?]
at
org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
~[stormjar.jar:?]
at
org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
~[stormjar.jar:?]
at
org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
~[stormjar.jar:?]
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
~[stormjar.jar:?]
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_lock(ThriftHiveMetastore.java:3781)
~[stormjar.jar:?]
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.lock(ThriftHiveMetastore.java:3768)
~[stormjar.jar:?]
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.lock(HiveMetaStoreClient.java:1736)
~[stormjar.jar:?]
at
org.apache.hive.hcatalog.streaming.HiveEndPoint$TransactionBatchImpl.beginNextTransactionImpl(HiveEndPoint.java:570)
~[stormjar.jar:?]
at
org.apache.hive.hcatalog.streaming.HiveEndPoint$TransactionBatchImpl.beginNextTransaction(HiveEndPoint.java:544)
~[stormjar.jar:?]
at org.apache.storm.hive.common.HiveWriter.nextTxnBatch(HiveWriter.java:259)
~[stormjar.jar:?]
at org.apache.storm.hive.common.HiveWriter.<init>(HiveWriter.java:72)
~[stormjar.jar:?]
... 13 more
2017-03-26 21:43:05.112 h.metastore [INFO] Trying to connect to
metastore with URI thrift://ambari.local:9083
pozdrawiam
Marcin Kasiński
http://itzone.pl
