----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69462/#review211381 -----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java Line 184 (original), 184 (patched) <https://reviews.apache.org/r/69462/#comment296389> Just realized this needs a new metastore connection. Thrift connections are not thread safe - when you mulitplex calls on a single connection, response messages sometimes get lost or matched to the wrong request. If you look at how Heartbeating is done in DbTxnHandler, it does something similar except that it relies on ThreadLocal in Hive.get(conf).getMSC(). ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java Line 187 (original), 187 (patched) <https://reviews.apache.org/r/69462/#comment296388> why is this added here? The CompactionHeartbeater should do this - Eugene Koifman On Dec. 17, 2018, 9:59 a.m., Jaume Marhuenda wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69462/ > ----------------------------------------------------------- > > (Updated Dec. 17, 2018, 9:59 a.m.) > > > Review request for hive. > > > Repository: hive-git > > > Description > ------- > > Allow the Worker thread in the metastore to run outside of it > > > Diffs > ----- > > > hcatalog/streaming/src/test/org/apache/hive/hcatalog/streaming/TestStreaming.java > b290a40734 > > itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/TestCompactor.java > 5af047f465 > jdbc/src/java/org/apache/hive/jdbc/Utils.java 852942e6a2 > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Cleaner.java 18253c9bab > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java > 42ce1746fd > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorThread.java > f5b901d6e8 > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Initiator.java > cdcc0e9548 > > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MetaStoreCompactorThread.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/RemoteCompactorThread.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java 21043415d3 > ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands2.java 546ff955b7 > ql/src/test/org/apache/hadoop/hive/ql/TxnCommandsBaseForTests.java > 52453a2ec4 > service/src/java/org/apache/hive/service/server/HiveServer2.java 0c55654475 > > standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/CompactionInfoStruct.java > PRE-CREATION > > standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/OptionalCompactionInfoStruct.java > PRE-CREATION > > standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java > b6a0893524 > > standalone-metastore/metastore-common/src/gen/thrift/gen-php/metastore/ThriftHiveMetastore.php > 3170798663 > > standalone-metastore/metastore-common/src/gen/thrift/gen-php/metastore/Types.php > 39f8b1f05a > > standalone-metastore/metastore-common/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote > d57de353c6 > > standalone-metastore/metastore-common/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py > a896849989 > > standalone-metastore/metastore-common/src/gen/thrift/gen-py/hive_metastore/ttypes.py > 4ef4aadfee > > standalone-metastore/metastore-common/src/gen/thrift/gen-rb/hive_metastore_types.rb > 97dc0696b7 > > standalone-metastore/metastore-common/src/gen/thrift/gen-rb/thrift_hive_metastore.rb > a5f976bc5c > > standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java > 9eb1193a27 > > standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java > fa19440ba2 > > standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java > e25a8cf9a1 > standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift > cb899d791f > > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java > 598847df03 > > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreThread.java > 6ef2e3560d > > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/CompactionInfo.java > b34b7d70de > > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java > 3f611d6270 > > standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClientPreCatalog.java > 9fe9a65677 > streaming/src/test/org/apache/hive/streaming/TestStreaming.java 2170178a81 > > > Diff: https://reviews.apache.org/r/69462/diff/3/ > > > Testing > ------- > > > Thanks, > > Jaume Marhuenda > >
