> On Dec. 13, 2018, 12:04 a.m., Eugene Koifman wrote: > > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java > > Line 97 (original), 94 (patched) > > <https://reviews.apache.org/r/69462/diff/2/?file=2112899#file2112899line98> > > > > this is also intialzied in init() - should it throw here instead? It > > seems that the contract is that init() must be called first
init() should be called first. This is part of the retry logic in case the msc is created again. > On Dec. 13, 2018, 12:04 a.m., Eugene Koifman wrote: > > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java > > Lines 241 (patched) > > <https://reviews.apache.org/r/69462/diff/2/?file=2112899#file2112899line248> > > > > What is the purpose of this? Why doesn't the existing catch(Throwable) > > with the same log msg work? The logic there is to try restart the msc if we think something is wrong with it. I think that only (TException | IOException) would indicate that so we don't need to do it for every exception. > On Dec. 13, 2018, 12:04 a.m., Eugene Koifman wrote: > > service/src/java/org/apache/hive/service/server/HiveServer2.java > > Lines 1016 (patched) > > <https://reviews.apache.org/r/69462/diff/2/?file=2112902#file2112902line1016> > > > > Are there any tests that actually enable HIVE_MAPREDUCE_AVAILABLE ? Hmm, the problem with this is that the Worker thread seems to already be tested without actually running in the metastore. It's just created with w = new Worker() and w.run() > On Dec. 13, 2018, 12:04 a.m., Eugene Koifman wrote: > > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java > > Lines 1476 (patched) > > <https://reviews.apache.org/r/69462/diff/2/?file=2112919#file2112919line1478> > > > > I would put both of these in CompactionInfo. If someone adds fields to > > CompactionInfo, they are unlikely to ever find these methods and so some > > info will be lost in the marshalling back and forth. > > > > Alternatively, could CompactionInfo be a subclass of > > CompactionInfoStruct? I think it could be a subclass and it would be a clean approach, but I'd rather do this in a different patch. I'll move the methods to CompactionInfo. - Jaume ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69462/#review211256 ----------------------------------------------------------- On Dec. 17, 2018, 5:59 p.m., Jaume Marhuenda wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69462/ > ----------------------------------------------------------- > > (Updated Dec. 17, 2018, 5:59 p.m.) > > > Review request for hive. > > > Repository: hive-git > > > Description > ------- > > Allow the Worker thread in the metastore to run outside of it > > > Diffs > ----- > > > hcatalog/streaming/src/test/org/apache/hive/hcatalog/streaming/TestStreaming.java > b290a40734 > > itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/TestCompactor.java > 5af047f465 > jdbc/src/java/org/apache/hive/jdbc/Utils.java 852942e6a2 > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Cleaner.java 18253c9bab > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java > 42ce1746fd > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorThread.java > f5b901d6e8 > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Initiator.java > cdcc0e9548 > > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MetaStoreCompactorThread.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/RemoteCompactorThread.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java 21043415d3 > ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands2.java 546ff955b7 > ql/src/test/org/apache/hadoop/hive/ql/TxnCommandsBaseForTests.java > 52453a2ec4 > service/src/java/org/apache/hive/service/server/HiveServer2.java 0c55654475 > > standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/CompactionInfoStruct.java > PRE-CREATION > > standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/OptionalCompactionInfoStruct.java > PRE-CREATION > > standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java > b6a0893524 > > standalone-metastore/metastore-common/src/gen/thrift/gen-php/metastore/ThriftHiveMetastore.php > 3170798663 > > standalone-metastore/metastore-common/src/gen/thrift/gen-php/metastore/Types.php > 39f8b1f05a > > standalone-metastore/metastore-common/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote > d57de353c6 > > standalone-metastore/metastore-common/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py > a896849989 > > standalone-metastore/metastore-common/src/gen/thrift/gen-py/hive_metastore/ttypes.py > 4ef4aadfee > > standalone-metastore/metastore-common/src/gen/thrift/gen-rb/hive_metastore_types.rb > 97dc0696b7 > > standalone-metastore/metastore-common/src/gen/thrift/gen-rb/thrift_hive_metastore.rb > a5f976bc5c > > standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java > 9eb1193a27 > > standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java > fa19440ba2 > > standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java > e25a8cf9a1 > standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift > cb899d791f > > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java > 598847df03 > > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreThread.java > 6ef2e3560d > > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/CompactionInfo.java > b34b7d70de > > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java > 3f611d6270 > > standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClientPreCatalog.java > 9fe9a65677 > streaming/src/test/org/apache/hive/streaming/TestStreaming.java 2170178a81 > > > Diff: https://reviews.apache.org/r/69462/diff/3/ > > > Testing > ------- > > > Thanks, > > Jaume Marhuenda > >
