Re: Allow other implementations of IMetaStoreClient in Hive

2015-12-15 Thread Austin Lee
FYI - I have created the following JIRA for this: https://issues.apache.org/jira/browse/HIVE-12679 Thanks, Austin On Tue, Dec 15, 2015 at 12:49 PM, Alan Gates wrote: > I think opening a JIRA is a good next step. > > Alan. > > Austin Lee > December 15, 2015 at 11:19 > Thank you so much Alan fo

Re: Allow other implementations of IMetaStoreClient in Hive

2015-12-15 Thread Alan Gates
I think opening a JIRA is a good next step. Alan. Austin Lee December 15, 2015 at 11:19 Thank you so much Alan for your prompt responses and for the information you provided. I will have a look at the HBase work. I am new to the process and it's not 100% clea

Re: Allow other implementations of IMetaStoreClient in Hive

2015-12-15 Thread Austin Lee
Thank you so much Alan for your prompt responses and for the information you provided. I will have a look at the HBase work. I am new to the process and it's not 100% clear to me, but the wiki seems to suggest I should use this forum to get to consensus on a proposal before creating a JIRA ticket

Re: Allow other implementations of IMetaStoreClient in Hive

2015-12-15 Thread Alan Gates
I don't see an issue with this, it seems fine. One caveat though is we see this as an internal interface and we change it all the time. I wouldn't want to be pushed into making backwards compatibility guarantees for IMetaStoreClient. Which means that if you develop a different implementation

Re: Allow other implementations of IMetaStoreClient in Hive

2015-12-15 Thread Alan Gates
For work along the same lines you should check out the HBase metastore work in Hive 2.0. It still uses the thrift server and RawStore but puts HBase behind it instead of an RDBMS. We did this because we found that most of the inefficiencies of Hive's metadata access had to do with the layout

Re: Allow other implementations of IMetaStoreClient in Hive

2015-12-15 Thread Austin Lee
Yes, a more efficient implementation is what I am trying to achieve. I also want to retain the ability to talk to a remote metastore that is not necessarily thrift. To be more precise, what I would like is a more efficient metastore. In looking at the current architecture, I came to a conclusion

Allow other implementations of IMetaStoreClient in Hive

2015-12-14 Thread Austin Lee
Hi, I would like to propose a change that would make it possible for users to choose an implementation of IMetaStoreClient via HiveConf, i.e. hive-site.xml. Currently, in Hive the choice is hard coded to be SessionHiveMetaStoreClient in org.apache.hadoop.hive.ql.metadata.Hive. There is no other d