Thanks Ashutosh, For quick reply.
1. For non-secure MetaServer: I'm wondering if I add LoadBalancer like HAProxy in between then we don't need to handle the failover at the Thrift client side. So, If I use LB in between Thrift Client and MetaServer, then it should be good to use ? May be I'm missing something, but I'll check out the code and see what is the status of the patch and what additional work is required. 2. For Secure MetaServer: I need to dig further into code, and then will ask more questions if required. I believe storing token into DB patch is available for review but not available as the one which is ready to use. Or I can try out that patch when using the secured one? Again thanks your help !! Regards, Manish On Thu, Nov 8, 2012 at 9:39 AM, Ashutosh Chauhan <hashut...@apache.org>wrote: > Hi Manish, > > Your understanding is mostly correct, though there is one additional bit. > MetastoreClient in current incarnation don't automatically reconnect in > case connection gets dropped for a connected session. As a result, it won't > failover active connections. New connections would be fine. Fortunately, > though there is a work in progress for this on : > https://issues.apache.org/jira/browse/HIVE-3400 In case you want to help > out you should help there. > > For secure case, as you pointed out you additionally need ZooKeeper to > store security tokens. So, you need to bring up a ZK cluster. But, if you > think dedicating 3 nodes for ZK for metastore is an overhead then you would > need https://issues.apache.org/jira/browse/HIVE-3255 With that patch, > tokens are stored in same backend db, so there would be no need to bring up > ZK cluster. > > Hopefully, both of these patches gets in for 0.10 release. > > Thanks, > Ashutosh > > On Wed, Nov 7, 2012 at 11:28 PM, Manish Malhotra < > manish.malhotra.w...@gmail.com> wrote: > >> Hi, >> >> I need to build a failover/LB solution for Hive Services. >> MySQL DB is fine, and can work out. >> But for Hive Metastore Service, can I simply put the Load Balancer like >> HA Proxy etc. in between the client and achieve this. >> Thrift Servers and default stateless, not sure about hive one. >> >> I red very few comments on this problem. >> Similar approach blogged at : >> http://www.edwardcapriolo.com/roller/edwardcapriolo/entry/running_a_hive_thrift_cluster >> >> Very Imp from HCatalog mailing thread: >> http://mail-archives.apache.org/mod_mbox/incubator-hcatalog-user/201109.mbox/%3cdcae2122-f3c6-428a-be70-5c994c0dc...@yahoo-inc.com%3E >> >> >> As per this mailing thread, if security is used in HIVE thrift meta >> server then need to do more modification in the server as it maintain the >> token of user for that session. (user connection). >> >> Please help me to move forward on this problem and please verify if my >> understanding is correct or not on the above 2 blogs / mail. >> >> Is there any initial work done under HCatalog or Hive, which I can look >> into and extend / patch. >> >> >> Regards, >> Manish >> > >
