Hive Meta Server (Thrift Server) Failover / Redundancy / Load Balancing

2012-11-07 Thread Manish Malhotra
Hi, I need to build a failover/LB solution for Hive Services. MySQL DB is fine, and can work out. But for Hive Metastore Service, can I simply put the Load Balancer like HA Proxy etc. in between the client and achieve this. Thrift Servers and default stateless, not sure about hive one. I red very

Re: TRANSFORM + LATERAL VIEW?

2012-11-07 Thread Jamie Olson
Thanks. I was just using cat as an example. I'm actually running some R scripts over TRANSFORM. So I'll just have to manually JOIN the results with the original table (or just append the new columns in the TRANSFORM script). Jamie Olson On Wed, Nov 7, 2012 at 1:04 AM, Mark Grover wrote: > J

Re: Alter table is giving error

2012-11-07 Thread Chunky Gupta
Okay Mark, I will be looking into this JIRA regularly. Thanks again for helping. Chunky. On Wed, Nov 7, 2012 at 12:22 PM, Mark Grover wrote: > Chunky, > I just tried it myself. It turns out that the directory you are adding as > partition has to be empty for msck repair to work. This is obviously

Re: Hive NR map progress inconsistent and regurlarly restart from 0%

2012-11-07 Thread Alexandre Fouche
Well, that is my problem here: i checked the logs on resourcemanager and nodemanagers and saw nothing suspicious, but these logs are too verbose. Could it be the data or avro records which have issues ? -- Alexandre Fouche Lead operations engineer, cloud architect http://www.cleverscale.com | @

Re: Hive NR map progress inconsistent and regurlarly restart from 0%

2012-11-07 Thread Jan DolinĂ¡r
This usually happens when some task fail, their progress is then not counted, hence the 'restart'. Check your task logs for failures. Jan On Wed, Nov 7, 2012 at 12:30 PM, Alexandre Fouche wrote: > I have a Yarn MR (with two ec2 instances to mapreduce) job on a dataset of > approximately a thous