Re: Hive Mapred local task distribution

2014-09-06 Thread Xuefu Zhang
You might be able to control what tasks to run locally. However, once they run locally, they have to do so on HiveServer2 host. It's possible to run the local tasks in separate JVMs. Still, the same host. --Xuefu On Sat, Sep 6, 2014 at 7:51 AM, Abhilash L L wrote: > Hi Xuefu, > > Yea, cur

Re: Parquet Binary Column Support

2014-09-06 Thread Xuefu Zhang
I don't think there is any issue keeping it away. The only issue is resource. We welcome effort from the community to move it forward. I'm willing to coach/review it. --Xuefu On Sat, Sep 6, 2014 at 8:18 AM, John Omernik wrote: > Greetings all - > > We really want to look into the Parquet file

Hive Index and ORC

2014-09-06 Thread Alain Petrus
Hello, I am wondering whether is it possible to use Hive index and ORC format? Does it make sense? Also, is Hive indexing a mature functionality? What are your experiences using Hive indexing? Thanks, Alain

Hive Indexing and ORC

2014-09-06 Thread Alain Petrus
Hello, Is it possible to create an index on table stored as ORC and compressed as Snappy? Does it make sense? I am wondering if Hive indexing is a mature functionality? Thanks, Alain

Hive Indexing and ORC

2014-09-06 Thread Alain Petrus
Hello, Is it possible to create an index on table stored as ORC and compressed as Snappy? Does it make sense? I am wondering if Hive indexing is a mature functionality? Thanks, Alain

Hive Indexing and ORC

2014-09-06 Thread Alain Petrus
Hello, Is it possible to create an index on table stored as ORC and compressed as Snappy? Does it make sense? I am wondering if Hive indexing is a mature functionality? Thanks, Alain

Parquet Binary Column Support

2014-09-06 Thread John Omernik
Greetings all - We really want to look into the Parquet file format more, however, without supporting all the Hive Column types, we are hesitant to dive in more. Currently, it looks like it's just the BINARY column type (which I use) based on the JIRA below, there hasn't been any movement in on t

Re: Hive Mapred local task distribution

2014-09-06 Thread Abhilash L L
Hi Xuefu, Yea, currently we have only one HiveServer2 host where the map red local tasks run. Any other solution other than LBing it ? Regards, Abhilash L L Capillary Technologies M:919886208262 abhil...@capillarytech.com | www.capillarytech.com Email from people at capillarytech.com ma

Re: Hive Mapred local task distribution

2014-09-06 Thread Xuefu Zhang
By "same host", don't you mean your HiveServer2 host? One solution is to have multiple HiveServer2 instances and do load balance among them. --Xuefu On Fri, Sep 5, 2014 at 11:37 PM, Abhilash L L wrote: > Hello, > >We are using Hive 0.11 connecting to it via Hive Thrift server 2. > >A l

Re: Mysql - Hive Sync

2014-09-06 Thread Stephen Sprague
interesting. thanks Muthu. a colleague of mine pointed out this one too, linkedin's databus ( https://github.com/linkedin/databus/wiki) this one looks extremely heavy weight and again not sure its worth the headache. i like the idea of a trigger on the mysql table and then broadcasting the data