How to conver into date in hive

2012-10-15 Thread yogesh.kumar13
Hi all, Is there any way to convert months in names. like. 1 to Jan 2 to Feb 3 to Mar and so on (here 1, 2, 3 are having string data type) Please suggest. Thanks & Regards Yogesh Kumar The information contained in this electronic message and any attachments to this message

RE: Hive Query Unable to distribute load evenly in reducers

2012-10-15 Thread Saurabh Mishra
by using mapjoin if you are implying setting set hive.auto.convert.join=true; then this configuration i am already using, but to no avail...:( Date: Tue, 16 Oct 2012 14:17:47 +0900 Subject: Re: Hive Query Unable to distribute load evenly in reducers From: navis@nexr.com To: user@hive.apache.o

Re: Hive Query Unable to distribute load evenly in reducers

2012-10-15 Thread Navis류승우
How about using MapJoin? 2012/10/16 Saurabh Mishra > no there is apparently no heavy skewing. also another stats i wanted to > point was, following is approximate table contents in this 4 table join > query : > tableA : 170 million (actual number, + i am also exploding these records, > so the nu

Lost data during INSERT query

2012-10-15 Thread Jim Krehl
I'm trying to load a table using an INSERT query [1]. Not all the data is making it from the original table into the new table. The Hadoop task tracker logs show that the query works without error until the last second of the job. The job typically takes about 45 minutes, but in actual last seco

RE: Hive Query Unable to distribute load evenly in reducers

2012-10-15 Thread Saurabh Mishra
no there is apparently no heavy skewing. also another stats i wanted to point was, following is approximate table contents in this 4 table join query : tableA : 170 million (actual number, + i am also exploding these records, so the number could be much much higher) tableB:15 tableC:45 tableD:45

Hive Issues RCFile - Binary fields Corruption or field storage issues?

2012-10-15 Thread John Omernik
I am putting binary data into binary columns in hive and using RCFile. Most data is just fine in my very large table, however queries over certain time frames get me RCFile/Compression issues. The data goes in fine. Is this a FS level corruption issue? Is this something tunable? How would I even

Re: Partitions on hive hbase table

2012-10-15 Thread bharath vissapragada
I'm not sure about partitioning but the scans are currently limited based on start and stop keys ( if predicates on rowkeys are provided in the query) See Hive-1643 ,2815 jiras ! On Mon, Oct 15, 2012 at 10:09 PM, kulkarni.swar...@gmail.com < kulkarni.swar...@gmail.com> wrote: > All, > > So, I ha

Re: Any advice about complex Hive tables?

2012-10-15 Thread Sadananda Hegde
Thanks, Chuck. On Sat, Oct 13, 2012 at 7:50 PM, Connell, Chuck wrote: > You can get the jar from the "Downloads" link on the serde's home page. > It is on the right side of the page. > > > -- > *From:* Sadananda Hegde [saduhe...@gmail.com] > *Sent:* Saturday, October

Re: Hive Query Unable to distribute load evenly in reducers

2012-10-15 Thread Philip Tromans
Is your data heavily skewed towards certain values of a.x etc? On 15 October 2012 15:23, Saurabh Mishra wrote: > The queries are simple joins, something on the lines of > select a, b, c, count(D) from tableA join tableB on a.x=b.y join group > by a, b,c; > > >> From: liy...@gmail.com >> Date:

RE: Hive Query Unable to distribute load evenly in reducers

2012-10-15 Thread Saurabh Mishra
The queries are simple joins, something on the lines of select a, b, c, count(D) from tableA join tableB on a.x=b.y join group by a, b,c; > From: liy...@gmail.com > Date: Mon, 15 Oct 2012 21:10:39 +0800 > Subject: Re: Hive Query Unable to distribute load evenly in reducers > To: user@hive.a

RE: Hive bug in Partition

2012-10-15 Thread Chalcy Raja
Of course we have a trash policy in place. So I am worried about losing data instantly. It drops all the paritions of the table one by one. Finding the bug was accidental and then realize how scary if we had to lose all partitions. Fixing it may prevent some accidental dropping. Thanks, Chal

got exception trying to get groups for user

2012-10-15 Thread ashok.samal
Hello Hive, please find below error from JT log.All of my queries are stuck in 100% 33% or 0% 0%.Any help ll be appreciated. got exception trying to get groups for user XX org.apache.hadoop.util.Shell$ExitCodeException: id: XX: No such user at org.apache.hadoop.util.Shell.runCommand(She

Re: Hive Query Unable to distribute load evenly in reducers

2012-10-15 Thread MiaoMiao
And your queries were? On Mon, Oct 15, 2012 at 8:09 PM, Saurabh Mishra wrote: > Hi, > I am firing some hive queries joining tables containing upto 30millions > records each. Since the load on the reducers is very significant in these > cases, i specifically set the following parameters before exe

Hive Query Unable to distribute load evenly in reducers

2012-10-15 Thread Saurabh Mishra
Hi, I am firing some hive queries joining tables containing upto 30millions records each. Since the load on the reducers is very significant in these cases, i specifically set the following parameters before executing the queries : set mapred.reduce.tasks=100; set hive.exec.reducers.bytes.per.r

Re: Index column out of bounds

2012-10-15 Thread Robin Verlangen
Does anyone out here have a solution to this? Best regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named