Re: UDFs with package names

2013-07-31 Thread Michael Malak
Yup, it was the directory structure com/mystuff/whateverUDF.class that was missing.  Thought I had tried that before posting my question, but... Thanks for your help! From: Edward Capriolo To: "user@hive.apache.org" ; Michael Malak Sent: Tuesda

UDFs with package names

2013-07-30 Thread Michael Malak
Thus far, I've been able to create Hive UDFs, but now I need to define them within a Java package name (as opposed to the "default" Java package as I had been doing), but once I do that, I'm no longer able to load them into Hive. First off, this works: add jar /usr/lib/hive/lib/hive-contrib-0.1

Re: Best Performance on Large Scale Join

2013-07-29 Thread Michael Malak
Perhaps you can first create a temp table that contains only the records that will match?  See the UNION ALL trick at http://www.mail-archive.com/hive-user@hadoop.apache.org/msg01906.html From: Brad Ruderman To: user@hive.apache.org Sent: Monday, July 29, 201

Re: Oracle to Hive

2013-07-10 Thread Michael Malak
Untested: SELECT a.c100, a.c300, b.c400   FROM t1 a   JOIN t2 b   ON a.c200 = b.c200   JOIN (SELECT DISTINCT a.c100           FROM t1 a2           JOIN t2 b2           ON a2.c200 = b2.c200         WHERE b2.c400 >= SYSDATE - 1) a3   ON a.c100 = a3.c100   WHERE b.c400 >= SYSDATE - 1    AND a.c300 =

Re: How Can I store the Hive query result in one file ?

2013-07-04 Thread Michael Malak
I have found that for output larger than a few GB, redirecting stdout results in an incomplete file.  For very large output, I do CREATE TABLE MYTABLE AS SELECT ... and then copy the resulting HDFS files directly out of  /user/hive/warehouse. From: Bertrand De

Re: Fwd: Need urgent help in hive query

2013-06-28 Thread Michael Malak
Just copy and paste the whole long expressions to their second occurrences. From: dyuti a To: user@hive.apache.org Sent: Friday, June 28, 2013 10:58 AM Subject: Fwd: Need urgent help in hive query Hi Experts, I'm trying with the below SQL query in Hive, whi

Re: how to combine some rows into 1 row in hive

2013-06-23 Thread Michael Malak
ang wrote: Thanks Michael! That worked without modification! > > > >On Sat, Jun 22, 2013 at 5:05 PM, Michael Malak wrote: > >Or, the single-language (HiveQL) alternative might be (i.e. I haven't tested >it): >>  >>select f1, >>   f2, >> 

Re: how to combine some rows into 1 row in hive

2013-06-22 Thread Michael Malak
Or, the single-language (HiveQL) alternative might be (i.e. I haven't tested it):   select f1,    f2,    if(max(if(f3='P',f4,null)) is null,0,max(if(f3='P',f4,null))) pf4,   if(max(if(f3='P',f5,null)) is null,0,max(if(f3='P',f5,null))) pf5,    if(max(if(f3='N',f4,null)) is null,0,

Re: INSERT non-static data to array?

2013-06-20 Thread Michael Malak
thing. From: Edward Capriolo To: "user@hive.apache.org" ; Michael Malak Sent: Thursday, June 20, 2013 9:15 PM Subject: Re: INSERT non-static data to array? i think you could select into as sub query and then use lateral view.not exactly the same but somethin

Re: INSERT non-static data to array?

2013-06-20 Thread Michael Malak
I've created https://issues.apache.org/jira/browse/HIVE-4771 to track this issue. - Original Message - From: Michael Malak To: "user@hive.apache.org" Cc: Sent: Wednesday, June 19, 2013 2:35 PM Subject: Re: INSERT non-static data to array? The example code for inlin

Re: INSERT non-static data to array?

2013-06-19 Thread Michael Malak
c int[]); INSERT INTO table_a   SELECT a, b, ARRAY(SELECT c FROM table_c WHERE table_c.parent = table_b.id)   FROM table_b From: Edward Capriolo To: "user@hive.apache.org" ; Michael Malak Sent: Wednesday, June 19, 2013 2:06 PM Subject: Re: INSERT non

INSERT non-static data to array?

2013-06-19 Thread Michael Malak
Is the only way to INSERT data into a column of type array<> to load data from a pre-existing file, to use hard-coded values in the INSERT statement, or copy an entire array verbatim from another table?  I.e. I'm assuming that a) SQL1999 array INSERT via subquery is not (yet) implemented in Hive

Re: Hive Group By Limitations

2013-05-06 Thread Michael Malak
--- On Mon, 5/6/13, Peter Chu wrote: > In Hive, I cannot perform a SELECT GROUP BY on fields not in the GROUP BY > clause. Although MySQL allows it, it is not ANSI SQL. http://stackoverflow.com/questions/1225144/why-does-mysql-allow-group-by-queries-without-aggregate-functions

Re: Hive QL - NOT IN, NOT EXIST

2013-05-05 Thread Michael Malak
--- On Sun, 5/5/13, Peter Chu wrote: > I am wondering if there is any way to do this without resorting to > using left outer join and finding nulls. I have found this to be an acceptable substitute. Is it not working for you?

Re: NULLable STRUCTs

2013-02-19 Thread Michael Malak
If no one has any objection, I'm going to update HIVE-4022, which I entered a week ago when I thought the behavior was Avro-specific, to indicate it actually affects even native Hive tables. https://issues.apache.org/jira/browse/HIVE-4022 --- On Fri, 2/15/13, Michael Malak wrote: &

NULLable STRUCTs

2013-02-15 Thread Michael Malak
It seems that all Hive columns (at least those of primitive types) are always NULLable? What about columns of type STRUCT? The following: echo 1,2 >twovalues.csv hive CREATE TABLE tc (x INT, y INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY ','; LOAD DATA LOCAL INPATH 'twovalues.csv' INTO TABLE

Re: INSERT INTO table with STRUCT, SELECT FROM

2013-02-13 Thread Michael Malak
o I would write to a different directory and then move the files over... dean On Wed, Feb 13, 2013 at 1:26 PM, Michael Malak wrote: Is it possible to INSERT INTO TABLE t SELECT FROM where t has a column with a STRUCT? Based on http://grokbase.com/t/hive/user/109r87hh3e/insert-data-into-a-co