HiveImport: FAILED: Error in semantic analysis: Non-Partition column appears in the partition specification

2012-09-21 Thread Alex Haralampiev
I am testing sqoop loads to hive from Oracle 11.2.0.3 (CDH4 w/ hive-common-0.8.1-cdh4.0.1.jar) and getting the error in the subject line. The BU_NM column is the partition key in Oracle for the EDW_PROD table so I am not sure what the error means in the hive "lingo" ;-) Any suggestions or pointers

Does Hive expose anything useful over JMX?

2012-09-21 Thread Jakub Holy
Hello, Does Hive expose any interesting information or stats over JMX, similarly to Hadoop's JobTracker and NameNode? I haven't found any in hive 0.8.1 even though f.ex. https://issues.apache.org/jira/browse/HIVE-1818 would indicate that there should be some metrics exposed. (I executed Hive wit

Re: Map issue in Hive.

2012-09-21 Thread Manish
Hey Bejoy! Thanks a ton. Things are so easy in Hive :) This is how my SQL looks like after defining Map (associative Array). select pv.c_14["+UserType"] from page_view_tmp_2 pv where pv.c_14["+LastLogin"] IS NOT NULL Thanks Again, Manish. On Fri, 2012-09-21 at 00:45 -0700, Bejoy KS wrote: >

Re: Hive Transform Scripts Ending Cleanly

2012-09-21 Thread Edward Capriolo
There is a setting in hive site which allows transform scripts to continue even if they take a long time to return a single row. Edward On Fri, Sep 21, 2012 at 8:55 AM, John Omernik wrote: > Greetings All - > > I have a transform script that some some awesome stuff (at least to my eyes) > > Basi

Hive Transform Scripts Ending Cleanly

2012-09-21 Thread John Omernik
Greetings All - I have a transform script that some some awesome stuff (at least to my eyes) Basically, here is the SQL SELECT TRANSFORM (filename) USING 'worker.sh' as (col1, col2, col3, col4, col5) FROM mysource_filetable worker.sh is actually a wrapper script that looks like this:

Re: Hive ignoring buckets when using dynamic where

2012-09-21 Thread Robin Verlangen
@Jamie: I was trying this too with a view, like: DROP VIEW IF EXISTS standard_today; CREATE VIEW standard_today AS SELECT * FROM standard_feed WHERE bdate='2012-09-21'; However when I run select * FROM standard_today it starts iterating over all data again. Best regards, Robin Verlangen *Softwa

RE: Map issue in Hive.

2012-09-21 Thread Manish . Bhoge
No Problem. I had something different in mind where I wanted to split this complete string into different columns to simplify the queries, like ASP.NET_SessionId, Rviewd, UserId, UserType, LastLogin Now let me try with your approach. I have seen this DDL in hive tutorial but wasn't sure whether

Re: Map issue in Hive.

2012-09-21 Thread Bejoy KS
Hey Manish Sorry If my post was not clear. You need to use either Array or Map for that based on the data it holds. looking at your sample data  ASP.NET_SessionId=bzqgdenuhxxyqmc2vv5tvrdw;+Rviewd=;+UserId=%7bb5cecc61-cd09-4aa6-bc92-cae367f1753b%7d;+UserType=G;+LastLogin=9/11/2012+12:00:01+AM I

RE: Map issue in Hive.

2012-09-21 Thread Manish . Bhoge
Thanks Bejoy, So you mean to say in the below scenario we have to have both collection and map together? Do I need to define Array and MAP together for the same column? As I understand from your mail this column has not only MAP but collection of Maps. Is this assumption is right? Thank You, Ma