Challenge : How to load data of multiple rows into a single row in hive table (can regex be used in this scenario)

2012-05-01 Thread m sreenath
Hi I have a large data-set of the form : A    12323:213 B    adasd C    sedweqwd A    12323:213 B    adasd C    sedweqwd The general form is : \t \t \t\n In the given format, I need the part from the three rows in a table containing three columns (time,name,content). Can regex be used for thi

Re: add column to hive table

2012-05-01 Thread Gabi D
you probably noticed this already, but if you add a column in the middle and it did not exist in your older files then when you select from older dates you will get wrong values in the wrong columns since you will be looking at old files with the new format. Dangerous. We also went with the sqoop t

RE: '\N' is displayed in case of null column values in exporting hive query results to CSV file

2012-05-01 Thread Garg, Rinku
Hi Ashwanth, Thanks for the quick reply. NULL column value is not located in last(in schema). We are getting '\N' for every column which has a null values whether it is a first column, middle column or last column. Thanks & Regards, Rinku Garg IT Specialist - Global Commercial Services From:

Re: '\N' is displayed in case of null column values in exporting hive query results to CSV file

2012-05-01 Thread Ashwanth Kumar
Well a quick dirty way would be to use CASE WHEN with your SELECT statement. Eg. create table tmp_test ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' as select [normalCols], CASE WHEN col = NULL THEN 'NULL' ELSE col END AS col, [otherCols] from table1; PS - Well this isn'

Re: '\N' is displayed in case of null column values in exporting hive query results to CSV file

2012-05-01 Thread Ashwanth Kumar
Is the NULL value column located in last (in schema)? On Wed, May 2, 2012 at 10:44 AM, Garg, Rinku wrote: > Hi All, > > We did a successful setup of hadoop-0.20.203.0 and hive-0.7.1. We also > loaded a large number of CSV files into HDFS successfully. We can query > through hive CLI. Now whe

'\N' is displayed in case of null column values in exporting hive query results to CSV file

2012-05-01 Thread Garg, Rinku
Hi All, We did a successful setup of hadoop-0.20.203.0 and hive-0.7.1. We also loaded a large number of CSV files into HDFS successfully. We can query through hive CLI. Now when we execute any query on the hive command prompt, we are getting result back. While we did following steps to get back

RE: add column to hive table

2012-05-01 Thread Chalcy Raja
In my situation, the tables I was importing into hive are daily tables. Couple of columns were added in a month which are not added to the end of the table. Also one field got dropped in between. Also I have data for a year. Anyway the way, I handled is, created an external table on top of the

Re: Hive on Standalone Machine

2012-05-01 Thread buddhika chamith
Yes. My requirement was to avoid using bin/hadoop. I still required the job to be run in a new JVM. However I think we can make it run without a fork with some changes. I will give it a try as well and see. Regards Buddhika On Tue, May 1, 2012 at 11:23 PM, Edward Capriolo wrote: > Ok your still

Re: Hive on Standalone Machine

2012-05-01 Thread Edward Capriolo
Ok your still building a command line it seems. I assumed you were working to launch hive without a fork. Pretty cool though. On Tue, May 1, 2012 at 1:08 PM, buddhika chamith wrote: > Hi Ed, > > On Thu, Apr 26, 2012 at 12:26 PM, buddhika chamith > wrote: >> >> >> >> On Thu, Apr 26, 2012 at 10:07

Re: Hive on Standalone Machine

2012-05-01 Thread buddhika chamith
Hi Ed, On Thu, Apr 26, 2012 at 12:26 PM, buddhika chamith wrote: > > > On Thu, Apr 26, 2012 at 10:07 AM, Edward Capriolo > wrote: > >> we have an open ticket to run jobs without bin/hadoop do you care to >> share that code? >> > > Sure I will provide a patch. > Sorry for the delay. Attached a p

Re: nested UDFs on Partition column

2012-05-01 Thread Edward Capriolo
@Deterministic was added after many of the UDFs some that are deterministic may not be marked as such. On Tue, May 1, 2012 at 9:38 AM, Ramkumar wrote: > Thanks. Yes, thats what the UDF writers seemed to have intended. The way > this has been done is by using the 'deterministic' attribute in  UDFT

Re: nested UDFs on Partition column

2012-05-01 Thread Ramkumar
Thanks. Yes, thats what the UDF writers seemed to have intended. The way this has been done is by using the 'deterministic' attribute in  UDFType annotation. See http://javasourcecode.org/html/open-source/hive/hive-0.7.1/org/apache/hadoop/hive/ql/udf/UDFUnixTimeStamp.java.html

For all those that complain about the buggy hive web interface...

2012-05-01 Thread Edward Capriolo
http://people.apache.org/~hashutosh/hive-clover/hwi/ Clover says it has the most test coverage in the enter project at 53% :) CLI comes in at 0% The situation here is that clover does not understand the .q files and velocity templating which would drive the clover % very high for QL (and the rest

Re: JOIN + LATERAL VIEW + MAPJOIN = no output?!

2012-05-01 Thread Edward Capriolo
Yes make a ticket On Tuesday, May 1, 2012, Ruben de Vries wrote: > I really do feel like this isn't as intended, should I make a ticket in JIRA? > > -Original Message- > From: Ruben de Vries [mailto:ruben.devr...@hyves.nl] > Sent: Thursday, April 26, 2012 3:37 PM > To: user@hive.apache.or

RE: JOIN + LATERAL VIEW + MAPJOIN = no output?!

2012-05-01 Thread Ruben de Vries
I really do feel like this isn't as intended, should I make a ticket in JIRA? -Original Message- From: Ruben de Vries [mailto:ruben.devr...@hyves.nl] Sent: Thursday, April 26, 2012 3:37 PM To: user@hive.apache.org Subject: RE: JOIN + LATERAL VIEW + MAPJOIN = no output?! https://gist.gith