Re: tons of bugs and problem found

2011-01-27 Thread Laurent Laborde
On Fri, Jan 28, 2011 at 1:12 AM, Namit Jain wrote: > Hi Laurent, > > 1. Are you saying that _top.sql did not exist in the home directory. > Or that, _top.sql existed, but hive was not able to read it after loading It exist, it's loaded, and i can see it in the hive's warehouse directory. it's jus

Re: Hive Error on medium sized dataset

2011-01-27 Thread hadoop n00b
Return code 2 essentially means a hadoop error. Congrats on locating and fixing your issue. However, can somebody still throw some light on this particular error code? On Fri, Jan 28, 2011 at 6:16 AM, Christopher, Pat < patrick.christop...@hp.com> wrote: > It was the SerDe. There was a null p

Re: csv import with quotes

2011-01-27 Thread Larry Ogrodnek
Are you using the jar I built? It should be: create table my_table(a string, b string, c string) row format serde 'com.bizo.hive.serde.csv.CSVSerde' stored as textfile ; I.e. the class needs to be "com.bizo.hive.serde.csv.CSVSerde" unless you built your own jar? On Fri, Jan 21, 2011 at 1:

RE: Hive Error on medium sized dataset

2011-01-27 Thread Christopher, Pat
It was the SerDe. There was a null pointer error. It was getting reported to a hadoop logfile and not to anywhere in Hive. I found the hadoop log and fixed the problem. Thanks for the help! Pat From: Christopher, Pat Sent: Thursday, January 27, 2011 11:21 AM To: user@hive.apache.org Subject

Re: Newlines in data

2011-01-27 Thread Viral Bajaria
Well even I learned the hard way that Hive does not obey escaping (atleast not in it's current iteration) --> correct me if I am wrong. I currently get rid of the column separators (\t) and line separators (\n) in our table definitions for now. When i was using hive 0.3 it used to allow \r\n as th

Re: tons of bugs and problem found

2011-01-27 Thread Namit Jain
Hi Laurent, 1. Are you saying that _top.sql did not exist in the home directory. Or that, _top.sql existed, but hive was not able to read it after loading 2. I don¹t think reserved words are documented somewhere. Can you file a jira for this ? 3. The bad row is printed in the task log. 1. 2

Newlines in data

2011-01-27 Thread Jay Ramadorai
How can we preserve newlines in data in Hive columns? Currently newlines in data terminate the Hive record. I am using Sqoop to import data into Hive from external databases. Even using the the Escaped By clause of CREATE TABLE, Hive does not escape newlines. I see there is a JIRA for this http

RE: Hive Error on medium sized dataset

2011-01-27 Thread Christopher, Pat
I removed the part of the SerDe that handled the arbitrary key/value pairs and I was able to process my entire data set. Sadly the part I removed has all the interesting data. I'll play more with the heap settings and see if that lets me process the key/value pairs. Is the below the correct w

RE: Hive Error on medium sized dataset

2011-01-27 Thread Christopher, Pat
It will be tricky to clean up the data format as I'm operating on somewhat arbitrary key-value pairs in part of the record. I will try and create something similar though, might take a bit. Thanks. I've tried resetting the heap size, I think. I added the following block to my mapred-site.xml

tons of bugs and problem found

2011-01-27 Thread Laurent Laborde
Friendly greetings ! I am very frustrated about hive. i like the program but i find a problem avery day i try to use it and i can't find any support on irc. so here i am ! 1st problem : hive> create table top_test like top; OK hive> load data local inpath '/home/ker2x/over-blog/archive-20101213/_

Re: Stopping Hive Metastore Service

2011-01-27 Thread Dave Brondsema
That's what we do, seems to be ok. On Tue, Jan 25, 2011 at 2:40 PM, Matias Silva wrote: > Hi I’m in the process of setting up an init script for the hive metastore. > Whats the proper way > > to shutdown the hive metastore without killing pid? Or is killing the pid > the only way? > > > > http://