Hive job not distributing

2012-09-01 Thread John Omernik
I have a job that has lots of tiny map tasks that finish very fast (I think my max time was 9 seconds) I understand that I should change my input to avoid that... and it's difficult because these processes are using a transform script on binary data so it makes it difficult to pull off (Long story)

Re: Newbie - Hive Tutorial question, what format is the sample data file in?

2012-09-01 Thread David Swearingen
Thanks! I found it odd that someone would go to all this trouble creating a nice tutorial and then leave it to the user to write data generation code. Strange. Thanks again Sent from my iPhone On Sep 1, 2012, at 11:33 AM, Edward Capriolo wrote: > It is up to the user to decide what that INT me

Re: Newbie - Hive Tutorial question, what format is the sample data file in?

2012-09-01 Thread Edward Capriolo
It is up to the user to decide what that INT means in this case. This tutorial was created very early on. Since then hive has added support for timestamp type which has a clear meaning. On Sat, Sep 1, 2012 at 10:33 AM, David Swearingen wrote: > Thanks. Still not clear to me what a time field is a

Re: Newbie - Hive Tutorial question, what format is the sample data file in?

2012-09-01 Thread David Swearingen
Thanks. Still not clear to me what a time field is as an INT: milleseconds since the epoch? That was my question. Sent from my iPhone On Sep 1, 2012, at 9:37 AM, Edward Capriolo wrote: > I do not think their is a sample file. You can tell the format by > create table statement. > > COMMENT 'Th

Re: Newbie - Hive Tutorial question, what format is the sample data file in?

2012-09-01 Thread Edward Capriolo
I do not think their is a sample file. You can tell the format by create table statement. COMMENT 'This is the staging page view table' ROW FORMAT DELIMITED FIELDS TERMINATED BY '44' LINES TERMINATED BY '12' STORED AS TEXTFILE LOCATION '/user/data/staging/page_view'; http://www.ascii

Newbie - Hive Tutorial question, what format is the sample data file in?

2012-09-01 Thread David Swearingen
I'm going through the tutorial at https://cwiki.apache.org/Hive/tutorial.html . It's not clear to me what the exact format of the log file would be for the sample queries described eg at https://cwiki.apache.org/Hive/tutorial.html#Tutorial-LoadingData I can't find a link to download such a file a

Hive Index Rebuild showing TTransportException

2012-09-01 Thread Prasanna Kumar Jalakam
Hi All, I am new to Hive. I am trying to rebuild the index. Continuously i am getting the following exception. java.sql.SQLException: org.apache.thrift.transport.TTransportException at org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:191) at org.apac