Problem With Simple Select From Hive

2011-07-06 Thread Sal Scalisi
I'm teaching myself hive and I'm having an issue with a simple select statement. I'm using Amazon's Elastic MapReduce. I'm creating a table with some sample web log data I downloaded from a sample given by Amazon. I create this table: CREATE EXTERNAL TABLE log_parser_stage ( remoteAddr

wiki write access

2011-07-06 Thread Greg Dingle
As instructed on this page, https://cwiki.apache.org/confluence/display/Hive/AboutThisWiki, I'd like write access to the wiki so I can add a description of DESCRIBE FORMATTED. Thanks.

RE: Hive session locking up after 4 queries using S3

2011-07-06 Thread Aggarwal, Vaibhav
Hi Wouter You may want to upgrade to jets3t 0.7.1 or higher which will likely solve your problem. The release notes of jets3t 0.7.1 do say : REST implementation was mistakenly limited to 20 simultaneous connections. Alternatively, you can avoid some of these types of problems by using Amazon El

Re: Partitions

2011-07-06 Thread Aniket Mokashi
Thanks Prashanth. But, it means I have to fire one alter table add partition query for every date-sub-directory I have inside '/abc/xyz'. Although, this doesn't seem unreasonable but it would have been simpler if hive could automatically identify the arrival of data. There was a similar example o

Re: Hive session locking up after 4 queries using S3

2011-07-06 Thread Wouter de Bie
Hi! I'm using Hive 0.7.0 and Hadoop 0.20, both from Cloudera's cdh3u0. Jets3t is used from jets3t-0.6.1.jar. I've just found a post (https://forums.aws.amazon.com/thread.jspa?threadID=19076&tstart=0) that describes this issue and I'm trying to figure out if this bug is in this version. // Wo

RE: Hive session locking up after 4 queries using S3

2011-07-06 Thread Aggarwal, Vaibhav
Could you please tell us which Hadoop and Hive version are you using? Looks like you might be using an older version of Hadoop (more specifically one which ships with old version of jets3t). Thanks Vaibhav From: Wouter de Bie [mailto:wou...@spotify.com] Sent: Wednesday, July 06, 2011 9:07 AM To:

Re: Partitions

2011-07-06 Thread Prashanth R
Hey Aniket, Well. I dont think there is a way to insert data as you had described via your second command. However you could have a cron that invokes a script that keeps changing the insertdate and you could point it to the directory where it has nothing but only the files (that has data) which wi

Hive session locking up after 4 queries using S3

2011-07-06 Thread Wouter de Bie
Hi all, I'm using Hive with the s3native FS. Today, I noticed that hive locks up after 4 queries that directly access S3 (select * from mytable limit 10). With debug logging on, I get the following output: 2011-07-06 15:54:31,459 DEBUG s3native.NativeS3FileSystem (NativeS3FileSystem.java:getFi