Hive 0.7.1 does not work with PostgreSQL 9.0.2

Hi, I have just set up a PostgreSQL 9.0.2 server for hive 0.7.1 metastore, and I am using the postgresql-9.0-801.jdbc4.jar jdbc driver, when I test the following HQL, CREATE TABLE records (year STRING, temperature INT, quality INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'; the following e

RE: org.apache.hadoop.fs.ChecksumException: Checksum error:

This is a really curious case. How many replicas of each block do you have? Are you able to copy the data directly using HDFS client? You could try the hadoop fs -copyToLocal command and see if it can copy the data from hdfs correctly. That would help you verify that the issue really is at HDFS

RE: Alter table Set Locations for all partitions

You could also specify fully qualified hdfs path in the create table command. It could look like create external table test(key string ) row format delimited fields terminated by '\000' collection items terminated by ' ' location 'hdfs://new_master_host:port/table_path'; Then you can use the 'ins

Re: Ignore subdirectories when querying external table

On similar lines, I want to have hive inlcude subdirs. That is.. I have an external table paritioned by month (data for each month under a folder). Under the current month I want to keep adding folders daily . Is this possible without having to subclass InputFormat ? On Aug 19, 20

org.apache.hadoop.fs.ChecksumException: Checksum error:

For some reason, my questions sent two days ago again never shows up, even though I can google the question. I apologize if you have seen this question before. After loading around 2G or so data in a few files into hive, the "select count(*) from table" query keep failing. The JobTracker UI gives

Alter table Set Locations for all partitions

Hey Folks, I am wondering what is the easiest way to migrate data off one hadoop/hive cluster to another. I distcp all data to new cluster, and then copied the metadata directory to new cluster. hive comes up fine and show tables etc but the hive location is still pointing to old cluster There i

unsubscribe

Re: Ignore subdirectories when querying external table

I solved my own problem. For anyone who's curious: It turns out that subclassing an InputFormat allows one to override the listStatus method, which returns the list of files for Hive (or mapreduce in general) to process. All I had to do was subclass org.apache.hadoop.mapred.TextInputFormat and ove

Re: problem in reading data fro table (fields terminated by '||')

You are going to have to write your own serde. AFAIK when you create a standard table the LazySimpleSerde is the default and it only accepts single character delimiters. It you want to 'cheat' the lazy simple serde, you can use 'fields terminated by |' and introduce a dummy column after each real o

RE: problem in reading data fro table (fields terminated by '||')

You can write a map/reduce job to do it for you accordingly, leveraging its power of parallel processing. ** Cheers !!! Siddharth Tiwari Have a refreshing day !!! Date: Fri, 19 Aug 2011 18:46:22 +0530 Subject: Re: problem in reading data fro table (fields terminated

Re: problem in reading data fro table (fields terminated by '||')

hey sid!! thanks bro... but cant parse the file actually have 3TB data in that format . so i need to find the solution and 1 more thing it ll take much time to parse it. regards Vikas Srivastava On Fri, Aug 19, 2011 at 6:41 PM, Siddharth Tiwari wrote: > You will have to parse this da

RE: problem in reading data fro table (fields terminated by '||')

You will have to parse this data accordingly ** Cheers !!! Siddharth Tiwari Have a refreshing day !!! Date: Fri, 19 Aug 2011 16:05:21 +0530 Subject: problem in reading data fro table (fields terminated by '||') From: vikas.srivast...@one97.net To: user@hive.apache.or

problem in reading data fro table (fields terminated by '||')

HI team, I m facing a problem like i have made a table which is (** fields terminated by '||' and lines terminated by '\n'*) but when i fetching data from this table . i didnt get the desire output . *data is in below format* IN||2011-03-28 21:59:24||2011-03-28 22:00:09||919040573650122||0|