Re: Maximum Number of Hive Partitions = 256?

same here ... we have way more than 256 partitions in multiple tables. I am sure the issue has something to do with an empty string passed to the substr function. can you validate that the table has no null/empty string for user_name or try running the query with len(user_name) > 1 (not sure about

RE: Maximum Number of Hive Partitions = 256?

I have way more than 256 partitions per table. AFAIK, there is no partition limit. >From your stack trace, you have some host name issue somewhere. From: Time Less [mailto:timelessn...@gmail.com] Sent: Tuesday, May 03, 2011 6:52 PM To: user@hive.apache.org Subject: Maximum Number of Hive Partit

Maximum Number of Hive Partitions = 256?

I created a partitioned table, partitioned daily. If I query the earlier partitions, everything works. The later ones fail with error: hive> select substr(user_name,1,1),count(*) from u_s_h_b where dtpartition='2010-10-24' group by substr(user_name,1,1) ; Total MapReduce jobs = 1 Launching Job 1 o

RE: Export data with column names

I'm not aware. -Original Message- From: Raghunath, Ranjith [mailto:ranjith.raghuna...@usaa.com] Sent: Tuesday, May 03, 2011 3:29 PM To: user@hive.apache.org Subject: RE: Export data with column names Thanks for the help on this. The option to set the header worked out great. Is there a

RE: Export data with column names

Thanks for the help on this. The option to set the header worked out great. Is there a way that I can export data out to the local file system with the header information? Thank you, Ranjith N. Raghunath -Original Message- From: Steven Wong [mailto:sw...@netflix.com] Sent: Tuesday, Ma

Problem using thriftHive on PHP

I'm getting this error when I try to connect to Hive Server Fatal error: Uncaught exception 'TException' with message 'TSocket: Could not read 4 bytes from localhost:1' in /var/www/html/php/transport/TSocket.php on line 236 TException: TSocket: Could not read 4 bytes from localhost:1 in /

Re: Hive SQL Job Tracker

Hi, What you want should be possible. Do you have a hiveserver (hive thrift) up and running? Check out the logs of the hive server Jasper 2011/5/3 Stuart Scott > Hi, > > > > Wonder if anyone can help please? > > We have set up Hive and are running SQL queries against this using a JDBC > client

RE: Export data with column names

Loren, can you point me to the jira that describes the problem/symptom in more detail? If there is no jira yet, can you describe it? Thanks. -Original Message- From: Loren Siebert [mailto:lo...@siebert.org] Sent: Tuesday, May 03, 2011 11:23 AM To: user@hive.apache.org Cc: user@hive.apa

Question about Pig UDF and sqoop export

New to Pig UDF so requesting help to resolve the issue Created a Pig UDF to transform an input tuple and generate an transformed tuple. Executed a Store to save the transformed tuple into a file B = FOREACH A GENERATE PackageName.MyMethod(); STORE B into 'output' USING PigStorage(); My question

Re: Export data with column names

Sorry, null pointer exception On May 3, 2011, at 12:23, "Raghunath, Ranjith" wrote: > Thanks Loren. Pardon my ignorance but what is an NPE? > > Thank you, > Ranjith N. Raghunath > > -Original Message- > From: Loren Siebert [mailto:lo...@siebert.org] > Sent: Tuesday, May 03, 2011 1

RE: Export data with column names

Thanks Loren. Pardon my ignorance but what is an NPE? Thank you, Ranjith N. Raghunath -Original Message- From: Loren Siebert [mailto:lo...@siebert.org] Sent: Tuesday, May 03, 2011 1:23 PM To: user@hive.apache.org Cc: user@hive.apache.org Subject: Re: Export data with column names Loo

Re: HIVE Server multiple instances

Nothing specifically about our Hive setup although some of us at Forward have blogged bits and pieces about Hive + Hadoop and have a few Hadoop/Hive related libs on our GitHub account: https://github.com/forward. I've blogged a few bits (http://www.oobaloo.co.uk/) as has one of my colleagues (h

Re: HIVE Server multiple instances

Thanks Paul. That is really useful information. - Original Message From: Matthew Rathbone To: user@hive.apache.org Sent: Tue, May 3, 2011 11:18:17 AM Subject: Re: HIVE Server multiple instances Hey Paul, I'd be very interested in reading about your hadoop/hive setup, do you have a

Re: Export data with column names

Look out for NPEs if you happen to set this option to true before doing add/drop table statements. On May 3, 2011, at 11:10, Steven Wong wrote: > set hive.cli.print.header=true; > > > -Original Message- > From: Raghunath, Ranjith [mailto:ranjith.raghuna...@usaa.com] > Sent: Tuesday,

Re: HIVE Server multiple instances

Hey Paul, I'd be very interested in reading about your hadoop/hive setup, do you have a blog post or anything describing this setup, or some of the issues you've have with hive? -- Matthew Rathbone Foursquare | Software Engineer | Server Engineering Team matt...@foursquare.com | @rathboma | 4s

Re: HIVE Server multiple instances

HiveServer does seem to support multiple connections but I think it still has thread-safety problems (https://issues.apache.org/jira/browse/HIVE-80). We've (www.forward.co.uk) certainly had instability problems with the thrift server in the past and now run 5 or so instances behind the HAProxy

RE: Export data with column names

set hive.cli.print.header=true; -Original Message- From: Raghunath, Ranjith [mailto:ranjith.raghuna...@usaa.com] Sent: Tuesday, May 03, 2011 11:03 AM To: 'user@hive.apache.org' Subject: Re: Export data with column names Thanks. Is there something I can do with cli? Thanks, Ranjith

Re: Export data with column names

Thanks. Is there something I can do with cli? Thanks, Ranjith - Original Message - From: Matthew Rathbone To: user@hive.apache.org Sent: Tue May 03 09:45:58 2011 Subject: Re: Export data with column names If you use the thrift client you can call connection.getSchema() -- Matth

Re: HIVE Server multiple instances

Thanks. That really helps and answers my question. - Original Message From: Matthew Rathbone To: user@hive.apache.org Sent: Tue, May 3, 2011 10:59:37 AM Subject: Re: HIVE Server multiple instances Even if it is single threaded it certainly seems to support multiple connections.

Re: HIVE Server multiple instances

Even if it is single threaded it certainly seems to support multiple connections. We run 5 workers all connected at the same time executing a different query each ( with a different connection per worker). Hope that helps Matthew On Tuesday, May 3, 2011 at 1:40 PM, V.Senthil Kumar wrote: Tha

Re: HIVE Server multiple instances

Thanks Matthew. The wiki page http://wiki.apache.org/hadoop/Hive/HiveServer says its single threaded. I have a queue of queries which gets added dynamically all the time. By the time I run 1 query using 1 JDBC connection, the queue gets added more queries and builds up a backlog. So, I was th

Hive SQL Job Tracker

Hi, Wonder if anyone can help please? We have set up Hive and are running SQL queries against this using a JDBC client connection. All working fine. What we can't seem to do is to get it work using the JobTracker to distribute the SQL/mapreduce queries. We have tried Bees Wax and this does

Re: HIVE Server multiple instances

Why would you want to run two? I think it is multithreaded, so you can query it from two different connections -- Matthew Rathbone Foursquare | Software Engineer | Server Engineering Team matt...@foursquare.com | @rathboma | 4sq On Monday, May 2, 2011 at 6:41 PM, V.Senthil Kumar wrote: Hello,

Re: Export data with column names

If you use the thrift client you can call connection.getSchema() -- Matthew Rathbone Foursquare | Software Engineer | Server Engineering Team matt...@foursquare.com | @rathboma | 4sq On Monday, May 2, 2011 at 11:12 PM, Raghunath, Ranjith wrote: Does anyone know how to export data out with column