Re: pass entire row as parameter in hive UDF

2011-11-01 Thread Chen Song
Can this be only used in regular select statement or also as arguments to UDF? In this case, how shall I define my UDF/GenericUDF method signature to accept column in this form? Will Hive automatically expand the column list and pass them to customized UDF? If there is any example, that would

RE: What is best way to load data into hive tables/hadoop file system

2011-11-01 Thread Steven Wong
Run multiple concurrent LOAD DATAs, one per file. Alternatively, if your TT nodes have access to the source file system, use a map-only Hadoop job, such as distcp. From: Shantian Purkad [mailto:shantian_pur...@yahoo.com] Sent: Monday, October 31, 2011 4:34 PM To: common-u...@hadoop.apache.org;

RE: pass entire row as parameter in hive UDF

2011-11-01 Thread Steven Wong
Would https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Select#LanguageManualSelect-REGEXColumnSpecification work for you in the meantime? From: Chen Song [mailto:chens_alb...@yahoo.com] Sent: Monday, October 31, 2011 9:15 AM To: hive dev list; hive user list Subject: pass entire

RE: High number of input files problems

2011-11-01 Thread Steven Wong
I suspect very few people are still using Hive 0.6 or older. Try upgrading. From: Florin Diaconeasa [mailto:florin.diacone...@gmail.com] Sent: Monday, October 31, 2011 6:37 AM To: user@hive.apache.org Subject: High number of input files problems Hello, Lately our user base has increased so the

Using the map data type

2011-11-01 Thread Jonathan Meed
Hi Hive Experts. I think I may have found a bug in hive and was wondering if there is a better workaround then the one I am using. When I run the script below all the values for the map are null, I.E. beacon.QS['cid'] is null. FROM (from beacons select TRANSFORM(ipaddress, ipaddress2, ident

Re: Handling hyphens in table/database/usernames

2011-11-01 Thread Sriram Krishnan
Excellent – that seems to work as I want. I tried every combination but the back tick :). Thanks again, Sriram From: Jander g mailto:jande...@gmail.com>> Reply-To: "user@hive.apache.org" mailto:user@hive.apache.org>> Date: Tue, 1 Nov 2011 00:01:59 -0700 To: "user@hi

hive.map.aggr

2011-11-01 Thread Steven Wong
I have a query doing JOIN and GROUP BY: SELECT ... FROM x JOIN y ON (...) GROUP BY ...; In the first MR job, the reduce phase performs hash aggregation when hive.map.aggr=true, but the reduce phase doesn't perform hash aggregation when hive.map.aggr=false. Why does hive.map.aggr affect the redu

Re: Handling hyphens in table/database/usernames

2011-11-01 Thread Jander g
OK, I see. You can use "grant all on database default to user `sri-krish`;" to finish your work. On Tue, Nov 1, 2011 at 2:15 PM, Sriram Krishnan wrote: > Thanks Jander. > > Not supporting "-" in database/table names is fine, we can work around > that. What about usernames? Like I said, our main