Hi Hive users, I thought I'll update the status on this. I could spend some time on cracking hive code, map field related in particular,
1. Explode works great and it solves a very needed problem for me. 2. There is already a str_to_map udf available in cdh3u2. I stumbled upon this while I was looking to create a udf 3. Here comes the awesome part. I finally created a udf to convert a map to string. I would like to contribute it to the hive community. Let me know how I can do that. 4. Also there are map_keys and map_values functions which would give a list of arrays of keys and values respectively. Let me know if you have any questions. Thanks, Chalcy From: Chalcy Raja [mailto:chalcy.r...@careerbuilder.com] Sent: Wednesday, November 23, 2011 1:20 PM To: user@hive.apache.org; 'Bejoy Ks' Subject: RE: hive map field question Wow, this is great news. I read about explode and wished map had that. Thanks so much, Chalcy From: Bejoy Ks [mailto:bejoy...@yahoo.com] Sent: Wednesday, November 23, 2011 1:05 PM To: user@hive.apache.org Subject: Re: hive map field question Hi Chalcy Support for splitting a column of map data type into key and values is available on the hive latest trunk. You can use a explode() UDF to achieve the same, which used to support only arrays previously. http://svn.apache.org/viewvc/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFExplode.java?view=markup You need to patch your hive source installation to get this functionality running. Or as a quick fix (not highly recommended) take the source code include the same as a Custom UDF for your temporary usage. You can utilize the explode functionality as follows combining the same with Lateral Views. Sample DDL CREATE TABLE test_map ( clmn1 STRING, clmn2 STRING, clmnMap MAP<STRING,STRING> ) Sample DML - to extraxt key values from map SELECT clmn1,clmn2,clmnMap_key,clmnMap_value FROM test_map LATERAL VIEW explode(clmnMap) cl AS clmnMap_key,clmnMap_value; Hope it helps!... Regards Bejoy.K.S ________________________________ From: Mark Grover <mgro...@oanda.com> To: user@hive.apache.org Sent: Wednesday, November 23, 2011 9:55 PM Subject: Re: hive map field question Chalcy, I don't know of an existing UDF that lists all keys and values of a map. You may want to check ql/src/java/org/apache/hadoop/hive/ql/udf in Hive source to double check. But, as you might have guessed, it'd be pretty straightforward to do that in your own UDF:-) Mark ----- Original Message ----- From: "Chalcy Raja" <chalcy.r...@careerbuilder.com<mailto:chalcy.r...@careerbuilder.com>> To: "user@hive.apache.org<mailto:user@hive.apache.org>" <user@hive.apache.org<mailto:user@hive.apache.org>> Sent: Wednesday, November 23, 2011 11:14:09 AM Subject: RE: hive map field question Thanks, Mark. I'll try that. Also, is there a way to list all the keys and values in a map field? Thanks, Chalcy -----Original Message----- From: Mark Grover [mailto:mgro...@oanda.com<mailto:mgro...@oanda.com>] Sent: Wednesday, November 23, 2011 11:01 AM To: user@hive.apache.org<mailto:user@hive.apache.org> Subject: Re: hive map field question Hi Chalcy, You could write a UDF that takes in a Map object and returns you the string you need. Alternatively, you could use Hive Transform. Details here:https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Transform When a map gets passed to your custom transform script, you could read it as a serialized string. Mark ----- Original Message ----- From: "Chalcy Raja" <chalcy.r...@careerbuilder.com<mailto:chalcy.r...@careerbuilder.com>> To: user@hive.apache.org<mailto:user@hive.apache.org> Sent: Wednesday, November 23, 2011 10:48:07 AM Subject: hive map field question Hello HiveUsers, I have a need to convert a map<string,string> field to string field and vice versa in a hive table. I could not do cast. I created two external tables with one has string and another map. I can join both to get what I want, takes a long time. Any ideas of how it can be done efficiently ? Thanks, Chalcy
