Re: HIVE UDF Error

2014-04-08 Thread Rishabh Bhardwaj
I added, package rishabh.udf.hive; in the above code. and repeated the steps. But Now getting the following error, hive> create temporary function helloworld as 'rishabh.udf.hive.SimpleUDFExample'; FAILED: Class rishabh.udf.hive.SimpleUDFExample not found FAILED: Execution Error, return code 1 f

Re: HIVE UDF Error

2014-04-08 Thread Nitin Pawar
in your code and that code package is missing what you need to do is define package something like package org.apache.hadoop.hive.ql.udf; then your add function definition becomes CREATE TEMPORARY FUNCTION AS 'org.apache.hadoop.hive.ql.udf.'; feel free to use any package name you wish but mak

Re: HIVE UDF Error

2014-04-08 Thread Rishabh Bhardwaj
Hi Nitin, Thanks for the concern. Here is the code of the UDF, import org.apache.hadoop.hive.ql.exec.Description; import org.apache.hadoop.hive.ql.exec.UDF; import org.apache.hadoop.io.Text; @Description(   name="SimpleUDFExample",   value="returns 'hello x', where x is whatever you give it (STRI

Re: HIVE UDF Error

2014-04-08 Thread Nitin Pawar
Can you put first few lines of your code here or upload code on github and share the link? On Wed, Apr 9, 2014 at 11:59 AM, Rishabh Bhardwaj wrote: > Hi all, > I have done the following steps to create a UDF in hive but getting > error.Please help me. > 1. Created the udf as described > here<

HIVE UDF Error

2014-04-08 Thread Rishabh Bhardwaj
Hi all, I have done the following steps to create a UDF in hive but getting error.Please help me. 1. Created the udf as described here. 2. Compiled it successfully. 3. Copy the class file to a directory hiveudfs. 4. Added it to a jar with this command: jar -cf hiveudfs.jar hiveudfs/SimpleUDFExamp

Compressed Data Column in Hive Table

2014-04-08 Thread Kanna Karanam
Hi - One of the columns in my hive table contains compressed data (Json document in Gzip format). Is there any recommended way to decompress it and explode the json doc? Thanks, Kanna

Re: Push down hive filters to hbase store with composite rowkey ..

2014-04-08 Thread Gautam
Any thoughts on this? Am I going down the right track? Is there an easier workaround? -Gautam. On Mon, Apr 7, 2014 at 11:29 PM, Gautam wrote: > Hello All, > *Short version*: Looking to leverage hbase fuzzy row scans > on fields in a composite rowkey ( on hbase 0.94.6 ). > > *Lo

Question on converting an array of structs into a map

2014-04-08 Thread Sunderlin, Mark
I have data in two tables that looks like this: Table_1 Sequence_numint, User_id string Table_2 Sequence_numint User_attributes array> Sequence number joins the two and it is a one to one and only one relationship. Where I want to end up: A singl

Re: get_json_object for nested field returning a String instead of an Array

2014-04-08 Thread David Quigley
Hi Narayanan, We have had some success with a similar use case using a custom input format / record reader to recursively split arbitrary json into a set of discreet records at runtime. No schema is needed. Doing something similar might give you the functionality you are looking for. https://githu

Re: get_json_object for nested field returning a String instead of an Array

2014-04-08 Thread Narayanan K
Thanks Yong! On Mon, Apr 7, 2014 at 5:07 PM, java8964 wrote: > Hi, Narayanan: > > The current problem is that for a generic solution, there is no way that we > know that element in the Json is an array. Keep in mind that in any element > of Json, it could be any valid structure. So it could be ar

Re: Deserializing into multiple records

2014-04-08 Thread David Quigley
The nested json was a hard requirement On Tue, Apr 8, 2014 at 5:52 AM, Edward Capriolo wrote: > Use avro or protobuf support. > > > On Tuesday, April 8, 2014, Petter von Dolwitz (Hem) < > petter.von.dolw...@gmail.com> wrote: > > Good stuff! > > > > I am glad that I could help. > > > > Br, > > Pe

Re: Deserializing into multiple records

2014-04-08 Thread Edward Capriolo
Use avro or protobuf support. On Tuesday, April 8, 2014, Petter von Dolwitz (Hem) < petter.von.dolw...@gmail.com> wrote: > Good stuff! > > I am glad that I could help. > > Br, > Petter > > > 2014-04-04 6:02 GMT+02:00 David Quigley : >> >> Thanks again Petter, the custom input format was exactly wh

Re: Deserializing into multiple records

2014-04-08 Thread Petter von Dolwitz (Hem)
Good stuff! I am glad that I could help. Br, Petter 2014-04-04 6:02 GMT+02:00 David Quigley : > Thanks again Petter, the custom input format was exactly what I needed. > > Here is example of my code in case anyone is interested > https://github.com/quicklyNotQuigley/nest > > Basically gives yo