Thanks Roberto. Will try that out. regards Sunita
On Thu, Jan 30, 2014 at 10:14 AM, Roberto Congiu <roberto.con...@openx.com>wrote: > Hi Sunita, > yes, it's definitely possible and you should use Generic UDFs. > I wrote one UDF that takes n arrays (each one with the same number of > elements) and returns an array of structs which is usually used in a > lateral view. > > A good article on how to write a generic UDF is this one: > http://www.baynote.com/2012/11/a-word-from-the-engineers/ > > > On Thu, Jan 30, 2014 at 7:06 AM, Sunita Arvind <sunitarv...@gmail.com>wrote: > >> Can someone please suggest if this is doable or not? Is generic udf the >> only option? How would using generic vs simple udf make any difference >> since I would be returning the same object either ways. >> >> Thank you >> Sunita >> >> ---------- Forwarded message ---------- >> From: *Sunita Arvind* <sunitarv...@gmail.com> >> Date: Wednesday, January 29, 2014 >> Subject: Simple UDF to return array >> To: "user@hive.apache.org" <user@hive.apache.org> >> >> >> Hello Experts, >> >> I am trying to write a UDF to parse a logline and provide the output in >> the form of an array. Basically I want to be able to use LATERAL VIEW >> explode subsequently to make it into columns. >> >> This is how a typical log entry looks: >> >> 24-JUN-2012 05:00:42 * >> (CONNECT_DATA=(SERVICE_NAME=abcd.efg.hij.com)(failover_mode=(type=select)(method=basic))(CID=(PROGRAM=sqlplus)(HOST=xyz)(USER=u1))(SERVER=dedicated)(INSTANCE_NAME=aaa)) >> * (ADDRESS=(PROTOCOL=tcp)(HOST=9.9.9.9)(PORT=60000)) * establish * >> abcd.efg.hij.com * 0 >> >> Attached is my LogParser class which is basically the UDF. Excerpts below: >> >> class LogParser extends UDF { >> int current_index=0; >> >> ArrayList<String> record= new ArrayList<>(); >> public ArrayList<String> evaluate(Text input) { >> ...... >> String logdate = null; >> ... >> logdate = getDate(line); >> record.add(logdate); >> return record; >> >> >> I've tried changing the return type to ArrayList<Text>, Object etc.I just >> get an error like this when I try to use the UDF: >> >> select explode(strparse(record)) as newcols from logdump limit 1; >> >> OK converting to local hdfs://tlbd-ns/user/TestUser1/LogParserStrArr.jar >> Added >> /tmp/3c583384-0592-41a3-ad9e-b12d2207df7b_resources/LogParserStrArr.jar to >> class path Added resource: >> /tmp/3c583384-0592-41a3-ad9e-b12d2207df7b_resources/LogParserStrArr.jar OK >> FAILED: UDFArgumentException explode() takes an array or a map as a >> parameter >> >> I tried cast to array and that fails as well. >> >> Requesting help from the community. I am considering writing generic UDF, >> but this is a simple requirement and would like to be able to use simple >> UDF if I can. >> >> regards >> Sunita >> >> >> > > > -- > ---------------------------------------------------------- > Good judgement comes with experience. > Experience comes with bad judgement. > ---------------------------------------------------------- > Roberto Congiu - Data Engineer - OpenX > tel: +1 626 466 1141 >
