Hello Experts, I am trying to write a UDF to parse a logline and provide the output in the form of an array. Basically I want to be able to use LATERAL VIEW explode subsequently to make it into columns.
This is how a typical log entry looks:
24-JUN-2012 05:00:42 *
(CONNECT_DATA=(SERVICE_NAME=abcd.efg.hij.com)(failover_mode=(type=select)(method=basic))(CID=(PROGRAM=sqlplus)(HOST=xyz)(USER=u1))(SERVER=dedicated)(INSTANCE_NAME=aaa))
* (ADDRESS=(PROTOCOL=tcp)(HOST=9.9.9.9)(PORT=60000)) * establish *
abcd.efg.hij.com * 0
Attached is my LogParser class which is basically the UDF. Excerpts below:
class LogParser extends UDF {
int current_index=0;
ArrayList<String> record= new ArrayList<>();
public ArrayList<String> evaluate(Text input) {
......
String logdate = null;
...
logdate = getDate(line);
record.add(logdate);
return record;
I've tried changing the return type to ArrayList<Text>, Object etc.I just
get an error like this when I try to use the UDF:
select explode(strparse(record)) as newcols from logdump limit 1;
OK converting to local hdfs://tlbd-ns/user/TestUser1/LogParserStrArr.jar
Added
/tmp/3c583384-0592-41a3-ad9e-b12d2207df7b_resources/LogParserStrArr.jar to
class path Added resource:
/tmp/3c583384-0592-41a3-ad9e-b12d2207df7b_resources/LogParserStrArr.jar OK
FAILED: UDFArgumentException explode() takes an array or a map as a
parameter
I tried cast to array and that fails as well.
Requesting help from the community. I am considering writing generic UDF,
but this is a simple requirement and would like to be able to use simple
UDF if I can.
regards
Sunita
LogParser.java
Description: Binary data
