Abhishek,
The code doesn't seem to be complete.
Look at
https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDAFPercentile.javafor
reference. It has two terminate()'s - one for UDAF and one for the
Evaluator.
Do you mind posting your complete code on github somewhere so it's easier
to analyze?
Mark
On Fri, Feb 8, 2013 at 2:05 PM, Abhishek Bhattacharya <abhat...@fiu.edu>wrote:
> Hi,
>
> I have implemented a simple UDAF for top-n-percent as follows:
> import java.util.ArrayList;
> import java.util.Collections;
>
> import org.apache.hadoop.hive.ql.exec.UDAF;
> import org.apache.hadoop.hive.ql.exec.UDAFEvaluator;
>
> public class UDAFTopNPercent extends UDAF{
>
> public static class Result {
> ArrayList<Double> list;
> double min;
> }
>
> public class TopNPercentEvaluator implements UDAFEvaluator {
>
> private Result res;
> private int rowIndex;
> private int percent;
>
> public TopNPercentEvaluator() {
> super();
> res = new Result();
> init();
> rowIndex = 0;
> }
> @Override
> public void init() {
> res.list = new ArrayList<Double>();
> res.min = Double.MAX_VALUE;
> }
>
> public boolean iterate(Double rowVal, int pct) {
> ArrayList<Double> resList = res.list;
> rowIndex++;
> resList.add(rowVal);
> percent = pct;
> return true;
> }
>
> public ArrayList<Double> terminatePartial() {
> ArrayList<Double> resList = res.list;
> Collections.sort(resList);
> return resList;
> }
>
> public boolean merge(ArrayList<Double> otherList) {
> ArrayList<Double> resList = res.list;
> resList.addAll(otherList);
> return true;
> }
>
> public ArrayList<Double> terminate() {
> ArrayList<Double> resList = res.list;
> double num_rows = (double)percent/100.0*rowIndex;
> Collections.sort(resList);
> int lastIdx = resList.size()- (int) num_rows;
> if(lastIdx <= 0) {
> return resList;
> }
> for(int i=0; i<lastIdx; i++) {
> resList.remove(i);
> }
> return resList;
> }
> }
>
> /**
> * @param args
> */
> public static void main(String[] args) {
> // TODO Auto-generated method stub
>
> }
>
> }
>
> But throws some error such as first few lines are:
> FAILED: Hive Internal Error:
> java.lang.ClassCastException(org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableFloatObjectInspector
> cannot be cast to
> org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector)
> java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableFloatObjectInspector
> cannot be cast to
> org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters.getConverter(ObjectInspectorConverters.java:116)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFUtils$ConversionHelper.<init>(GenericUDFUtils.java:300)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFBridge$GenericUDAFBridgeEvaluator.init(GenericUDAFBridge.java:129)
>
> Please help me to debug this!
> Is it throwing from returning ArrayList<Double> in terminate()?
> How should I return a List from UDAF?
>
> Thanks,
> Abhishek
>
