RE: MapredContext not available when tez enabled

[Frank Luo]

Gopal, 

Here is basically my code and I can clearly see configure() was not called and 
JavaCode on GenericUDF#configure reads:  "This is only called in runtime of 
MapRedTask.". Also based on my observation, the query is not executed as a M/R 
because Yarn monitoring knows nothing about the job. Seems to be the query is 
executed locally (forgive me for knowing very little on how Tez works 
internally).

=========================================== 

public class GenericUDFTracker extends GenericUDF {
    private MapredContext context;

    @Override
    public void configure(MapredContext aContext) {
        super.configure(aContext);
        context = aContext;
        LOGGER.info("configure setting context:{}", context); //this never 
executed
    }

    @Override
    public Object evaluate(DeferredObject[] arguments)
        throws HiveException {
        LOGGER.info(ArrayUtils.toString(arguments));

        String arg0 = elementOI0.getPrimitiveJavaObject(arguments[0].get());
        String arg1 = elementOI1.getPrimitiveJavaObject(arguments[1].get());
        LOGGER.info("context = {}", context);  //it prints 'context = null'
        Reporter reporter = context.getReporter(); //NPE is thrown
        LOGGER.info("reporter = {}", reporter);
       ...
  }
}

-----Original Message-----
From: Gopal Vijayaraghavan [mailto:go...@hortonworks.com] On Behalf Of Gopal 
Vijayaraghavan
Sent: Tuesday, April 21, 2015 11:41 PM
To: user@hive.apache.org
Subject: Re: MapredContext not available when tez enabled


 
> A bit digging shows that GenericUDF#configure method was not called. 
>So in this case, is it possible to get counters through other means, or 
>we have to implement Counter concept ourselves?

You should be getting a TezContext object there (which inherits from 
MapRedContext).

And the method should get called depending on a needConfigure() check - if it 
is not getting called, that is very strange.

Cheers,
Gopal