Hi all:
Now I'm fixing PIG-4232. I want to ask question about UDFContext
I found when call UDFContext#getUDFProperties, in this function , it will put a
new entry which key is UDFContextKey( its class is c) and value is an empty
property when the specified property is not found (p= null).
public Properties getUDFProperties(Class c) {
UDFContextKey k = generateKey(c, null);
Properties p = udfConfs.get(k);
if (p == null) {
p = new Properties();
udfConfs.put(k, p);
}
return p;
}
UDFContext#setupUDFContext: before deserializing udfc, it first judges that
udfc.isUDFConfEmpty(). I think that this judgement will have problem when
UDFContext#getUDFProperties(Class c) is executed first in current thread, then
UDFContext#setupUDFContext(Configuration job) is executed. In this situation,
UDFContext#udfConfs only has an entry which value is an empty property but
udfc.IsUDFConfEmpty returns false.
public static void setupUDFContext(Configuration job) throws IOException {
UDFContext udfc = UDFContext.getUDFContext();
udfc.addJobConf(job);
// don't deserialize in front-end
if (udfc.isUDFConfEmpty()) {
udfc.deserialize();
}
}
public boolean isUDFConfEmpty() {
return udfConfs.isEmpty();
}
I think following way can solve the situation mentioned.
public boolean isUDFConfEmpty() {
- return udfConfs.isEmpty();
+ // return udfConfs.isEmpty();
+ if( udfConfs.isEmpty()){
+ return true;
+ }else{
+ boolean res = true;
+ for(UDFContextKey udfContextKey:udfConfs.keySet()){
+ if(!udfConfs.get(udfContextKey).isEmpty()){
+ res = false;
+ break;
+ }
+ }
+ return res;
+ }
+ }
Can anyone tell me in the situation I mentioned, is there any other way to
solve? Very thanks
Best Regards
Zhang,Liyun
