Re: RE: Re: Custom OutputCommitter not called by Tez

2022-05-01 Thread Jennifer Chen
unsubscribe On Sat, Apr 30, 2022 at 9:10 PM Julien Phalip wrote: > After all, I was able to have my MetaHook class' commitInsertTable() > method be properly called by Tez. However, it looks like it's in fact a > different instance of that class, and therefore it doesn't share the same > Configur

RE: RE: Re: Custom OutputCommitter not called by Tez

2022-04-30 Thread Julien Phalip
After all, I was able to have my MetaHook class' commitInsertTable() method be properly called by Tez. However, it looks like it's in fact a different instance of that class, and therefore it doesn't share the same Configuration object as the one that was initialized at the beginning of the job. So

RE: RE: Re: Custom OutputCommitter not called by Tez

2022-04-30 Thread Julien Phalip
Hi Peter, So I've looked into the approach that you've pointed at in this pull request (https://github.com/apache/hive/pull/2161), which is to rely on HiveMetaHook.commitInsertTable() instead of the "traditional" OutputCommitter.commitJob(). I've tried to implement a similar approach, however som