RE: RE: Re: Custom OutputCommitter not called by Tez

2022-04-30 Thread Julien Phalip
After all, I was able to have my MetaHook class' commitInsertTable() method be properly called by Tez. However, it looks like it's in fact a different instance of that class, and therefore it doesn't share the same Configuration object as the one that was initialized at the beginning of the job. So

RE: RE: Detecting write mode (append, overwrite) in custom storage handler

2022-04-30 Thread Julien Phalip
I realize this is in fact quite related to another thread that I recently started: https://lists.apache.org/thread/s0pzmgmq6trdjtxc50qwpww2dlzxql9b So this discussion could continue there. On 2022/04/30 04:08:27 Julien Phalip wrote: > I've noticed that the DefaultHiveMetaHook > < https://github.co

RE: RE: Re: Custom OutputCommitter not called by Tez

2022-04-30 Thread Julien Phalip
Hi Peter, So I've looked into the approach that you've pointed at in this pull request (https://github.com/apache/hive/pull/2161), which is to rely on HiveMetaHook.commitInsertTable() instead of the "traditional" OutputCommitter.commitJob(). I've tried to implement a similar approach, however som