We had the same issue with the IcebergOutputCommitter. The first solution was this: https://issues.apache.org/jira/browse/HIVE-25006 <https://issues.apache.org/jira/browse/HIVE-25006> It needed https://issues.apache.org/jira/browse/TEZ-4279 <https://issues.apache.org/jira/browse/TEZ-4279>
Later we ended up with this final solution: https://issues.apache.org/jira/browse/HIVE-25208 <https://issues.apache.org/jira/browse/HIVE-25208> I hope this helps, Peter > On 2022. Apr 27., at 1:46, Julien Phalip <jpha...@gmail.com> wrote: > > Hi, > > I'm working on a custom storage handler. My custom output committer class > gets called normally when using the "mr" engine. However, it seems to be > entirely ignored when using the "tez" engine. > > I'm setting the JobConf's "mapred.output.committer.class" key to my > fully-qualified output committer class name in the handler's > configureJobConf() method. I've also tried the > "hive.tez.mapreduce.output.committer.class" key and also tried setting those > keys in the job properties in the configureOutputJobProperties() method. But > that didn't work either. > > By the way, I'm using Hive 3.1.2 and Tez 0.9.1. > > Do you know what I might be missing or doing wrong? > > Thanks, > > Julien
