Thanks Peter. By chance could I get things to work by keeping my current version of Hive (3.1.2) and only upgrading Tez? Which version(s) should I use?
Thank you, Julien On 2022/04/27 08:59:08 Peter Vary wrote: > We had the same issue with the IcebergOutputCommitter. > > The first solution was this: https://issues.apache.org/jira/browse/HIVE-25006 < https://issues.apache.org/jira/browse/HIVE-25006> > It needed https://issues.apache.org/jira/browse/TEZ-4279 < https://issues.apache.org/jira/browse/TEZ-4279> > > Later we ended up with this final solution: https://issues.apache.org/jira/browse/HIVE-25208 < https://issues.apache.org/jira/browse/HIVE-25208> > > I hope this helps, > Peter > > > On 2022. Apr 27., at 1:46, Julien Phalip <jp...@gmail.com> wrote: > > > > Hi, > > > > I'm working on a custom storage handler. My custom output committer class gets called normally when using the "mr" engine. However, it seems to be entirely ignored when using the "tez" engine. > > > > I'm setting the JobConf's "mapred.output.committer.class" key to my fully-qualified output committer class name in the handler's configureJobConf() method. I've also tried the "hive.tez.mapreduce.output.committer.class" key and also tried setting those keys in the job properties in the configureOutputJobProperties() method. But that didn't work either. > > > > By the way, I'm using Hive 3.1.2 and Tez 0.9.1. > > > > Do you know what I might be missing or doing wrong? > > > > Thanks, > > > > Julien > >
