Hi Julien, Hive 3.1.2 is dependent on 0.9 Tez, and I seem to remember having issues running Hive 3.1.2 with Tez 0.10. OTOH you might get away with patching 0.9 Tez with the appropriate changes. I would ask this on the Tez mailing list.
Are you trying out Hive-Iceberg integration, or it is another custom SerDe? Thanks, Peter > On 2022. Apr 27., at 19:12, Julien Phalip <jpha...@gmail.com> wrote: > > Thanks Peter. > > By chance could I get things to work by keeping my current version of Hive > (3.1.2) and only upgrading Tez? Which version(s) should I use? > > Thank you, > > Julien > > On 2022/04/27 08:59:08 Peter Vary wrote: > > We had the same issue with the IcebergOutputCommitter. > > > > The first solution was this: > > https://issues.apache.org/jira/browse/HIVE-25006 > > <https://issues.apache.org/jira/browse/HIVE-25006> > > <https://issues.apache.org/jira/browse/HIVE-25006 > > <https://issues.apache.org/jira/browse/HIVE-25006>> > > It needed https://issues.apache.org/jira/browse/TEZ-4279 > > <https://issues.apache.org/jira/browse/TEZ-4279> > > <https://issues.apache.org/jira/browse/TEZ-4279 > > <https://issues.apache.org/jira/browse/TEZ-4279>> > > > > Later we ended up with this final solution: > > https://issues.apache.org/jira/browse/HIVE-25208 > > <https://issues.apache.org/jira/browse/HIVE-25208> > > <https://issues.apache.org/jira/browse/HIVE-25208 > > <https://issues.apache.org/jira/browse/HIVE-25208>> > > > > I hope this helps, > > Peter > > > > > On 2022. Apr 27., at 1:46, Julien Phalip <jp...@gmail.com > > > <mailto:jp...@gmail.com>> wrote: > > > > > > Hi, > > > > > > I'm working on a custom storage handler. My custom output committer class > > > gets called normally when using the "mr" engine. However, it seems to be > > > entirely ignored when using the "tez" engine. > > > > > > I'm setting the JobConf's "mapred.output.committer.class" key to my > > > fully-qualified output committer class name in the handler's > > > configureJobConf() method. I've also tried the > > > "hive.tez.mapreduce.output.committer.class" key and also tried setting > > > those keys in the job properties in the configureOutputJobProperties() > > > method. But that didn't work either. > > > > > > By the way, I'm using Hive 3.1.2 and Tez 0.9.1. > > > > > > Do you know what I might be missing or doing wrong? > > > > > > Thanks, > > > > > > Julien > > > >
