Hi Peter,
Thanks a lot for the breakdown, it all makes sense.
Unfortunately I work with companies who are stuck with older versions of
Hive, so I'm trying to find some workarounds.
I was actually able to make it mostly work. Here's what I do:
- In configureJobConf():
- Create a work di
Hi Julien,
With Iceberg we went through the same process, so here is some our findings:
- Writers are running on the executors (LLAP or MR nodes).
- OutputCommitter taskCommit runs on the same executors
- We did some experimenting but found that even when we were able to call the
OutputCommitter
