RE: Detecting write mode (append, overwrite) in custom storage handler

2022-04-29 Thread Julien Phalip
I've noticed that the DefaultHiveMetaHook class has a commitInsertTable(Table table, boolean overwrite) method. So perhaps this is the method tha

Detecting write mode (append, overwrite) in custom storage handler

2022-04-29 Thread Julien Phalip
Hi, I'm working on a custom storage handler and am wondering if there's a way to detect the appropriate write mode for the output table. For example, an "INSERT" statement is expected to append rows to the table, whereas an "INSERT OVERWRITE" statement should first clear the table before adding ne

RE: Re: Custom OutputCommitter not called by Tez

2022-04-29 Thread Julien Phalip
Hi Peter, Looking at https://issues.apache.org/jira/browse/TEZ-4279, it seems that the fix might have been applied to 0.9.3. Is that correct? If so, do you think that just upgrading Tez to that version might be enough to allow the "setUpJob()", "commitJob()" and "abortJob()" to be called appropria