Eric Liang created SPARK-19183:
----------------------------------
Summary: Add deleteWithJob hook to internal commit protocol API
Key: SPARK-19183
URL: https://issues.apache.org/jira/browse/SPARK-19183
Project: Spark
Issue Type: Improvement
Components: SQL
Reporter: Eric Liang
Currently in SQL we implement overwrites by calling fs.delete() directly on the
original data. This is not ideal since we the original files end up deleted
even if the job aborts. We should extend the commit protocol to allow file
overwrites to be managed as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
