David Hodeffi created SPARK-18096:
-------------------------------------
Summary: Spark on have - 'Update' save mode
Key: SPARK-18096
URL: https://issues.apache.org/jira/browse/SPARK-18096
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 2.0.1
Reporter: David Hodeffi
when creating ETL with Spark on Hive, it is needed to update incrementally the
destination table.
In case it is partitioned table it means that we don't need to update all
partitions, but just the one who mutated.
right now there is only one way to update a Dataframe which is
SaveMode.Overwrite , the problem is that when doing it incrementally you don't
need to update all partitions but just those who changed/updated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
