Related to this, there exists an API in Hive to simplify the integrations of other frameworks with Hive's ACID feature:
See: https://cwiki.apache.org/confluence/display/Hive/HCatalog+Streaming+Mutation+API It contains code for maintaining heartbeats, handling locks and transactions, and submitting mutations in a distributed environment. We have used it to write to transactional tables from Cascading based processes. Elliot. On 2 June 2016 at 09:54, Mich Talebzadeh <mich.talebza...@gmail.com> wrote: > > Hi, > > Spark does not support transactions because as I understand there is a > piece in the execution side that needs to send heartbeats to Hive metastore > saying a transaction is still alive". That has not been implemented in > Spark yet to my knowledge." > > Any idea on the timelines when we are going to have support for > transactions in Spark for Hive ORC tables. This will really be useful. > > > Thanks, > > > Dr Mich Talebzadeh > > > > LinkedIn * > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* > > > > http://talebzadehmich.wordpress.com > > >
