Re: ACID transactions on data added from Spark not working

2016-09-14 Thread Mich Talebzadeh
Hi, I believe this is an issue with Spark handing transactional tables in Hive. When you add rows from Spark to ORC transactional table, Hive metadata tables HIVE_LOCKS and TXNS tables are not updated. This does not happen with Hive itself. As a result these new rows are left in an inconsistent s

ACID transactions on data added from Spark not working

2016-09-14 Thread Jack Wenger
Hi there, I'm trying to use ACID transactions in Hive but I have a problem when the data are added with Spark. First, I created a table with the following statement : __ CREATE TABLE testdb.test(id string, col1 s