[ https://issues.apache.org/jira/browse/HIVE-18192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16364353#comment-16364353 ]
Sankar Hariappan edited comment on HIVE-18192 at 2/14/18 4:22 PM: ------------------------------------------------------------------ Added 12.patch with below updates # Scripts to add Metastore tables for write id management. Add for other non-derby databases too. # CompactionTxnHandler cleanup COMPLETED_TXN_COMPONENTS based on highest write id instead of highest txn id. # ImportSemanticAnalyzer to use write id instead of txn id to import data files. Out of scope of HIVE-18192: Will raise separate tickets for each of this items. # Cleaner for TXNS_TO_WRITE_ID table entries. Also, need to maintain the LWM for each table write id. # Update classes to use WriteID instead of TxnId as methods and variable names. # HiveEndPoint: Opening txns and getting writeids should be 1 metastore call not 2 for performance. # Need to change transactonid to writerid in RecordIdentifier.Field.transactionId. # Exchange partition doesn't work as write id not valid across tables. # Explain plan allocate write but don't use it and never remove the entry from meta tables. # Failure of TestAcidOnTez.testGetSplitsLocks. was (Author: sankarh): Added 12.patch with below updates # Scripts to add Metastore tables for write id management. Add for other non-derby databases too. # CompactionTxnHandler cleanup COMPLETED_TXN_COMPONENTS based on highest write id instead of highest txn id. Out of scope of HIVE-18192: Will raise separate tickets for each of this items. # Cleaner for TXNS_TO_WRITE_ID table entries. Also, need to maintain the LWM for each table write id. # Update classes to use WriteID instead of TxnId as methods and variable names. # HiveEndPoint: Opening txns and getting writeids should be 1 metastore call not 2 for performance. # Need to change transactonid to writerid in RecordIdentifier.Field.transactionId. # Exchange partition doesn't work as write id not valid across tables. # Explain plan allocate write but don't use it and never remove the entry from meta tables. # Failure of TestAcidOnTez.testGetSplitsLocks. > Introduce WriteID per table rather than using global transaction ID > ------------------------------------------------------------------- > > Key: HIVE-18192 > URL: https://issues.apache.org/jira/browse/HIVE-18192 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, Transactions > Affects Versions: 3.0.0 > Reporter: anishek > Assignee: Sankar Hariappan > Priority: Major > Labels: ACID, DR, pull-request-available > Fix For: 3.0.0 > > Attachments: HIVE-18192.01.patch, HIVE-18192.02.patch, > HIVE-18192.03.patch, HIVE-18192.04.patch, HIVE-18192.05.patch, > HIVE-18192.06.patch, HIVE-18192.07.patch, HIVE-18192.08.patch, > HIVE-18192.09.patch, HIVE-18192.10.patch, HIVE-18192.11.patch, > HIVE-18192.12.patch > > > To support ACID replication, we will be introducing a per table write Id > which will replace the transaction id in the primary key for each row in a > ACID table. > The current primary key is determined via > <original transaction id, bucketid , rowid> > which will move to > <original write id, bucketid, rowid> > For each table modified by the given transaction will have a table level > write ID allocated and a persisted map of global txn id -> to table -> write > id for that table has to be maintained to allow Snapshot isolation. > Readers should use the combination of ValidTxnList and > ValidWriteIdList(Table) for snapshot isolation. > > [Hive Replication - ACID > Tables.pdf|https://issues.apache.org/jira/secure/attachment/12903157/Hive%20Replication-%20ACID%20Tables.pdf] > has a section "Per Table Sequences (Write-Id)" with more detials -- This message was sent by Atlassian JIRA (v7.6.3#76005)