[ https://issues.apache.org/jira/browse/HIVE-14841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15668011#comment-15668011 ]
Sergey Shelukhin commented on HIVE-14841: ----------------------------------------- Is it possible to do work in the branch? This causes immense conflicts with hive-14535 branch, and I see tons of comments that purport with FIXMEs and stuff to move code around and refactor this and that. I think this should be done on the branch and merged once when ready, so that conflicts with parallel changes to the code affected by the moves are minimized. > Replication - Phase 2 > --------------------- > > Key: HIVE-14841 > URL: https://issues.apache.org/jira/browse/HIVE-14841 > Project: Hive > Issue Type: New Feature > Components: repl > Affects Versions: 2.1.0 > Reporter: Sushanth Sowmyan > Assignee: Sushanth Sowmyan > > Per email sent out to the dev list, the current implementation of replication > in hive has certain drawbacks, for instance : > * Replication follows a rubberbanding pattern, wherein different tables/ptns > can be in a different/mixed state on the destination, so that unless all > events are caught up on, we do not have an equivalent warehouse. Thus, this > only satisfies DR cases, not load balancing usecases, and the secondary > warehouse is really only seen as a backup, rather than as a live warehouse > that trails the primary. > * The base implementation is a naive implementation, and has several > performance problems, including a large amount of duplication of data for > subsequent events, as mentioned in HIVE-13348, having to copy out entire > partitions/tables when just a delta of files might be sufficient/etc. Also, > using EXPORT/IMPORT allows us a simple implementation, but at the cost of > tons of temporary space, much of which is not actually applied at the > destination. > Thus, to track this, we now create a new branch (repl2) and a uber-jira(this > one) to track experimental development towards improvement of this situation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)