[
https://issues.apache.org/jira/browse/KUDU-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17209353#comment-17209353
]
ASF subversion and git services commented on KUDU-2612:
-------------------------------------------------------
Commit 90aa4fa7d1527f376803440a4642668e3d798748 in kudu's branch
refs/heads/master from Andrew Wong
[ https://gitbox.apache.org/repos/asf?p=kudu.git;h=90aa4fa ]
KUDU-2612 p11: persist txn metadata in superblock
Currently, transaction metadata is entirely stored in the WALs; once a
transaction is begun, it anchors WALs indefinitely. This is untenable,
as any tablet that participates in a transaction would never be able to
GC its WALs.
This patch addresses this by storing transaction metadata in the tablet
superblock for the following participant ops:
- BEGIN_TXN: an empty record is added to the superblock for the given
transaction ID. In the future, this can be used to store the owner of
the transaction.
- FINALIZE_COMMIT: the commit timestamp is added to the superblock for
the given transaction ID. The commit timestamp of a transaction will
be used when scanning over mutations applied to a tablet instead of
the apply timestamp.
- ABORT_TXN: an abort bit is added to the superblock for the given
transaction ID.
Upon applying each of these ops, the op's OpId is anchored in the WALs
until the tablet metadata is successfully flushed, ensuring that if we
don't flush the tablet metadata before restarting, we can rebuild any
ops that whose mutations had not been persisted by replaying the WALs.
What about BEGIN_COMMIT ops? While these ops will still need to be
anchored, BEGIN_COMMIT ops don't need to persist any long-term
information. As such, their anchoring is slightly different -- rather
than mutating the tablet metadata and unanchoring on a metadata flush,
BEGIN_COMMIT ops will update the in-memory state for the in-flight Txn,
and unanchor once the corresponding FINALIZE_COMMIT or ABORT_TXN begins
applying.
On bootstrap, the following rules are applied:
- If we've persisted terminal information (i.e. abort or commit) about a
transaction ID, we can skip replaying any participant op for that
transaction.
- If we otherwise have a persisted record for the transaction ID, we
should start an open transaction and replay all participant ops for
that transaction.
- If we have no record of a transaction persisted, we must replay all
participant ops for the given transaction.
While storing things in the superblock is not ideal, there are further
improvements that can be made to reduce its size, e.g. after finalizing
a transaction, the transactions mutations may be merged in with the rest
of the tablet; once all mutations of a given transaction have been
merged, the transaction metadata may be removed from the metadata.
Change-Id: I2f32808aef10484f4e0ad3942bb005f61fbdb34a
Reviewed-on: http://gerrit.cloudera.org:8080/16492
Tested-by: Andrew Wong <aw...@cloudera.com>
Reviewed-by: Alexey Serbin <aser...@cloudera.com>
> Implement multi-row transactions
> --------------------------------
>
> Key: KUDU-2612
> URL: https://issues.apache.org/jira/browse/KUDU-2612
> Project: Kudu
> Issue Type: Task
> Reporter: Mike Percy
> Priority: Major
> Labels: roadmap-candidate
>
> Tracking Jira to implement multi-row / multi-table transactions in Kudu.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
