[
https://issues.apache.org/jira/browse/KUDU-3023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17015525#comment-17015525
]
ASF subversion and git services commented on KUDU-3023:
-------------------------------------------------------
Commit c235ba188d282ebba6c83e19a0aab07846d4b91a in kudu's branch
refs/heads/master from Alexey Serbin
[ https://gitbox.apache.org/repos/asf?p=kudu.git;h=c235ba1 ]
[tablet] KUDU-3023 validate RPC vs transaction size limit
Added a group validator to enforce consistency between the maximum RPC
size and the per-tablet transaction memory limit. This is to make it
possible to apply big enough write requests replicated by Raft (i.e.,
very close to the maximum allowed RPC size) by the local replica itself.
I also added a validator for the --tablet_transaction_memory_limit_mb
flag and updated a test that was using custom setting for the flag
to pass the newly added group flag validator.
Change-Id: Ia9384038a078027e997d7774b2be0e753c822fec
Reviewed-on: http://gerrit.cloudera.org:8080/15030
Reviewed-by: Adar Dembo <a...@cloudera.com>
Tested-by: Kudu Jenkins
> Enforce consistency between maximum RPC size and maximum size of transaction
> ----------------------------------------------------------------------------
>
> Key: KUDU-3023
> URL: https://issues.apache.org/jira/browse/KUDU-3023
> Project: Kudu
> Issue Type: Improvement
> Components: master, tserver
> Reporter: Alexey Serbin
> Assignee: Alexey Serbin
> Priority: Major
> Labels: Availability, consistency, scalability
>
> It seems that with the absence of constraints for the
> {{\-\-rpc_max_message_size}} and the {{\-\-tablet_transaction_memory}} flags,
> there might a transaction that can get into the WALs of tablet replicas, but
> no replica can apply the transaction.
> It would be nice to clarify on this and if necessary, establish a group
> validator to prevent such situations from happening, if possible.
> Few other points to clarify:
> * What if the data in RPC message that client sends to leader replica is
> compressed? Can we guarantee that corresponding Raft update sent to
> followers after persisting the data in the leader's WAL is compressed as well?
> * How much cruft is added by converting the incoming data into a WAL entry
> and then into a Raft update RPC message? Is it possible for a leader replica
> to accept an update on the tablet since it's under the max RPC size limit,
> but be unable push corresponding Raft update message to follower replicas
> because of some extra cruft added when converting the accepted
> {{WriteRequestPB}} into {{ConsensusRequestPB}} containing {{ReplicateMsg}}
> with embedded original {{WriteRequestPB}}?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
