[
https://issues.apache.org/jira/browse/KUDU-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17884779#comment-17884779
]
ASF subversion and git services commented on KUDU-3619:
-------------------------------------------------------
Commit 05043e6aba6ab45c1b77de9f0762de3dfc5a54c0 in kudu's branch
refs/heads/branch-1.17.x from Alexey Serbin
[ https://gitbox.apache.org/repos/asf?p=kudu.git;h=05043e6ab ]
KUDU-3619 disable KUDU-3367 behavior by default
As it turned out, KUDU-3367 has introduced a regression due to
a deficiency in its implementation, where major compactions would fail
with errors like below if it had kicked in:
Corruption: Failed major delta compaction on RowSet(1): No min key found:
CFile base data in RowSet(1)
Since KUDU-3367 isn't quite relevant in Kudu versions of 1.12.0 and
newer when working with data that supports live row count (see
KUDU-1625), a quick-and-dirty fix is to set the default value for the
corresponding flag --all_delete_op_delta_file_cnt_for_compaction
to a value that effectively disables KUDU-3367 behavior.
This patch does exactly so.
Change-Id: Iec0719462e379b7a0fb05ca011bb9cdd991a58ef
Reviewed-on: http://gerrit.cloudera.org:8080/21848
Reviewed-by: KeDeng <[email protected]>
Tested-by: Alexey Serbin <[email protected]>
(cherry picked from commit 3666d2026d48adb5ff636321ef22320a8af5facb)
Conflicts:
src/kudu/tablet/delta_tracker.cc
Reviewed-on: http://gerrit.cloudera.org:8080/21855
Reviewed-by: Abhishek Chennaka <[email protected]>
> The 'supplement to GC algorithm' breaks major delta compaction
> --------------------------------------------------------------
>
> Key: KUDU-3619
> URL: https://issues.apache.org/jira/browse/KUDU-3619
> Project: Kudu
> Issue Type: Bug
> Components: compaction, tserver
> Affects Versions: 1.17.0
> Reporter: Alexey Serbin
> Priority: Major
>
> With the functionality introduced with
> [ad920e69f|https://github.com/apache/kudu/commit/ad920e69fcd67ceefa25ea81a38a10a27d9e3afc]
> doesn't handle the appearance of an empty rowset as the result of major
> delta compaction scheduled, and that leads to errors like below once it's run
> its course:
> {noformat}
> W20240906 10:59:01.768857 189660 tablet_mm_ops.cc:364] T
> 64144a1d4b864aa080e6cc53056546a5 P 574954b3b13a415c83a1660e7f51ee4e: Major
> delta compaction failed on 64144a1d4b864aa080e6cc53056546a5: Corruption:
> Failed major delta compaction on RowSet(1675): No min key found: CFile base
> data in RowSet(1675)
> {noformat}
> Similarly, the {{mt-tablet-test}} is sporadically failing due to the same
> issue when the test workload happens to create similar situation with
> all-the-rows-deleted rowsets:
> {noformat}
> MultiThreadedHybridClockTabletTest/5.UpdateNoMergeCompaction:
> src/kudu/tablet/mt-tablet-test.cc:489: Failure
> Failed
> Bad status: Corruption: Failed major delta compaction on RowSet(1): No min
> key found: CFile base data in RowSet(1)
> {noformat}
> There is a simple test scenario that triggers the issue:
> [https://gerrit.cloudera.org/#/c/21809/|https://gerrit.cloudera.org/#/c/21809/].
> As a workaround, it's possible to set the
> {{\-\-all_delete_op_delta_file_cnt_for_compaction}} to a very high value,
> e.g. 1000000.
> To address the issue properly, it's necessary to update the major delta
> compaction code to handle situations where the result rowset is completely
> empty. In theory, swapping out the result rowset with an empty one should be
> enough: for example, see how it's done in [changelist
> 705954872|https://github.com/apache/kudu/commit/705954872dc86238556456abed0a879bb1462e51].
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
