[
https://issues.apache.org/jira/browse/KUDU-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Chennaka updated KUDU-3583:
------------------------------------
Description:
As a part of KUDU-2677, the default value of the flag
--tablet_history_max_age_sec was raised from 15*60 seconds to 60*60*24*7
seconds to primarily accommodate diff Kudu scans for incremental backups and
multiple full table Kudu scans in a single job during full backups for Kudu
backup and restore tool.
It was seen that this change has caused a lot of write amplifications in update
heavy workloads due to storing of the long history of deltas. While
optimizations are being done in this area, it doesn't make much sense to store
all these deltas if we do not perform such scans in the first place.
We can probably revert back the value of the flag and have a helpful error
message in case of such scans.
was:
As a part of KUDU-2677, the default value of the flag
--tablet_history_max_age_sec was raised from 15*60 seconds to 60*60*24*7
seconds to primarily accommodate diff Kudu scans for incremental backups and
multiple full table Kudu scans in a single job during full backups for Kudu
backup and restore tool.
It was seen that this change has caused a lot of write amplifications in update
heavy workloads due to storing of the long history of deltas. While
optimizations are being done in this area, it doesn't make much sense to store
all these deltas if we do not perform such scans in the first place.
We can probably revert back the value of the flag and have a helpful error
message in case of long/table wide scans.
> Revert flag tablet_history_max_age_sec to 15 minutes
> ----------------------------------------------------
>
> Key: KUDU-3583
> URL: https://issues.apache.org/jira/browse/KUDU-3583
> Project: Kudu
> Issue Type: Improvement
> Reporter: Abhishek Chennaka
> Priority: Major
>
> As a part of KUDU-2677, the default value of the flag
> --tablet_history_max_age_sec was raised from 15*60 seconds to 60*60*24*7
> seconds to primarily accommodate diff Kudu scans for incremental backups and
> multiple full table Kudu scans in a single job during full backups for Kudu
> backup and restore tool.
> It was seen that this change has caused a lot of write amplifications in
> update heavy workloads due to storing of the long history of deltas. While
> optimizations are being done in this area, it doesn't make much sense to
> store all these deltas if we do not perform such scans in the first place.
> We can probably revert back the value of the flag and have a helpful error
> message in case of such scans.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
