[
https://issues.apache.org/jira/browse/IGNITE-26067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18018748#comment-18018748
]
Pavel Pereslegin commented on IGNITE-26067:
-------------------------------------------
[~korlov], please take a look at the proposed changes.
> Add modification counter to partition
> -------------------------------------
>
> Key: IGNITE-26067
> URL: https://issues.apache.org/jira/browse/IGNITE-26067
> Project: Ignite
> Issue Type: Improvement
> Components: tables ai3
> Reporter: Konstantin Orlov
> Assignee: Pavel Pereslegin
> Priority: Major
> Labels: ignite-3
> Time Spent: 40m
> Remaining Estimate: 0h
>
> To implement staleness check for query plan auto-reoptimization, it's
> proposed to introduce modification counter on per-partition basis. We don’t
> really need a fair modification counter, but rather a timestamp when we
> detected staleness for the last time. Hence, for every partition we need to
> maintain a triplet of volatile attributes *{{(modificationCounter,
> nextMilestone, lastMilestoneReachedTs)}}*. This triplet should be initialized
> at the start of the partition with values *{{(0,
> computeNextMilestone(currentPartitionSize), HybridClock.now())}}*. To
> implement *{{computeNextMilestone}}*, we need to introduce 2 new
> configuration parameters:
> {code:java}
> sql.statistics.staleRowsFraction = 0.2 (default) // fraction of table
> required to be modified to trigger statistics recollection, i.e. currentSize
> * staleRowsFraction to compute actual number of rows required
> sql.statistics.minStaleRowsCount = 500 (default) // minimal number of updates
> required to trigger statistics recollection
> {code}
> These are cluster wide parameters, later we will introduce overrides on a
> per-table basis.
> A *{{modificationCounter}}* should be incremented by 1 per any RowId from
> update command, i.e. it should be incremented by 1 for *{{UpdateCommand}}*,
> and by *{{size(batch)}}* for *{{UpdateAllCommand}}*.
> When *{{modificationCounter}}* exceeds *{{nextMilestone}}*, a new milestone
> should be computed, and *{{lastMilestoneReachedTs}}* is set with
> *{{commitTs}}*.
> The counters should be exposed as metrics
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
