matanper opened a new issue, #70771:
URL: https://github.com/apache/doris/issues/70771

   ### Search before asking
   
   - [x] I had searched in the 
[issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no 
similar issues.
   
   ### Version
   
   4.1.4. master and branch-4.1 have the same code in 
`be/src/storage/compaction/cumulative_compaction_time_series_policy.cpp`.
   
   ### What's Wrong?
   
   A time-series table can stop cumulative compaction permanently on tablets 
that had already compacted before `time_series_compaction_level_threshold` was 
raised from 1 to 2. Each affected tablet keeps one of the highest cumulative 
scores (hundreds of rowsets above its cumulative point) but is never compacted:
   
   - `/api/compaction/show` reports `last cumulative status: [E-2000]` 
(CUMULATIVE_NO_SUITABLE_VERSION) and no successful cumulative compaction.
   - The cloud tablet manager logs every minute: `highest-scoring tablet not 
picked for compaction, type=2 tablet_id=... score=209 reason=skipped (recent 
failure, no suitable version or frozen)`.
   
   On one such tablet (cumulative point 6315), the meta-service rowset metas 
above the cumulative point are:
   
   - 5 merged rowsets (`[6315-6342]` … `[6427-6450]`, 1 segment each) with 
`compaction_level = 0`, written by cumulative compactions that ran before the 
threshold change;
   - 203 merged rowsets with `compaction_level = 1`, older than 24 h;
   - 1 empty single-version rowset from a late load.
   
   ### Why
   
   In `TimeSeriesCumulativeCompactionPolicy::pick_input_rowsets` (the overload 
with `last_cumu`):
   
   1. **Level-1 pick:** `first_rowset_iter` skips every leading rowset that is 
neither a single version nor empty, so the main loop never sees the 208 merged 
rowsets. It sees only the trailing load.
   2. **Level-2 scan:** the `compaction_level >= 2` block stops at the first 
rowset with `compaction_level() == 0`. The first candidate is a merged rowset 
that was compacted while the threshold was 1, and `get_compaction_level` 
returns 0 for every output below threshold 2, so the scan collects nothing.
   3. **Lone rowset:** once the trailing load is older than 
`time_series_compaction_time_threshold_seconds`, condition 3 returns that 
single non-overlapping rowset. 
`CloudCumulativeCompaction::pick_rowsets_to_compact` rejects a lone 
non-overlapping pick with CUMULATIVE_NO_SUITABLE_VERSION, and the level-2 block 
is after condition 3, so it is never reached. Condition 1 already skips a lone 
non-overlapping rowset; conditions 2 and 3 do not.
   
   Point 2 alone stalls the tablet from the moment the threshold changes. Point 
3 also stalls any tablet whose oldest load waiting above level-1 rowsets is a 
lone non-overlapping rowset past the time threshold.
   
   `calc_cumulative_compaction_score` has the same level-0 `break` in its 
level-2 scan, and `get_compaction_level` would give a level-2 merge that starts 
at such a rowset the output level 1.
   
   ### What You Expected?
   
   Raising `time_series_compaction_level_threshold` on an existing table lets 
the level-2 compaction merge the rowsets that are already merged, and a single 
trailing load does not block compaction.
   
   ### How to Reproduce?
   
   As a BE unit test (`cumulative_compaction_time_series_policy_test.cpp`): a 
tablet with `time_series_compaction_level_threshold = 2`, cumulative point 2, 
and rowsets above it:
   
   - `[2-5]`, `[6-9]`: merged, 1 segment, `compaction_level = 0`, created 2 
days ago;
   - `[10-13]`, `[14-17]`, `[18-21]`: merged, `compaction_level = 1`, created 2 
days ago;
   - optionally `[22-22]`: empty, created 2 hours ago (time threshold 3600 s).
   
   With the trailing load, `pick_input_rowsets` returns only `[22-22]`. Without 
it, it returns nothing. Neither picks the level-2 run.
   
   On a cluster: create a time-series table with the default level threshold 1, 
load and let cumulative compaction merge rowsets, run `ALTER TABLE ... SET 
("time_series_compaction_level_threshold" = "2")`, keep loading until level-1 
rowsets accumulate, then stop loading except for an occasional single load.
   
   ### Anything Else?
   
   Proposed fix:
   
   1. Count a merged, non-empty rowset with `compaction_level == 0` (start 
version != end version, segments > 0) as level 1 in the level-2 scan of both 
`pick_input_rowsets` and `calc_cumulative_compaction_score`, and in 
`get_compaction_level`. Below threshold 2 no level is recorded, so such a 
rowset is already a level-1 result. The level-2 merge then includes it and 
writes level 2, which moves the cumulative point.
   2. In conditions 2 and 3, skip a lone non-overlapping rowset as condition 1 
does, and fall through to the level-2 and empty-rowset conditions.
   
   We have a patch with unit tests and can open a PR.
   
   ### Are you willing to submit PR?
   
   - [x] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to