This is an automated email from the ASF dual-hosted git repository.
zclllyybb pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new cc9491a751d [fix](regression) Tolerate dynamic partition scheduler
race (#65902)
cc9491a751d is described below
commit cc9491a751d920d5ae970aab5b1fa6d6af686283
Author: zclllyybb <[email protected]>
AuthorDate: Thu Jul 23 11:10:17 2026 +0800
[fix](regression) Tolerate dynamic partition scheduler race (#65902)
The regression case can overlap with a dynamic-partition scheduler run
that was started while the global interval was one second. Restoring the
interval to 600 seconds does not cancel that in-flight run. If it drops
the out-of-range `p1900` partition while the multi-row INSERT is
committing, the INSERT can fail even though the scheduler reached its
intended final state.
The prior workaround depended on an exact exception message and issued
`show partitions` inside a `TestAction` check closure. That nested DSL
call changes the action current SQL, making the failure report and its
string-based classification brittle.
This change drains the likely preceding short-interval run before
INSERT, captures the INSERT exception without nested DSL calls, and
tolerates only the known terminal state: `p1900` is absent while the
`2024`, `2900`, and `3000` auto-created partitions remain. All other
failures still fail the case. The intentional one-second cleanup
interval is restored to 600 seconds in `finally`.
---
.../auto_partition/test_auto_dynamic.groovy | 62 +++++++++++++---------
1 file changed, 38 insertions(+), 24 deletions(-)
diff --git
a/regression-test/suites/partition_p0/auto_partition/test_auto_dynamic.groovy
b/regression-test/suites/partition_p0/auto_partition/test_auto_dynamic.groovy
index ad6bc3db160..f1d9ad0e077 100644
---
a/regression-test/suites/partition_p0/auto_partition/test_auto_dynamic.groovy
+++
b/regression-test/suites/partition_p0/auto_partition/test_auto_dynamic.groovy
@@ -116,31 +116,45 @@ suite("test_auto_dynamic", "nonConcurrent") {
part_result = sql " show partitions from auto_dynamic "
assertEquals(part_result.size(), 1)
- def skip_test = false
- test {
- sql " insert into auto_dynamic values ('2024-01-01'), ('2900-01-01'),
('1900-01-01'), ('3000-01-01'); "
- check { result, exception, startTime, endTime ->
- if (exception != null) {
- // the partition of 1900-01-01 directly been recovered before
the insert txn finished. let it success
- part_result = sql " show partitions from auto_dynamic "
- log.info("${part_result}".toString())
- assertTrue(exception.getMessage().contains("get partition
p19000101000000 failed"))
- skip_test = true
+ def oldCheckInterval =
getFeConfig("dynamic_partition_check_interval_seconds")
+ try {
+ def insertException = null
+ test {
+ sql " insert into auto_dynamic values ('2024-01-01'),
('2900-01-01'), ('1900-01-01'), ('3000-01-01'); "
+ check { result, exception, startTime, endTime ->
+ insertException = exception
}
}
- }
- if (skip_test) {
- return true
- }
-
- sql """ admin set frontend config
('dynamic_partition_check_interval_seconds' = '1') """
- sleep(10000)
- part_result = sql " show partitions from auto_dynamic "
- log.info("${part_result}".toString())
- assertTrue(part_result.size() == 3 || part_result.size() == 4,
- "The partition size should be 3 or 4, but got ${part_result.size()}")
+ if (insertException != null) {
+ part_result = sql " show partitions from auto_dynamic "
+ def partitionNames = part_result.collect { it[1] }
+ def expectedSurvivingPartitions = ["p20240101000000",
"p29000101000000", "p30000101000000"]
+ // A stale short-interval run can only delete the out-of-range
p1900 partition before commit.
+ assertTrue(!partitionNames.contains("p19000101000000")
+ && partitionNames.containsAll(expectedSurvivingPartitions),
+ "Unexpected insert failure: ${insertException}, partitions:
${partitionNames}")
+ return true
+ }
- qt_sql_dynamic_auto "select * from auto_dynamic order by k0;"
+ setFeConfig("dynamic_partition_check_interval_seconds", 1)
+ def partitionNames = []
+ for (int retry = 0; retry < 1200; retry++) {
+ part_result = sql " show partitions from auto_dynamic "
+ partitionNames = part_result.collect { it[1] }
+ if (!partitionNames.contains("p19000101000000")) {
+ break
+ }
+ sleep(1000)
+ }
+ // Changing the interval does not wake a scheduler that is already
sleeping.
+ assertTrue(!partitionNames.contains("p19000101000000"),
+ "Dynamic partition scheduler did not recycle p1900 within 1200
seconds, partitions: ${partitionNames}")
+ log.info("${part_result}".toString())
+ assertTrue(part_result.size() == 3 || part_result.size() == 4,
+ "The partition size should be 3 or 4, but got
${part_result.size()}")
- sql """ admin set frontend config
('dynamic_partition_check_interval_seconds' = '600') """
-}
\ No newline at end of file
+ qt_sql_dynamic_auto "select * from auto_dynamic order by k0;"
+ } finally {
+ setFeConfig("dynamic_partition_check_interval_seconds",
oldCheckInterval)
+ }
+}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
