This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-4.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-4.1 by this push:
new 833204c2523 branch-4.1: [fix](test) stabilize Paimon thread lifecycle
check (#68130)
833204c2523 is described below
commit 833204c2523b423d1e97fd31da57d010d0cd2e89
Author: Socrates <[email protected]>
AuthorDate: Fri Sep 18 13:58:51 2026 +0800
branch-4.1: [fix](test) stabilize Paimon thread lifecycle check (#68130)
### What problem does this PR solve?
The Paimon thread lifecycle regression test required JVM and process
thread counts to return to a fixed pre-phase baseline. This is unstable
because later writes may be scheduled on bounded native workers that
have not entered the JVM during the fixed warm-up. Such workers can
cause a one-time increase even though the counts remain stable across
all measured phases.
For example, a run with JVM counts changing from 576 to 594, 594, 594,
594 is bounded stabilization rather than continuous thread growth.
### What is changed?
- Remove the pre-phase baseline sampling and return-to-baseline
assertions.
- Keep the early-versus-late phase low-water assertions for both JVM and
process threads.
- Clarify that bounded worker-pool warm-up is allowed, while persistent
phase-to-phase growth still fails.
This preserves coverage for the original JNI lifecycle regression, which
showed continuing growth across equal write phases.
### Validation
- git diff --check
- External Paimon regression environment was not available locally.
---
.../test_paimon_write_thread_lifecycle.groovy | 22 ++++++----------------
1 file changed, 6 insertions(+), 16 deletions(-)
diff --git
a/regression-test/suites/external_table_p0/paimon/write/test_paimon_write_thread_lifecycle.groovy
b/regression-test/suites/external_table_p0/paimon/write/test_paimon_write_thread_lifecycle.groovy
index e5f34149a76..420f53cd18b 100644
---
a/regression-test/suites/external_table_p0/paimon/write/test_paimon_write_thread_lifecycle.groovy
+++
b/regression-test/suites/external_table_p0/paimon/write/test_paimon_write_thread_lifecycle.groovy
@@ -95,8 +95,9 @@ suite("test_paimon_write_thread_lifecycle",
"p0,external,paimon,nonConcurrent")
sql """use ${dbName}"""
try {
- // Warm all writer and metrics paths before taking the baseline. This
keeps
- // one-time JVM attachment and SDK class initialization out of the
leak oracle.
+ // Warm common writer and metrics paths before measuring
phase-to-phase growth. The
+ // blocking scheduler may still assign later writes to native workers
that have not entered
+ // the JVM yet, so a bounded one-time increase after warm-up is not
evidence of a leak.
for (int round = 0; round < 12; round++) {
sql """
INSERT INTO t_thread_lifecycle
@@ -106,10 +107,6 @@ suite("test_paimon_write_thread_lifecycle",
"p0,external,paimon,nonConcurrent")
}
sleep(3000)
- def jvmBefore = minimumThreadCounts(jvmThreadCounts)
- def processBefore = minimumThreadCounts(processThreadCounts)
- logger.info("Paimon thread baseline: jvm=${jvmBefore},
process=${processBefore}")
-
def writePhase = { int firstRound ->
for (int round = firstRound; round < firstRound + 12; round++) {
sql """
@@ -135,10 +132,9 @@ suite("test_paimon_write_thread_lifecycle",
"p0,external,paimon,nonConcurrent")
(sql """SELECT COUNT(*) FROM t_thread_lifecycle""")[0][0] as
long)
backendEndpoints.keySet().each { backendId ->
- // Warm-up performs the same workload as every measured phase.
Judge persistent growth
- // from the actual pre-phase baseline and phase low-water marks
instead of failing on
- // an isolated background-thread spike: a leaked thread cannot
disappear in a later
- // phase, while an unrelated transient thread can.
+ // Judge persistent growth between equal measured phases.
Comparing the early and late
+ // low-water marks allows a bounded worker-pool warm-up or an
isolated background-thread
+ // spike, while a leaked thread cannot disappear in a later phase.
def jvmCounts = jvmPhases.collect { sample -> sample[backendId] as
long }
def processCounts = processPhases.collect { sample ->
sample[backendId] as long }
def earlyJvmFloor = jvmCounts.take(2).min()
@@ -146,14 +142,8 @@ suite("test_paimon_write_thread_lifecycle",
"p0,external,paimon,nonConcurrent")
def earlyProcessFloor = processCounts.take(2).min()
def lateProcessFloor = processCounts.drop(2).min()
- assertTrue(jvmCounts.min() <= jvmBefore[backendId] + 2,
- "JVM threads never returned to the warm-up baseline on
backend ${backendId}: "
- + "baseline=${jvmBefore[backendId]},
phases=${jvmCounts}")
assertTrue(lateJvmFloor <= earlyJvmFloor + 2,
"JVM threads kept growing on backend ${backendId}:
phases=${jvmCounts}")
- assertTrue(processCounts.min() <= processBefore[backendId] + 4,
- "Process threads never returned to the warm-up baseline on
backend ${backendId}: "
- + "baseline=${processBefore[backendId]},
phases=${processCounts}")
assertTrue(lateProcessFloor <= earlyProcessFloor + 4,
"Process threads kept growing on backend ${backendId}:
phases=${processCounts}")
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
