This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-4.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-4.1 by this push:
new cf0feee3d9c [branch-4.1][fix](test) Stabilize export parallelism CSV
reload (#65932)
cf0feee3d9c is described below
commit cf0feee3d9ce22542b383cc0214328753c7dcd4f
Author: Dongyang Li <[email protected]>
AuthorDate: Thu Jul 23 14:32:36 2026 +0800
[branch-4.1][fix](test) Stabilize export parallelism CSV reload (#65932)
### What problem does this PR solve?
Backport #65395 to branch-4.1. S3 P2 build #203804 failed
`export_p2.test_export_with_parallelism` while reloading CSV export
fragments: expected `[10, ftw-10, 28]`, actual `[100, null, null]`
(DORIS-27418).
### What is changed?
- use a unique S3 export prefix for each invocation to avoid cross-round
object interference
- provide an explicit `id:int;name:string;age:int` schema for CSV-family
reloads
### Validation
- cherry-pick of master fix a35099df1f5 applies cleanly
- `git diff --check` passed
- terminal validation requires rerunning the exact branch-4.1 S3 P2 case
after merge
Co-authored-by: shuke <[email protected]>
---
.../suites/export_p2/test_export_with_parallelism.groovy | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git
a/regression-test/suites/export_p2/test_export_with_parallelism.groovy
b/regression-test/suites/export_p2/test_export_with_parallelism.groovy
index ece87bafd8b..f22f032afb0 100644
--- a/regression-test/suites/export_p2/test_export_with_parallelism.groovy
+++ b/regression-test/suites/export_p2/test_export_with_parallelism.groovy
@@ -87,10 +87,9 @@ suite("test_export_with_parallelism", "p2") {
}
}
- def outFilePath = """${bucket}/export/exp_"""
-
def test_export = {format, file_suffix, isDelete, parallelism, whereExpr ->
def uuid = UUID.randomUUID().toString()
+ def outFilePath =
"""${bucket}/export/${table_export_name}/${uuid}/exp_"""
// exec export
sql """
EXPORT TABLE ${table_export_name} ${whereExpr} TO
"s3://${outFilePath}"
@@ -126,6 +125,13 @@ suite("test_export_with_parallelism", "p2") {
// because parallelism = min(parallelism, tablets_num)
for (int j = 0; j < parallelism && j < 3; ++j) {
+ def csvSchemaProps = ""
+ if (format in ["csv", "csv_with_names",
"csv_with_names_and_types"]) {
+ csvSchemaProps = """
+ "column_separator" = "\t",
+ "csv_schema" = "id:int;name:string;age:int",
+ """
+ }
// check data correctness
sql """ insert into ${table_load_name}
select * from s3(
@@ -133,6 +139,7 @@ suite("test_export_with_parallelism", "p2") {
"s3.access_key"= "${ak}",
"s3.secret_key" = "${sk}",
"format" = "${format}",
+ ${csvSchemaProps}
"region" = "${region}",
"provider" = "${getS3Provider()}",
"use_path_style" = "false" -- aliyun does not support
path_style
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
