sollhui commented on code in PR #67509:
URL: https://github.com/apache/doris/pull/67509#discussion_r4332555045
##########
fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadJob.java:
##########
@@ -2070,6 +2072,10 @@ public void gsonPostProcess() throws IOException {
// for ALTER ROUTINE LOAD
protected void modifyCommonJobProperties(Map<String, String>
jobProperties) throws UserException {
+ if (jobProperties.containsKey(CsvFileFormatProperties.PROP_ENCLOSE)
+ ||
jobProperties.containsKey(CsvFileFormatProperties.PROP_ESCAPE)) {
+ updateCsvParserProperties(jobProperties);
Review Comment:
[P2] Keep CSV parser properties unchanged when ALTER fails
`updateCsvParserProperties()` mutates `this.enclose`/`this.escape` before
the remaining ALTER validations have succeeded. For example, on a paused CSV
routine load job:
```sql
ALTER ROUTINE LOAD FOR job_name
PROPERTIES (
"enclose" = "^",
"unique_key_update_mode" = "UPDATE_FLEXIBLE_COLUMNS"
);
```
The new call first changes the cached `enclose` byte, then
`validateFlexiblePartialUpdateForAlter()` throws because flexible partial
update does not support CSV (or because the target table fails its
constraints). The exception prevents `modifyPropertiesInternal()` from reaching
`this.jobProperties.putAll(...)`, and `modifyProperties()` never writes the
ALTER journal entry. There is no rollback of the cached byte.
Consequently, although the statement reports failure, resuming the job makes
new tasks use `^`, while SHOW/persisted properties, followers, and checkpoint
restoration still use the old value. The same ordering applies to Kafka and
Kinesis through this shared method.
Please complete all potentially failing validations before committing the
parser-byte changes and the job properties. A regression test combining a valid
CSV-character change with a rejected update-mode change should assert that both
the task snapshot and persisted properties retain their original values after
the failure.
This finding is based on tracing the current source; I have not run this
scenario against a Doris cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
