github-actions[bot] commented on code in PR #67489:
URL: https://github.com/apache/doris/pull/67489#discussion_r3923729812
##########
fe/fe-core/src/main/java/org/apache/doris/job/extensions/insert/streaming/StreamingInsertTask.java:
##########
@@ -130,12 +156,32 @@ public void run() throws JobException {
}
throw new JobException(errMsg);
} catch (Exception e) {
+ String errorMessage = Util.getRootCauseMessage(e);
+ if (auditEnabled && ctx.getState().getStateType() !=
QueryState.MysqlStateType.ERR) {
+ ctx.getState().setError(ErrorCode.ERR_INTERNAL_ERROR,
errorMessage);
+ }
log.warn("execute insert task error, label is {},offset is {}",
taskCommand.getLabelName(),
runningOffset.toString(), e);
- throw new JobException(Util.getRootCauseMessage(e));
+ throw new JobException(errorMessage);
+ } finally {
+ if (auditEnabled) {
+ AuditLogHelper.logAuditLog(ctx, auditSql,
stmtExecutor.getParsedStmt(),
+ stmtExecutor.getQueryStatisticsForAuditLog(), true);
+ }
}
}
+ private String getAuditSql(TreeMap<Pair<Integer, Integer>, String>
replacements) {
+ List<UnboundTVFRelation> tvfRelations =
taskCommand.getAllTVFRelation();
+ Preconditions.checkState(replacements.size() == 1 &&
tvfRelations.size() == 1,
+ "S3 streaming insert must contain exactly one TVF");
+ String rewrittenProperties = new DatasourcePrintableMap<>(
+ tvfRelations.get(0).getProperties().getMap(), "=", true,
false, true).toString();
Review Comment:
[P2] Replace the URI property case-insensitively
The S3 `uri` property is accepted case-insensitively (the extractor uses
`equalsIgnoreCase`, and existing regression cases use uppercase `"URI"`), but
the rewrite copies the case-preserving map and adds a separate lowercase `uri`.
For a valid job whose source is `"URI" = "s3://.../*.csv"`, this printer
therefore writes both the original wildcard and the resolved file list into the
audit statement, and the downstream encryption pass preserves both. That
defeats the new audit record's purpose of identifying the exact imported files.
Please replace/remove the existing URI entry case-insensitively before
rendering, and cover the uppercase spelling in the test.
##########
fe/fe-core/src/main/java/org/apache/doris/job/extensions/insert/streaming/StreamingInsertTask.java:
##########
@@ -130,12 +156,32 @@ public void run() throws JobException {
}
throw new JobException(errMsg);
} catch (Exception e) {
+ String errorMessage = Util.getRootCauseMessage(e);
+ if (auditEnabled && ctx.getState().getStateType() !=
QueryState.MysqlStateType.ERR) {
+ ctx.getState().setError(ErrorCode.ERR_INTERNAL_ERROR,
errorMessage);
+ }
log.warn("execute insert task error, label is {},offset is {}",
taskCommand.getLabelName(),
runningOffset.toString(), e);
- throw new JobException(Util.getRootCauseMessage(e));
+ throw new JobException(errorMessage);
+ } finally {
+ if (auditEnabled) {
+ AuditLogHelper.logAuditLog(ctx, auditSql,
stmtExecutor.getParsedStmt(),
+ stmtExecutor.getQueryStatisticsForAuditLog(), true);
+ }
}
}
+ private String getAuditSql(TreeMap<Pair<Integer, Integer>, String>
replacements) {
+ List<UnboundTVFRelation> tvfRelations =
taskCommand.getAllTVFRelation();
+ Preconditions.checkState(replacements.size() == 1 &&
tvfRelations.size() == 1,
+ "S3 streaming insert must contain exactly one TVF");
+ String rewrittenProperties = new DatasourcePrintableMap<>(
Review Comment:
[P1] Keep quoted TVF options reparsable
`DatasourcePrintableMap` is a display formatter, not a SQL-literal
serializer: it wraps each value in double quotes but does not escape embedded
quotes. Doris supports the common CSV option `"enclose" = "\\\""`; after
parsing that value is a literal `"`, so this code synthesizes `"enclose" =
"""`. The INSERT still runs from the already-built plan, but `AuditLogHelper`
reparses `auditSql` via `geneEncryptionSQL`; that parse failure is caught and
the required audit event is silently dropped. Please use the parser's
SQL-literal escaping (or avoid reparsing rendered text) and add a test with the
supported quote encloser.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
