hudi-agent commented on code in PR #20133:
URL: https://github.com/apache/hudi/pull/20133#discussion_r4135371785
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieAppendHandle.java:
##########
@@ -401,7 +402,10 @@ protected boolean writeRecord(HoodieRecord<T>
hoodieRecord) {
return true;
} catch (Exception e) {
log.error("Error writing record {}", hoodieRecord, e);
- if (!config.getIgnoreWriteFailed() || ExceptionUtil.isCausedBy(e,
HoodieEarlyConflictDetectionException.class)) {
+ // A failed log flush affects the entire buffered batch, not just the
current record.
+ if (!config.getIgnoreWriteFailed()
+ || ExceptionUtil.isCausedBy(e, HoodieAppendException.class)
Review Comment:
🤖 Only rollover flush failures are non-ignorable here. Can
`appendRecord`/`appendDeleteRecord` also break the whole batch? E.g. the native
parquet writer fails while flushing a full row group, or `ensureDataFileWriter`
fails after `dataLogFile` is set. With ignore-failed on, the loop keeps going
on that writer, and `testRecordWriteFailureCanStillBeIgnored` locks that in.
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag
quality.</i></sub>
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieNativeLogAppendHandle.java:
##########
@@ -162,7 +162,7 @@ protected void flushAppend() {
writer.flushAppend(getUpdatedHeader(header));
processAppendResults(writer.getLastAppendResults());
}
- } catch (IOException e) {
+ } catch (Exception e) {
Review Comment:
🤖 If `dataFileWriter.close()` throws inside
`flushAppend`→`closeFileWriters`, `dataFileWriter` never gets nulled and
`deleteFileWriter` never gets closed. `closeLogWriterQuietly` then calls
`writer.close()`, which closes the data writer a second time and can throw
again before the delete writer is closed. Could that leak the delete file's
output stream?
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag
quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
