Gabriel39 commented on code in PR #69320:
URL: https://github.com/apache/doris/pull/69320#discussion_r4321908841
##########
be/src/format_v2/json/json_reader.cpp:
##########
@@ -750,7 +774,14 @@ Status JsonReader::_append_flat_array_jsonpath_rows(Block*
block, bool* is_empty
}
object_value = rooted_value.get_object();
}
- RETURN_IF_ERROR(_write_columns_by_jsonpath(&object_value, block,
&valid));
+ const size_t row_start = block->rows();
+ const auto status = _write_columns_by_jsonpath(&object_value, block,
&valid);
+ if (!status.ok()) {
+ if (!_request->reject_invalid_rows) {
+ return status;
+ }
+ RETURN_IF_ERROR(_handle_json_error(status, block, row_start,
is_empty_row));
Review Comment:
Fixed in a73b75002f. Missing and non-object configured roots now go through
the existing per-element rejection handler for load requests. Rollback uses the
current element's starting row count, preserving valid earlier siblings and
their presence masks; later siblings and subsequent pipe messages continue to
be read. Query skip behavior is unchanged.
Three real-pipe tests reproduced zero rejected rows before the fix and now
verify the expected counts, valid siblings around rejected roots, and an
entirely rejected message followed by a valid one. All 28 targeted ASAN/UBSAN
tests pass, and clang-format 16 checks pass. Added a JSON Lines Stream Load
regression for the reported input, checking 3 total / 2 filtered rows,
max_filter_ratio=0 failure, and max_filter_ratio=1 success. The cluster
regression has not been run locally and remains for CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
