mrhhsg opened a new pull request, #68448:
URL: https://github.com/apache/doris/pull/68448
### What problem does this PR solve?
Issue Number: None
Problem Summary:
CAST from strings to JSON accepted incomplete `null` literals and malformed
number tokens because the parser trusted simdjson token classification and the
large-number fallback without validating the complete raw token. As a result,
inputs such as `nul`, `[nul]`, `01`, and `1.` could be normalized and persisted
as valid JSON values.
This change validates `null` tokens, applies the JSON number grammar before
accepting large-number fallback errors, rejects non-finite numeric fallback
values, and verifies that the parser consumed the complete document. Valid
integers larger than 64 bits remain supported, including values surrounded by
JSON whitespace.
### Release note
Fix CAST from strings to JSON to reject malformed JSON tokens instead of
normalizing them.
### Check List (For Author)
- Test
- [x] Regression test
- `test_json_cast_validation`
- `test_json_type_cast`
- [x] Unit Test
- `./run-be-ut.sh --run --filter=JsonbParserTest.*` (59 tests
passed, ASAN_UT)
- [x] Manual test
- `./build.sh --be` (ASAN)
- `./build.sh --fe`
- `build-support/clang-format.sh`
- `build-support/check-format.sh`
- `build-support/check-build-hygiene.sh`
- Behavior changed:
- [x] Yes. Malformed JSON text now returns SQL NULL in non-strict casts
and raises an error in strict casts instead of being normalized and persisted.
- Does this need documentation?
- [x] No.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
