mrhhsg opened a new pull request, #68243:
URL: https://github.com/apache/doris/pull/68243
### What problem does this PR solve?
Issue Number: None
Problem Summary: A Doris STRING value is binary, so it may contain a NUL
byte. `topn()` keeps the whole value in its state, but writes the result JSON
with `writer.Key(element.second.c_str())`, which has no length argument and
therefore stops at the first NUL. `topn(concat('a', unhex('00'), 'b'), 1)`
returns `{"a":1}` instead of the real key, and two values that only differ
after the NUL collapse into the same duplicated key, e.g. `{"a":1,"a":1}`, so
the caller cannot recover the top-N values at all.
Pass the key together with its length so rapidjson escapes the NUL as
`\u0000` and emits the whole key.
Before:
```
mysql> SELECT topn(s, 2) FROM (SELECT concat('a', unhex('00'), 'b') AS s
-> UNION ALL SELECT concat('a', unhex('00'),
'c')) t;
+---------------------+
| {"a":1,"a":1} |
+---------------------+
```
After:
```
+-------------------------------+
| {"a\u0000b":1,"a\u0000c":1} |
+-------------------------------+
```
### Release note
None
### Check List (For Author)
- Test:
- Unit Test: Yes, `AggTopNTest.test_string_key_with_embedded_nul` covers
a string key with an embedded NUL and two keys that only differ after it.
- Regression test: Yes, `test_topn_embedded_nul` checks `topn()` on such
keys and keeps `topn_array()` as a reference.
- Behavior changed: Yes, `topn()` now emits the complete key with the NUL
escaped as `\u0000` instead of a key truncated at the NUL.
- Does this need documentation: No
https://claude.ai/code/session_01RZ36Pij3o8fGnYYg33PKnq
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
