CalvinKirs opened a new pull request, #69805:
URL: https://github.com/apache/doris/pull/69805
### What problem does this PR solve?
Related PR: #69314
JDBC `setBytes()` and Connector/ODBC binary bindings can send
`MYSQL_TYPE_VAR_STRING` / `MYSQL_TYPE_STRING` rather than BLOB. FE decoded
those bytes with Java's replacement UTF-8 decoder, silently changing arbitrary
binary data. For example, `0080FF4100C328` became `00EFBFBDEFBFBD4100EFBFBD28`
in `SELECT HEX(?)`. The BLOB handling added by #69314 did not cover these wire
types.
Decode string parameters strictly and retain non-UTF-8 payloads as binary
literals. Bind those payloads through a BE-side cast to the original string
category, preserving string function and comparison typing without replacing
bytes. Valid UTF-8 retains the existing text literal types, and BLOB types
retain their existing binary behavior. Use value-based binary literal child
hashes so equivalent casts remain equal after rebinding and forwarding.
### Release note
Preserve non-UTF-8 bytes in MySQL server-prepared string parameters,
including JDBC `setBytes()` and Connector/ODBC binary bindings.
### Check List (For Author)
- Test:
- [x] Regression test: generated and replayed
`prepared_stmt_p0/binary_parameters`; replayed
`prepared_stmt_p0/send_long_data`. Covers all byte values, malformed UTF-8,
parameter reuse, stored HEX/MD5, string expression predicates, NULL/empty
values and ordinary Unicode text.
- [x] Unit Test: 17 tests passed across `MysqlParameterLiteralTest` and
`MysqlConnectProcessorLongDataTest`, including string wire aliases, cached
types, uploads and forwarding.
- [x] Manual test: Connector/J 8.2.0 server-prepared `setBytes`,
`setBinaryStream` and `setBlob` preserve `0080FF4100C328`.
- [x] Manual test: Linux Connector/ODBC 5.3.14 Unicode, 17 cases passed,
including 1-MiB binary/Unicode inputs. An additional 17-case proxy-rewritten
VARCHAR wire-type probe also passed.
- [x] FE build and Checkstyle passed.
- Behavior changed:
- [x] Yes. Non-UTF-8 string-coded parameters preserve the input bytes
instead of substituting U+FFFD. Valid UTF-8 text behavior is unchanged.
- Does this need documentation?
- [x] No.
Runtime regression checks used the newly built master FE with an existing
4.1.4 BE and condition cache disabled. The predicate regression uses
`CONCAT(value, '') = ?`; direct constant predicate pushdown on a current master
BE was not validated. No BE, storage-format or protocol-field changes are
included.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
