[
https://issues.apache.org/jira/browse/FLINK-40578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sepuri sai krishna updated FLINK-40578:
---------------------------------------
Release Note: {{OVERLAY}} now keeps the remainder of the string and counts
offsets in characters. Previously some argument combinations discarded
everything after the replacement, inputs containing supplementary characters
could be split into unpaired surrogates, and a large length argument could
throw {{StringIndexOutOfBoundsException}}. Queries that relied on the truncated
result will now see the full string.
> OVERLAY returns wrong results for several argument combinations
> ---------------------------------------------------------------
>
> Key: FLINK-40578
> URL: https://issues.apache.org/jira/browse/FLINK-40578
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Runtime
> Affects Versions: 2.0.0, 2.1.0, 2.2.0, 2.3.0
> Reporter: sepuri sai krishna
> Priority: Major
> Labels: pull-request-available
>
> The documentation says {{OVERLAY}} "replaces INT2 (STRING2's length by
> default) characters of STRING1 with STRING2 from position INT1". It does not,
> for the arguments below.
> {{E}} stands for the single character U+1F600 GRINNING FACE, so {{'aEb'}} is
> a three-character string whose middle character is that emoji. It is written
> this way to keep this description inside the Basic Multilingual Plane.
> || Expression || Returns || Expected ||
> | {{OVERLAY('abcdef' PLACING 'X' FROM 2 FOR 0)}} | {{aX}} | {{aXbcdef}} |
> | {{OVERLAY('abcdef' PLACING 'X' FROM 1 FOR 0)}} | {{X}} | {{Xabcdef}} |
> | {{OVERLAY('aEb' PLACING 'X' FROM 2 FOR 1)}} | {{a}}, {{X}}, then an
> unpaired low surrogate, then {{b}} | {{aXb}} |
> | {{OVERLAY('EE' PLACING 'X' FROM 1 FOR 1)}} | {{X}}, an unpaired low
> surrogate, then {{E}} | {{XE}} |
> | {{OVERLAY('abc' PLACING E FROM 2)}} | {{aE}} | {{aEc}} |
> | {{OVERLAY('123456789' PLACING 'abc' FROM 2 FOR 4294967301)}} | {{1abc789}}
> | {{1abc}} |
> | {{OVERLAY('123456789' PLACING 'abc' FROM 2 FOR 2147483647)}} |
> {{java.lang.StringIndexOutOfBoundsException}} | {{1abc}} |
> The first two rows need no unusual data at all.
> h3. Impact
> Silent data loss, with nothing logged. The rows producing an unpaired
> surrogate additionally return invalid UTF-16, which does not round-trip:
> encoding the result yields a replacement character or malformed bytes.
> h3. Affects
> master, and release-2.0 through release-2.3, where the implementation is
> byte-identical.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
