[
https://issues.apache.org/jira/browse/SPARK-57932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Yang updated SPARK-57932:
------------------------------
Description:
*The issue:*
Spark string positions are 1-based code points (instr, locate, position,
substring, length). For strings containing supplementary characters (code
points > U+FFFF, e.g. emojis), these two functions gave wrong results:
- regexp_instr('😀ab', 'ab') returned 3 instead of 2.
- regexp_replace('😀aXa', 'a', 'Z', 3) returned 😀ZXZ instead of 😀aXZ (replaced a
match before position); regexp_replace('😀😀', '😀', 'Z', 3) returned 😀Z instead
of the input unchanged. This is the sibling of the LIKE fix in SPARK-55453.
> regexp_instr and regexp_replace treat string positions as UTF-16 code units
> instead of code points
> --------------------------------------------------------------------------------------------------
>
> Key: SPARK-57932
> URL: https://issues.apache.org/jira/browse/SPARK-57932
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 5.0.0
> Reporter: Eric Yang
> Priority: Major
>
> *The issue:*
> Spark string positions are 1-based code points (instr, locate, position,
> substring, length). For strings containing supplementary characters (code
> points > U+FFFF, e.g. emojis), these two functions gave wrong results:
> - regexp_instr('😀ab', 'ab') returned 3 instead of 2.
> - regexp_replace('😀aXa', 'a', 'Z', 3) returned 😀ZXZ instead of 😀aXZ (replaced
> a match before position); regexp_replace('😀😀', '😀', 'Z', 3) returned 😀Z
> instead of the input unchanged. This is the sibling of the LIKE fix in
> SPARK-55453.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
