chong created SPARK-52351:
-----------------------------
Summary: Casting string to timestamp can not handle pattern:
spaces + Thh:mm:ss
Key: SPARK-52351
URL: https://issues.apache.org/jira/browse/SPARK-52351
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 4.0.0
Reporter: chong
There is a difference between Spark350 and Spark400 when handing pattern: space
followed by a {{T}}
h3. Spark350:
{{scala> spark.sql("select cast(' T23:17:50' as timestamp)").show()}}
{{2025-05-30 23:17:50}}
h3. Spark400:
{{spark.conf.set("spark.sql.ansi.enabled", false)}}
{{spark.sql("select cast(' T23:17:50' as timestamp)").show()}}
{{ NULL}}
{{}}
h3. bug explain
[Code
link|https://github.com/apache/spark/blob/v4.0.0/sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/SparkDateTimeUtils.scala#L450-L468]
var j = getTrimmedStart(bytes) // ============== here j maybe not
zero after trim leftval strEndTrimmed = getTrimmedEnd(j, bytes) if (j ==
strEndTrimmed) { return (Array.empty, None, false)
} var digitsMilli = 0var justTime = falsevar yearSign: Option[Int] =
Noneif (bytes(j) == '-' || bytes(j) == '+') {
yearSign = if (bytes(j) == '-') Some(-1) else Some(1)
j += 1 } while (j < strEndTrimmed) { val b = bytes(j) val
parsedValue = b - '0'.toByte if (parsedValue < 0 || parsedValue > 9) {
if (j == 0 && b == 'T') {
justTime = true // ============== never reach to here if trimed left
and pattern is Thh:mm:ss i += 3 } else if (i < 2) {{{}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
