Hi,
Why is casting a string column to timestamp not giving the same results as
going through casting to long in-between? I'm tempted to consider it a bug.
scala> spark.version
res4: String = 2.4.3
scala> Seq("1", "2").toDF("ts").select($"ts" cast "timestamp").show
+----+
| ts|
+----+
|null|
|null|
+----+
scala> Seq("1", "2").toDF("ts").select($"ts" cast "long").select($"ts" cast
"timestamp").show
+-------------------+
| ts|
+-------------------+
|1970-01-01 01:00:01|
|1970-01-01 01:00:02|
+-------------------+
Pozdrawiam,
Jacek Laskowski
----
https://about.me/JacekLaskowski
The Internals of Spark SQL https://bit.ly/spark-sql-internals
The Internals of Spark Structured Streaming
https://bit.ly/spark-structured-streaming
The Internals of Apache Kafka https://bit.ly/apache-kafka-internals
Follow me at https://twitter.com/jaceklaskowski
