MaxGekk commented on code in PR #50270:
URL: https://github.com/apache/spark/pull/50270#discussion_r1996172846
##########
common/variant/src/main/java/org/apache/spark/types/variant/VariantUtil.java:
##########
@@ -123,6 +123,12 @@ public class VariantUtil {
// Long string value. The content is (4-byte little-endian unsigned integer
representing the
// string size) + (size bytes of string content).
public static final int LONG_STR = 16;
+ // Similar to TIMESTAMP, but the 8-byte value represents the number of
nanoseconds since the
Review Comment:
What's the supported range if the underlying type is `long`?
```python
>>> 2**64 / (360 * 24 * 60 * 60 * 1000*1000*1000)
593.0666175961147
```
Is it 593 years?
##########
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/variant/VariantExpressionSuite.scala:
##########
@@ -856,6 +858,45 @@ class VariantExpressionSuite extends SparkFunSuite with
ExpressionEvalHelper {
val bytes = builder.result().getValue
checkCast(bytes, StringType,
"01020304-0506-0708-090a-0b0c0d0e0f10")
+
+ // TimestampNanos
+ // Cast and to_json produce slightly different formats.
+ val timestamps = Seq(
+ ("2024-12-16T10:23:45-08:00", "2024-12-16 10:23:45", "2024-12-16
10:23:45-08:00"),
+ ("1970-01-01T00:00:00.123456789-08:00", "1970-01-01
00:00:00.123456789",
+ "1970-01-01 00:00:00.123456789-08:00"),
+ ("2100-12-31T23:59:59.00100-08:00", "2100-12-31 23:59:59.001",
+ "2100-12-31 23:59:59.001-08:00")
Review Comment:
Could you test the ancient timestamps like 1970-594 > 1376 year.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
