Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/8606#discussion_r38992815
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
---
@@ -56,6 +57,12 @@ object DateTimeUtils {
@transient lazy val defaultTimeZone = TimeZone.getDefault
+ // Constants defining the allowed ranges for timestampts that fit in
parquet files. Mostly
+ // used by tests to avoid duplication.
+ private[spark] final val MIN_TIMESTAMP: Long =
+ -(DateTimeUtils.JULIAN_DAY_OF_EPOCH * DateTimeUtils.SECONDS_PER_DAY *
1000)
--- End diff --
It's the right value if you consider the math. But I've seen really weird
behavior in how the Java classes print very large (positive or negative)
timestamps, and I didn't find anything in the javadocs about limits, so I'm not
sure what's the best way to proceed. We can choose an arbitrary minimum and
maximum, but what would those be based on?
For example, `java.sql.Timestamp` will print the same formatted date string
for both `-793495812000` and `-123456789012000L`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
