[
https://issues.apache.org/jira/browse/SPARK-18491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen updated SPARK-18491:
------------------------------
Priority: Minor (was: Major)
Description:
TimestampType is mapped to java.sql.Timestamp
DateType is mapped to java.sql.Date
Those both java types are mutable and thus their usage is highly discourage,
especially in distributed computing which uses lazy, functional approach
Mapping to immutable joda times should be enough for now (until scala 2.12 +
jdk8 java.time is available for spark)
was:
TimestampType is mapped to java.sql.Timestamp
DateType is mapped to java.sql.Date
Those both java types are mutable and thus their usage is highly discourage,
especially in distributed computing which uses lazy, functional approach
Mapping to immutable joda times should be enough for now (until scala 2.12 +
jdk8 java.time is available for spark)
It becomes more relevant now as schema is inferred from user provided case
classes in datasets
(definitely not Major)
These are the most logical classes for SQL-like handling of dates and times. I
don't think the risk of users modifying objects they shouldn't is that
significant, or limited to this class. (Most of the usages aren't even user
facing right?)
> Spark uses mutable classes for date/time types mapping
> ------------------------------------------------------
>
> Key: SPARK-18491
> URL: https://issues.apache.org/jira/browse/SPARK-18491
> Project: Spark
> Issue Type: Improvement
> Reporter: Damian Momot
> Priority: Minor
>
> TimestampType is mapped to java.sql.Timestamp
> DateType is mapped to java.sql.Date
> Those both java types are mutable and thus their usage is highly discourage,
> especially in distributed computing which uses lazy, functional approach
> Mapping to immutable joda times should be enough for now (until scala 2.12 +
> jdk8 java.time is available for spark)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
