[
https://issues.apache.org/jira/browse/SPARK-18491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15673590#comment-15673590
]
Damian Momot commented on SPARK-18491:
--------------------------------------
I don't mean internal usage - it's about user usage related to datasets.
If I want to create dataset which will be mapped to timestamp datatype i have
to use
{code}
case class Test(id: String, timestamp: java.sql.Timestamp)
{code}
It will work and save data properly to parquet etc. but all my further dataset
manipulation code will be prone to problems related to field mutability
> Spark uses mutable classes for date/time types mapping
> ------------------------------------------------------
>
> Key: SPARK-18491
> URL: https://issues.apache.org/jira/browse/SPARK-18491
> Project: Spark
> Issue Type: Improvement
> Reporter: Damian Momot
> Priority: Minor
>
> TimestampType is mapped to java.sql.Timestamp
> DateType is mapped to java.sql.Date
> Those both java types are mutable and thus their usage is highly discourage,
> especially in distributed computing which uses lazy, functional approach
> Mapping to immutable joda times should be enough for now (until scala 2.12 +
> jdk8 java.time is available for spark)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
