Cheng, That’s exactly what I was hoping for – native support for writing DateTime objects. As it stands Spark 1.5.2 seems to leave no option but to do manual conversion (to nanos, Timestamp, etc) prior to writing records to hive.
Regards, Bryan Jeffrey Sent from Outlook Mail From: Cheng Lian Sent: Tuesday, November 24, 2015 1:42 AM To: Bryan Jeffrey;user Subject: Re: DateTime Support - Hive Parquet Hey Bryan, What do you mean by "DateTime properties"? Hive and Spark SQL both support DATE and TIMESTAMP types, but there's no DATETIME type. So I assume you are referring to Java class DateTime (possibly the one in joda)? Could you please provide a sample snippet that illustrates your requirement? Cheng On 11/23/15 9:40 PM, Bryan Jeffrey wrote: > All, > > I am attempting to write objects that include a DateTime properties to > a persistent table using Spark 1.5.2 / HiveContext. In 1.4.1 I was > forced to convert the DateTime properties to Timestamp properties. I > was under the impression that this issue was fixed in the default Hive > supported with 1.5.2 - however, I am still seeing the associated errors. > > Is there a bug I can follow to determine when DateTime will be > supported for Parquet? > > Regards, > > Bryan Jeffrey
