Related thread: http://search-hadoop.com/m/q3RTtSfi342nveex1&subj=RE+NPE+when+using+Joda+DateTime
FYI On Fri, Jan 22, 2016 at 6:50 AM, Spencer, Alex (Santander) < [email protected]> wrote: > Hi Andy, > > Sorry this is in Scala but you may be able to do something similar? I use > Joda's DateTime class. I ran into a lot of difficulties with the > serializer, but if you are an admin on the box you'll have less issues by > adding in some Kryo serializers. > > import org.joda.time > > val dateFormat = format.DateTimeFormat.forPattern("yyyy-MM-dd"); > val tranDate = dateFormat.parseDateTime(someDateString) > > > Alex > > -----Original Message----- > From: Andrew Holway [mailto:[email protected]] > Sent: 21 January 2016 19:25 > To: [email protected] > Subject: Date / time stuff with spark. > > Hello, > > I am importing this data from HDFS into a data frame with > sqlContext.read.json(). > > {“a": 42, “a": 56, "Id": "621368e2f829f230", “smunkId": > "CKm26sDMucoCFReRGwodbHAAgw", “popsicleRange": "17610", "time": > "2016-01-20T23:59:53+00:00”} > > I want to do some date/time operations on this json data but I cannot find > clear documentation on how to > > A) specify the “time” field as a date/time in the schema. > B) the format the date should be in to be correctly in the raw data for an > easy import. > > Cheers, > > Andrew > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] For additional > commands, e-mail: [email protected] > > Emails aren't always secure, and they may be intercepted or changed after > they've been sent. Santander doesn't accept liability if this happens. If > you > think someone may have interfered with this email, please get in touch > with the > sender another way. This message doesn't create or change any contract. > Santander doesn't accept responsibility for damage caused by any viruses > contained in this email or its attachments. Emails may be monitored. If > you've > received this email by mistake, please let the sender know at once that > it's > gone to the wrong person and then destroy it without copying, using, or > telling > anyone about its contents. > Santander UK plc Reg. No. 2294747 and Abbey National Treasury Services plc > Reg. > No. 2338548 Registered Offices: 2 Triton Square, Regent's Place, London > NW1 3AN. > Registered in England. www.santander.co.uk. Authorised by the Prudential > Regulation Authority and regulated by the Financial Conduct Authority and > the > Prudential Regulation Authority. FCA Reg. No. 106054 and 146003 > respectively. > Santander Sharedealing is a trading name of Abbey Stockbrokers Limited > Reg. No. > 02666793. Registered Office: Kingfisher House, Radford Way, Billericay, > Essex > CM12 0GZ. Authorised and regulated by the Financial Conduct Authority. FCA > Reg. > No. 154210. You can check this on the Financial Services Register by > visiting > the FCA’s website www.fca.org.uk/register or by contacting the FCA on > 0800 111 > 6768. Santander UK plc is also licensed by the Financial Supervision > Commission > of the Isle of Man for its branch in the Isle of Man. Deposits held with > the > Isle of Man branch are covered by the Isle of Man Depositors’ Compensation > Scheme as set out in the Isle of Man Depositors’ Compensation Scheme > Regulations > 2010. In the Isle of Man, Santander UK plc’s principal place of business > is at > 19/21 Prospect Hill, Douglas, Isle of Man, IM1 1ET. Santander and the > flame logo > are registered trademarks. > Santander Asset Finance plc. Reg. No. 1533123. Registered Office: 2 Triton > Square, Regent’s Place, London NW1 3AN. Registered in England. Santander > Corporate & Commercial is a brand name used by Santander UK plc, Abbey > National > Treasury Services plc and Santander Asset Finance plc. > Ref:[PDB#1-4A] >
