I haven't dug into this, but I agree that something that is transient isn't meant to be restored by the default Java serialization mechanism. I'd expect the class handles restoring that value as needed or in a custom readObject method. And then I don't know how Kryo interacts with that.
I don't think you need to install anything. You may end up writing your own serialization for Kryo. Try not using Kryo just to narrow it down? Hackier solution: send around long timestamps and then make them into DateTime locally as needed. Not great. Or if possible use Java 8, where Joda APIs are part of the JDK. Possibly it works then. On Fri, Jan 15, 2016 at 9:32 AM, Spencer, Alex (Santander) <alex.spen...@santander.co.uk> wrote: > Hi, > > > > I tried Zhu’s recommendation and sadly got the same error. (Again, single > map worked by the groupBy / flatMap generates this error). > > > > Does Kryo has a bug i.e. it’s not serialising all components needed, or do > I just need to get our IT team to install those magro Serializers as > suggested by Todd? If that variable is transient then actually that means > Kryo is working as it’s meant to? > > > > Am I at the point where I should pull apart the source code and build my own > DateTime class? I hate reinventing the wheel though. > > > > Thanks, > > Alex. > > --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
