Re: Spark SQL and Kryo registration

2016-08-17 Thread Olivier Girardot
Hi everyone, it seems that it works now out of the box. So nevermind, registration is compatible with spark 2.0 when using dataframes. Regards, Olivier. On Fri, Aug 5, 2016 10:07 AM, Maciej Bryński mac...@brynski.pl wrote: Hi Olivier, Did you check performance of Kryo ? I have observations th

Re: Spark SQL and Kryo registration

2016-08-05 Thread Maciej Bryński
Hi Olivier, Did you check performance of Kryo ? I have observations that Kryo is slightly slower than Java Serializer. Regards, Maciek 2016-08-04 17:41 GMT+02:00 Amit Sela : > It should. Codegen uses the SparkConf in SparkEnv when instantiating a new > Serializer. > > On Thu, Aug 4, 2016 at 6:14

Re: Spark SQL and Kryo registration

2016-08-04 Thread Amit Sela
It should. Codegen uses the SparkConf in SparkEnv when instantiating a new Serializer. On Thu, Aug 4, 2016 at 6:14 PM Jacek Laskowski wrote: > Hi Olivier, > > I don't know either, but am curious what you've tried already. > > Jacek > > On 3 Aug 2016 10:50 a.m., "Olivier Girardot" < > o.girar...@

Re: Spark SQL and Kryo registration

2016-08-04 Thread Jacek Laskowski
Hi Olivier, I don't know either, but am curious what you've tried already. Jacek On 3 Aug 2016 10:50 a.m., "Olivier Girardot" < o.girar...@lateral-thoughts.com> wrote: > Hi everyone, > I'm currently to use Spark 2.0.0 and making Dataframes work with kryo. > registrationRequired=true > Is it eve

Spark SQL and Kryo registration

2016-08-03 Thread Olivier Girardot
Hi everyone, I'm currently to use Spark 2.0.0 and making Dataframes work with kryo.registrationRequired=true Is it even possible at all considering the codegen ? Regards, Olivier Girardot | Associé o.girar...@lateral-thoughts.com +33 6 24 09 17 94