Re: Kyro Serializer not getting set : Spark3

2022-09-23 Thread rajat kumar
Thanks Qian and Vinod for response. Yes , legacy way works same as for other properties, was thinking if there is any other way. Thanks Rajat On Fri, Sep 23, 2022 at 7:43 AM Qian SUN wrote: > Hi rajat > > I’m guessing you are setting the configuration at runtime, and correct me > if I’m wrong.

Re: Kyro Serializer not getting set : Spark3

2022-09-22 Thread Qian SUN
Hi rajat I’m guessing you are setting the configuration at runtime, and correct me if I’m wrong. Only certain subset of Spark SQL properties (prefixed with spark.sql) can be set on runtime, please refer to SparkConf.scala