About LocalProperty in sqlConf

2018-12-22 Thread JiaTao Tao
Hi all As we know, RDD can set property locally("SparkContext.setLocalProperty"). But Spark SQL can only set sqlConf globally, does anyone knows how I can set the local property in Spark SQL. -- Regards! Aron Tao

Async action in Dataframe

2018-12-22 Thread JiaTao Tao
Hi all As we all know, RDD has the operation: "collectAsync()", also submitJob can return a Future, but I cannot find the same thing in Dataset, anyone knows how I can archive this when using "Dataframe"? Thanks a lot. -- Regards! Aron Tao