Re: Extracting Logical Plan

2023-08-02 Thread Ruifeng Zheng
if I use the Spark > connect? I am not using the regular SparkSession. I am pretty new to these > APIs. Appreciate your thoughts. > > On Wed, Aug 2, 2023 at 2:00 PM Ruifeng Zheng wrote: > >> Hi Vibhatha, >>I think those APIs are

Re: Extracting Logical Plan

2023-08-02 Thread Ruifeng Zheng
he Spark version to 3.4.1 and I have encountered >>>>> a few issues. In my previous code, I was able to extract the logical plan >>>>> using `df.queryExecution` (df: DataFrame and in Scala), but it seems like >>>>> in the latest API it is not supported. Is there a way to extract the >>>>> logical plan or optimized plan from a dataframe or dataset in Spark 3.4.1? >>>>> >>>>> Best, >>>>> Vibhatha >>>>> >>>> -- >>> Vibhatha Abeykoon >>> >> -- > Vibhatha Abeykoon > -- Ruifeng Zheng E-mail: zrfli...@gmail.com

回复:[ANNOUNCE] Apache Spark 3.2.1 released

2022-01-28 Thread Ruifeng Zheng
It's Great! Congrats and thanks, huaxin! -- 原始邮件 -- 发件人: "huaxin gao"