Re: The draft of the Spark 3.3.0 release notes

2022-06-03 Thread Bjørn Jørgensen
- Support lambda `column` parameter of `DataFrame.rename`(SPARK-38763 ) This did work before 18.jan.2022, see JIRA post for more information. I think we can remove this one from the list. fre. 3. jun. 2022 kl. 20:42 skrev Dongjoon Hyun

Re: The draft of the Spark 3.3.0 release notes

2022-06-03 Thread Dongjoon Hyun
You are right. After SPARK-36837, we tried to ship Apache Spark 3.3.0 with Apache Kafka 3.1.1 via the following PR. https://github.com/apache/spark/pull/36135 [WIP][SPARK-38850][BUILD] Upgrade Kafka to 3.1.1 However, the final decision was to revert it from `branch-3.3` and move directly to Apac

Re: The draft of the Spark 3.3.0 release notes

2022-06-03 Thread Koert Kuipers
i thought SPARK-36837 didnt make it in? i see it in notes On Fri, Jun 3, 2022 at 4:31 AM Maxim Gekk wrote: > Hi All, > > I am preparing the release notes of Spark 3.3.0. Here is a draft document: > > https://docs.google.com/document/d/1gGySrLGvIK8bajKdGjTI_mDqk0-YPvHmPN64YjoWfOQ/edit?usp=sharing

Re: The draft of the Spark 3.3.0 release notes

2022-06-03 Thread Adam Binford
It's more of a missing feature than a bug, but I guess it's all subjective. Adam On Fri, Jun 3, 2022 at 10:24 AM Maxim Gekk wrote: > Hi Adam, > > The release notes focus on new features and user-facing improvements > mostly. SPARK-37618 seems

Re: The draft of the Spark 3.3.0 release notes

2022-06-03 Thread Maxim Gekk
Hi Adam, The release notes focus on new features and user-facing improvements mostly. SPARK-37618 seems like a bug fix, that's why I didn't put it in the doc. Maxim Gekk Software Engineer Databricks, Inc. On Fri, Jun 3, 2022 at 2:20 PM Adam

Re: The draft of the Spark 3.3.0 release notes

2022-06-03 Thread Adam Binford
I don't think I see https://issues.apache.org/jira/browse/SPARK-37618 which got merged post branch cut. Adam On Fri, Jun 3, 2022 at 4:25 AM Maxim Gekk wrote: > Hi All, > > I am preparing the release notes of Spark 3.3.0. Here is a draft document: > > https://docs.google.com/document/d/1gGySrLGv

The draft of the Spark 3.3.0 release notes

2022-06-03 Thread Maxim Gekk
Hi All, I am preparing the release notes of Spark 3.3.0. Here is a draft document: https://docs.google.com/document/d/1gGySrLGvIK8bajKdGjTI_mDqk0-YPvHmPN64YjoWfOQ/edit?usp=sharing Please take a look and let me know if I missed any major changes or something. Maxim Gekk Software Engineer Databr