[FYI] Kubernetes GA Preparation (SPARK-33005)

Hi, All. K8s GA preparation is on the way like the following. https://issues.apache.org/jira/browse/SPARK-33005 Apache Spark 3.1/3.2 is scheduled for December 2020 and mid of 2021 (TBD). If you hit K8s issues, please file a JIRA issue. To give more visibility to your issue, you can create yo

Re: What's the root cause of not supporting multiple aggregations in structured streaming?

Thanks for the great discussion! Also interested in this feature and did some investigation before. As Arun mentioned, similar to the "update" mode, the "complete" mode also needs more design. We might need an operation level output mode for the complete mode support. That is to say, if we use "co

Re: [FYI] Kubernetes GA Preparation (SPARK-33005)

Thanks, Dongjon, I pined two long-standing issues to the umbrella. https://issues.apache.org/jira/browse/SPARK-28895 https://issues.apache.org/jira/browse/SPARK-28992 This helps fix some problems of running Spark on k8s application with HDFS. Hoping to get reviewed soon Best