New metrics for WindowExec with number of partitions and frames?

Hi, Currently WindowExec gives no metrics in the web UI's Details for Query page. What do you think about adding the number of partitions and frames? That could certainly be super useful, but am unsure if that's the kind of metrics Spark SQL shows in the details. Pozdrawiam, Jacek Laskowski

Re: New metrics for WindowExec with number of partitions and frames?

That would be useful (number of partitions). On Fri, May 26, 2017 at 3:24 PM Jacek Laskowski wrote: > Hi, > > Currently WindowExec gives no metrics in the web UI's Details for Query > page. > > What do you think about adding the number of partitions and frames? > That could certainly be super us

Re: Uploading PySpark 2.1.1 to PyPi

Hi, Holden, That sounds good to me! Thanks, Xiao 2017-05-23 16:32 GMT-07:00 Holden Karau : > An account already exists, the PMC has the info for it. I think we will > need to wait for the 2.2 artifacts to do the actual PyPI upload because of > the local version string in 2.2.1, but rest assure

Spark Issues on ORC

Hi, All. Today, while I’m looking over JIRA issues for Spark 2.2.0 in Apache Spark. I noticed that there are many unresolved community requests and related efforts over `Feature parity for ORC with Parquet`. Some examples I found are the following. I created SPARK-20901 to organize these althoug