Re: Spark performance regression test suite

n Karau , Ted Yu , Michael Allman , dev Date: 11/07/2016 17:00 Subject:Re: Spark performance regression test suite I second any effort to update, automate, and communicate the results of spark-perf (https://github.com/databricks/spark-perf) On Fri, Jul 8, 2016 at 12:28 PM, Er

Re: Spark performance regression test suite

I second any effort to update, automate, and communicate the results of spark-perf (https://github.com/databricks/spark-perf) On Fri, Jul 8, 2016 at 12:28 PM, Eric Liang wrote: > Something like speed.pypy.org >

Re: Spark performance regression test suite

Something like speed.pypy.org or the Chrome performance dashboards would be very useful. On Fri, Jul 8, 2016 at 9:50 AM Holden Karau wrote: > There are also the spark-pe

Re: Spark performance regression test suite

There are also the spark-perf and spark-sql-perf projects in the Databricks github (although I see an open issue for Spark 2.0 support in one of them). On Friday, July 8, 2016, Ted Yu wrote: > Found a few issues: > > [SPARK-6810] Performance benchmarks for SparkR > [SPARK-2833] performance tests

Re: Spark performance regression test suite

Found a few issues: [SPARK-6810] Performance benchmarks for SparkR [SPARK-2833] performance tests for linear regression [SPARK-15447] Performance test for ALS in Spark 2.0 Haven't found one for Spark core. On Fri, Jul 8, 2016 at 8:58 AM, Michael Allman wrote: > Hello, > > I've seen a few messa

Spark performance regression test suite

Hello, I've seen a few messages on the mailing list regarding Spark performance concerns, especially regressions from previous versions. It got me thinking that perhaps an automated performance regression suite would be a worthwhile contribution? Is anyone working on this? Do we have a Jira iss