Hi Mark, The shuffle memory leaks that I identified in SPARK-11239 have been around for multiple releases and it's not clear whether they have caused performance problems in real workloads, so I would say that it's fine to move the release forward without including my patch. If we have to cut another release candidate for some other reason, though, then it might be nice to include it then, but I don't think it qualifies as a release-blocker by itself.
On Sun, Oct 25, 2015 at 3:50 PM, Mark Hamstra <m...@clearstorydata.com> wrote: > Should 1.5.2 wait for Josh's fix of SPARK-11293? > > On Sun, Oct 25, 2015 at 2:25 PM, Sean Owen <so...@cloudera.com> wrote: > >> The signatures and licenses are fine. I continue to get failures in >> these tests though, with "-Pyarn -Phadoop-2.6 -Phive >> -Phive-thriftserver" on Ubuntu 15 / Java 7. >> >> - Unpersisting HttpBroadcast on executors and driver in distributed >> mode *** FAILED *** >> java.util.concurrent.TimeoutException: Can't find 2 executors before >> 10000 milliseconds elapsed >> at >> org.apache.spark.ui.jobs.JobProgressListener.waitUntilExecutorsUp(JobProgressListener.scala:561) >> at >> org.apache.spark.broadcast.BroadcastSuite.testUnpersistBroadcast(BroadcastSuite.scala:313) >> at org.apache.spark.broadcast.BroadcastSuite.org >> $apache$spark$broadcast$BroadcastSuite$$testUnpersistHttpBroadcast(BroadcastSuite.scala:238) >> at >> org.apache.spark.broadcast.BroadcastSuite$$anonfun$12.apply$mcV$sp(BroadcastSuite.scala:149) >> at >> org.apache.spark.broadcast.BroadcastSuite$$anonfun$12.apply(BroadcastSuite.scala:149) >> at >> org.apache.spark.broadcast.BroadcastSuite$$anonfun$12.apply(BroadcastSuite.scala:149) >> at >> org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22) >> at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85) >> at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104) >> at org.scalatest.Transformer.apply(Transformer.scala:22) >> ... >> - Unpersisting TorrentBroadcast on executors only in local mode >> - Unpersisting TorrentBroadcast on executors and driver in local mode >> - Unpersisting TorrentBroadcast on executors only in distributed mode >> - Unpersisting TorrentBroadcast on executors and driver in distributed >> mode *** FAILED *** >> java.util.concurrent.TimeoutException: Can't find 2 executors before >> 10000 milliseconds elapsed >> at >> org.apache.spark.ui.jobs.JobProgressListener.waitUntilExecutorsUp(JobProgressListener.scala:561) >> at >> org.apache.spark.broadcast.BroadcastSuite.testUnpersistBroadcast(BroadcastSuite.scala:313) >> at org.apache.spark.broadcast.BroadcastSuite.org >> $apache$spark$broadcast$BroadcastSuite$$testUnpersistTorrentBroadcast(BroadcastSuite.scala:287) >> at >> org.apache.spark.broadcast.BroadcastSuite$$anonfun$16.apply$mcV$sp(BroadcastSuite.scala:165) >> at >> org.apache.spark.broadcast.BroadcastSuite$$anonfun$16.apply(BroadcastSuite.scala:165) >> at >> org.apache.spark.broadcast.BroadcastSuite$$anonfun$16.apply(BroadcastSuite.scala:165) >> at >> org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22) >> at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85) >> at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104) >> at org.scalatest.Transformer.apply(Transformer.scala:22) >> ... >> >> On Sun, Oct 25, 2015 at 7:07 AM, Reynold Xin <r...@databricks.com> wrote: >> > Please vote on releasing the following candidate as Apache Spark version >> > 1.5.2. The vote is open until Wed Oct 28, 2015 at 08:00 UTC and passes >> if a >> > majority of at least 3 +1 PMC votes are cast. >> > >> > [ ] +1 Release this package as Apache Spark 1.5.2 >> > [ ] -1 Do not release this package because ... >> > >> > >> > The release fixes 51 known issues in Spark 1.5.1, listed here: >> > http://s.apache.org/spark-1.5.2 >> > >> > The tag to be voted on is v1.5.2-rc1: >> > https://github.com/apache/spark/releases/tag/v1.5.2-rc1 >> > >> > The release files, including signatures, digests, etc. can be found at: >> > http://people.apache.org/~pwendell/spark-releases/spark-1.5.2-rc1-bin/ >> > >> > Release artifacts are signed with the following key: >> > https://people.apache.org/keys/committer/pwendell.asc >> > >> > The staging repository for this release can be found at: >> > - as version 1.5.2-rc1: >> > https://repository.apache.org/content/repositories/orgapachespark-1151 >> > - as version 1.5.2: >> > https://repository.apache.org/content/repositories/orgapachespark-1150 >> > >> > The documentation corresponding to this release can be found at: >> > >> http://people.apache.org/~pwendell/spark-releases/spark-v1.5.2-rc1-docs/ >> > >> > >> > ======================================= >> > How can I help test this release? >> > ======================================= >> > If you are a Spark user, you can help us test this release by taking an >> > existing Spark workload and running on this release candidate, then >> > reporting any regressions. >> > >> > ================================================ >> > What justifies a -1 vote for this release? >> > ================================================ >> > -1 vote should occur for regressions from Spark 1.5.1. Bugs already >> present >> > in 1.5.1 will not block this release. >> > >> > =============================================================== >> > What should happen to JIRA tickets still targeting 1.5.2? >> > =============================================================== >> > Please target 1.5.3 or 1.6.0. >> > >> > >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org >> For additional commands, e-mail: dev-h...@spark.apache.org >> >> >
