Hi All, I'd like to request a few days extension to the code freeze to complete the upgrade to Apache Arrow 0.10.0, SPARK-23874. This upgrade includes several key improvements and bug fixes. The RC vote just passed this morning and code changes are complete in https://github.com/apache/spark/pull/21939. We just need some time for the release artifacts to be available. Thoughts?
Thanks, Bryan On Wed, Aug 1, 2018, 5:34 PM shane knapp <skn...@berkeley.edu> wrote: > ++ssuchter (who kindly set up the initial k8s builds while i hammered on > the backend) > > while i'm pretty confident (read: 99%) that the pull request builds will > work on the new ubuntu workers: > > 1) i'd like to do more stress testing of other spark builds (in progress) > 2) i'd like to reimage more centos workers before moving the PRB due to > potential executor starvation, and my lead sysadmin is out until next monday > 3) we will need to get rid of the ubuntu-specific k8s builds and merge > that functionality in to the existing PRB job. after that: testing and > babysitting > > regarding (1): if these damn builds didn't take 4+ hours, it would be > going a lot quicker. ;) > regarding (2): adding two more ubuntu workers would make me comfortable > WRT number of available executors, and i will guarantee that can happen by > EOD on the 7th. > regarding (3): this should take about a day, and realistically the > earliest we can get this started is the 8th. i haven't even had a chance > to start looking at this stuff yet, either. > > if we push release by a week, i think i can get things sorted w/o > impacting the release schedule. there will still be a bunch of stuff to > clean up from the old centos builds (specifically docs, packaging and > release), but i'll leave the existing and working infrastructure in place > for now. > > shane > > On Wed, Aug 1, 2018 at 4:39 PM, Erik Erlandson <eerla...@redhat.com> > wrote: > >> The PR for SparkR support on the kube back-end is completed, but waiting >> for Shane to make some tweaks to the CI machinery for full testing support. >> If the code freeze is being delayed, this PR could be merged as well. >> >> On Fri, Jul 6, 2018 at 9:47 AM, Reynold Xin <r...@databricks.com> wrote: >> >>> FYI 6 mo is coming up soon since the last release. We will cut the >>> branch and code freeze on Aug 1st in order to get 2.4 out on time. >>> >>> >> > > > -- > Shane Knapp > UC Berkeley EECS Research / RISELab Staff Technical Lead > https://rise.cs.berkeley.edu >
