++ssuchter (who kindly set up the initial k8s builds while i hammered on the backend)
while i'm pretty confident (read: 99%) that the pull request builds will work on the new ubuntu workers: 1) i'd like to do more stress testing of other spark builds (in progress) 2) i'd like to reimage more centos workers before moving the PRB due to potential executor starvation, and my lead sysadmin is out until next monday 3) we will need to get rid of the ubuntu-specific k8s builds and merge that functionality in to the existing PRB job. after that: testing and babysitting regarding (1): if these damn builds didn't take 4+ hours, it would be going a lot quicker. ;) regarding (2): adding two more ubuntu workers would make me comfortable WRT number of available executors, and i will guarantee that can happen by EOD on the 7th. regarding (3): this should take about a day, and realistically the earliest we can get this started is the 8th. i haven't even had a chance to start looking at this stuff yet, either. if we push release by a week, i think i can get things sorted w/o impacting the release schedule. there will still be a bunch of stuff to clean up from the old centos builds (specifically docs, packaging and release), but i'll leave the existing and working infrastructure in place for now. shane On Wed, Aug 1, 2018 at 4:39 PM, Erik Erlandson <eerla...@redhat.com> wrote: > The PR for SparkR support on the kube back-end is completed, but waiting > for Shane to make some tweaks to the CI machinery for full testing support. > If the code freeze is being delayed, this PR could be merged as well. > > On Fri, Jul 6, 2018 at 9:47 AM, Reynold Xin <r...@databricks.com> wrote: > >> FYI 6 mo is coming up soon since the last release. We will cut the branch >> and code freeze on Aug 1st in order to get 2.4 out on time. >> >> > -- Shane Knapp UC Berkeley EECS Research / RISELab Staff Technical Lead https://rise.cs.berkeley.edu
