Much thanks for your work on infra @Shane. Especially, we (I and @huangtianhua) got really much help from you when make Arm CI work. [1]
> prepare jenkins worker ansible configs and stick in the spark repo https://github.com/apache/spark/pull/32178 I take a quick glance on it, it seems it doesn't contain any Arm node setup and config related code. *Do you have any plan to update the existing code to cover the Arm node setup and configuration?* or just some exiting script is also okay. *Do you have any special plan on Arm node migration?* If needed, I will help some the Arm related node setup and config in new infra to make sure Spark Arm CI work. BTW, We also is considering to move the Arm build from jenkins to Github Action (using self-host or cloud deploy https://github.com/actions/starter-workflows/tree/main/ci), there are some pre-work is being done by our team see PoC in [2]. (cc @mgrigorov)[2], maybe it could bring some idea on future infrastructure. [1] https://amplab.cs.berkeley.edu/jenkins/label/spark-arm/ [2] https://martin-grigorov.medium.com/githubactions-build-and-test-on-huaweicloud-arm64-af9d5c97b766 Regards, Yikun Holden Karau <hol...@pigscanfly.ca> 于2021年4月15日周四 上午8:29写道： > Thanks Shane for keeping the build infra structure running for all of > these years :) > > I've got some Kubernetes infra on AS399306 down in HE in Fremont but > it's also perhaps not of the newest variety, but so far no disk > failures or anything like that (knock on wood of course). The catch is > it's on a 15 amp circuit and frankly I'm still learning how BGP works. > > Maybe we could expirement with > https://github.com/lazybit-ch/actions-runner/tree/master/actions-runner > and try nested MiniKube (which I know is... not great but might make > things more portable)? > > Would the community (and or some of our corporate contributors) be > open to contributing some hardware + power money or cloud credits? > > On Wed, Apr 14, 2021 at 5:13 PM Hyukjin Kwon <gurwls...@gmail.com> wrote: > > > > Thanks Shane!! > > > > On Thu, 15 Apr 2021, 09:03 shane knapp ☠, <skn...@berkeley.edu> wrote: > >>> > >>> medium term (in 6 months): > >>> * prepare jenkins worker ansible configs and stick in the spark repo > >>> - nothing fancy, but enough to config ubuntu workers > >>> - could be used to create docker containers for testing in > <wavey-hands>THE CLOUD</wavey-hands> > >>> > >> fwiw, i just decided to bang this out today: > >> https://github.com/apache/spark/pull/32178 > >> > >> shane > >> -- > >> Shane Knapp > >> Computer Guy / Voice of Reason > >> UC Berkeley EECS Research / RISELab Staff Technical Lead > >> https://rise.cs.berkeley.edu > > > > -- > Twitter: https://twitter.com/holdenkarau > Books (Learning Spark, High Performance Spark, etc.): > https://amzn.to/2MaRAG9 > YouTube Live Streams: https://www.youtube.com/user/holdenkarau > > --------------------------------------------------------------------- > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > >
