Re: Build issues with apache-spark-on-k8s.

2018-03-29 Thread Yinan Li
For 2.3, the dockerfile is under kubernetes/ in the tarball, not under the directory where you started the build. Once you successfully build, copy the tarball out, untar it, and you should see the directory kubernetes/ in it. On Thu, Mar 29, 2018 at 3:00 AM, Atul Sowani wrote: > Thanks all for

Re: Build issues with apache-spark-on-k8s.

2018-03-29 Thread Rob Vesse
Subject: Re: Build issues with apache-spark-on-k8s. Thanks all for responding and helping me with the build issue. I tried building the code at git://github.com/apache/spark.git (master branch) in my ppc64le Ubuntu 16.04 VM and it failed. I tried building a specific branch (branch-2.2) using

Re: Build issues with apache-spark-on-k8s.

2018-03-29 Thread Atul Sowani
Thanks all for responding and helping me with the build issue. I tried building the code at git://github.com/apache/spark.git (master branch) in my ppc64le Ubuntu 16.04 VM and it failed. I tried building a specific branch (branch-2.2) using following command: build/mvn -DskipTests -Pkubernetes cle

Re: Build issues with apache-spark-on-k8s.

2018-03-28 Thread Anirudh Ramanathan
As Lucas said, those directories are generated and copied when you run a full maven build with the -Pkubernetes flag specified (or use instructions in https://spark.apache.org/docs/latest/building-spark.html#building-a-runnable-distribution ). Also, using the Kubernetes integration in the main Ap

Re: Build issues with apache-spark-on-k8s.

2018-03-28 Thread Lucas Kacher
Are you building on the fork or on the official release now? I built v2.3.0 from source w/out issue. One thing I noticed is that I needed to run the build-image command from the bin which was placed in dist/ as opposed to the one in the repo (as that's how it copies the necessary targets). (Failed

Build issues with apache-spark-on-k8s.

2018-03-28 Thread Atul Sowani
Hi, I built apache-spark-on-k8s from source on Ubuntu 16.04 and it got built without errors. Next, I wanted to create docker images, so as explained at https://apache-spark-on-k8s.github.io/userdocs/running-on-kubernetes.html I used sbin/build-push-docker-images.sh to create those. While using th