Dear devs. I'd like to propose new CI system for the Zeppelin project which is testing in NFLabs.
I knew in some cases current build system (travis) can not help developping Zeppelin. 1. It can't clsasloading test at the spark cluster because of now CI environment is not the actual environment. (zeppelin and Spark is running in the same host.) 2. It can't test on the various Operating System and JDK version environments. (only build in the travis host OS and single version of JDK) 3. It can't test on the various spark cluster environments. (now only build with the spark standalone.) 4. It can't parallel test because of build time. (now spark standalone test takes 40 minutes.) *. The proposed CI system can 1. clsasloading test at the spark cluster in actual environment by testing Zeppelin in the docker container completely separated with Spark cluster. (Spark cluster also runs on docker.) 2. test on the various Operating System and JDK version environments. 3. test on the various spark cluster environments. - now we build zeppelin with the spark standalone and yarn and mesos backends. 4. run parallel build - It takes less than 20 minutes when we build the zeppelin with the spark standalone and yarn and mesos backends concurrently. *. Build system introduction. Build system is simply composed of the Jenkins and Docker and some scripts. You can get the informations (how to run and how to add new build features etc..) here(https://github.com/NFLabs/zeppelin-ci). *. Further discussion 1. How this proposed CI system can integrated to current Zeppelin github repository (hook). 2. To run this CI system, we'll need infrastructrure. I hope it helps both zeppelin developers and users. Please share your opinion. Thanks in advance!
