This story [1] proposes adding a .gitlab-ci.yml file to make it easy to create artifacts and images for spark.
Using this mechanism, people can submit any subsequent version of spark for building and image hosting with gitlab.com. There is a companion WIP branch [2] with a candidate and example for doing this. The exact steps for building are in the yml file [3]. The images get published into the namespace of the user as here [4] One value of this is the ability to create versions of dependent packages such as spark-on-k8s-operator that might use upgraded packages or modifications for testing. For example, this was used to build a version of spark that included SPARK-28938 which has yet to be released and was necessary for spark-operator to work properly with GKE service accounts [5]. Comments about desirability? [1] https://issues.apache.org/jira/browse/SPARK-30275 [2] https://gitlab.com/jkleckner/spark/tree/add-gitlab-ci-yml [3] https://gitlab.com/jkleckner/spark/blob/add-gitlab-ci-yml/.gitlab-ci.yml [4] https://gitlab.com/jkleckner/spark/container_registry [5] https://gitlab.com/jkleckner/spark-on-k8s-operator/container_registry
