This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git
The following commit(s) were added to refs/heads/trunk by this push:
new cb77538 Deploy multi-architecture (amd+arm) docker testing images
cb77538 is described below
commit cb77538bc28c546ccac1078ffb658d313142e43e
Author: Mick Semb Wever <[email protected]>
AuthorDate: Mon Aug 9 14:03:13 2021 +0200
Deploy multi-architecture (amd+arm) docker testing images
patch by Mick Semb Wever; reviewed by Brandon Williams for CASSANDRA-16833
---
docker/testing/README.md | 28 +++++++++++++---------------
docker/testing/ubuntu2004_j11.docker | 4 ++--
jenkins-dsl/cassandra_job_dsl_seed.groovy | 14 --------------
3 files changed, 15 insertions(+), 31 deletions(-)
diff --git a/docker/testing/README.md b/docker/testing/README.md
index 195f95e..96ce247 100644
--- a/docker/testing/README.md
+++ b/docker/testing/README.md
@@ -5,33 +5,31 @@ Docker files in this directory are used to build images used
by ci-cassandra.apa
* Base image for Linux distribution to use for testing (e.g.
`ubuntu1910_j11.docker`)
* Caching image that contains git sources, maven and ccm dependencies
-## Building Images
+## Preparing Multi-Architecture Building
+
+ docker buildx create --name mybuilder
+ docker buildx use mybuilder
+ docker buildx inspect
+
+
+This is based on the documentation found
[here](https://www.docker.com/blog/multi-arch-images/).
+
+## Building and Publishing Images
Build images from the parent directory using the following commands. Change
tag (`-t`) as needed (prefix and current date):
Base image:
-`docker build -t apache/cassandra-testing-ubuntu2004-java11:$(date +"%Y%m%d")
-t apache/cassandra-testing-ubuntu2004-java11:latest -f ubuntu2004_j11.docker .`
+ docker buildx build --platform linux/amd64,linux/arm64 -t
apache/cassandra-testing-ubuntu2004-java11:$(date +"%Y%m%d") -t
apache/cassandra-testing-ubuntu2004-java11:latest -f ubuntu2004_j11.docker
--push .
Caching image:
-`docker build --no-cache -t
apache/cassandra-testing-ubuntu2004-java11-w-dependencies:$(date +"%Y%m%d") -t
apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest -f
ubuntu2004_j11_w_dependencies.docker .`
+ docker buildx build --platform linux/amd64,linux/arm64 --no-cache -t
apache/cassandra-testing-ubuntu2004-java11-w-dependencies:$(date +"%Y%m%d") -t
apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest -f
ubuntu2004_j11_w_dependencies.docker --push .
Please make sure to always tag also by date, so we can go back to that version
in case anything breaks after the next update!
-## Publishing Images
-
We are using Docker Hub for storing published images. See [Quick Start
Guide](https://docs.docker.com/docker-hub/) on how to setup docker.
-Push both image references:
-
-```
-docker push apache/cassandra-testing-ubuntu2004-java11:$(date +"%Y%m%d")
-docker push apache/cassandra-testing-ubuntu2004-java11:latest
-docker push apache/cassandra-testing-ubuntu2004-java11-w-dependencies:$(date
+"%Y%m%d")
-docker push apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
-```
-
## Updating circleci.yml
-Check if the correct image is used in `cassandra_job_dsl_seed.groovy` and
`.circleci/config*.yml`. It should either be set to the date dervived tag
created above, or `:latest`.
+Check if the correct image is used in `cassandra_job_dsl_seed.groovy` and
`.circleci/config*.yml`. It should either be set to the date derived tag
created above, or `:latest`.
diff --git a/docker/testing/ubuntu2004_j11.docker
b/docker/testing/ubuntu2004_j11.docker
index 48bc812..5540715 100644
--- a/docker/testing/ubuntu2004_j11.docker
+++ b/docker/testing/ubuntu2004_j11.docker
@@ -26,11 +26,11 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
RUN export DEBIAN_FRONTEND=noninteractive && \
add-apt-repository -y ppa:deadsnakes/ppa && \
apt-get update && \
- apt-get install -y git-core python2.7 python3-pip \
+ apt-get install -y git-core python2.7 python-dev python3-pip \
python3.6 python3.6-venv python3.6-dev \
python3.7 python3.7-venv python3.7-dev \
python3.8 python3.8-venv python3.8-dev \
- net-tools libev4 libev-dev wget gcc libssl-dev
+ net-tools libev4 libev-dev wget gcc libssl-dev libxml2 libxslt1-dev
RUN update-alternatives --install /usr/bin/python python /usr/bin/python3.6 2
RUN update-alternatives --install /usr/bin/python python /usr/bin/python3.7 3
diff --git a/jenkins-dsl/cassandra_job_dsl_seed.groovy
b/jenkins-dsl/cassandra_job_dsl_seed.groovy
index 8c69f96..36809f3 100644
--- a/jenkins-dsl/cassandra_job_dsl_seed.groovy
+++ b/jenkins-dsl/cassandra_job_dsl_seed.groovy
@@ -564,13 +564,6 @@ cassandraBranches.each {
node / scm / branches /
'hudson.plugins.git.BranchSpec' / name(branchName)
}
steps {
- if (arch == "-arm64") {
- shell("""
- # docker image has to be built on arm64
(they are not currently published to dockerhub)
- cd cassandra-builds/docker/testing ;
- docker build -t ${dtestDockerImage}:latest
-f ubuntu2004_j11.docker .
- """)
- }
shell("""
./cassandra-builds/build-scripts/cassandra-dtest-pytest-docker.sh apache
${branchName} https://github.com/apache/cassandra-dtest.git trunk ${buildsRepo}
${buildsBranch} ${dtestDockerImage} ${targetName} \${split}/${splits} ;
wget --retry-connrefused --waitretry=1
"\${BUILD_URL}/timestamps/?time=HH:mm:ss&timeZone=UTC&appendLog" -qO - >
console.log.xz || echo wget failed
@@ -1020,13 +1013,6 @@ archs.each {
echo "cassandra-builds at: `git -C cassandra-builds
log -1 --pretty=format:'%h %an %ad %s'`" ;
echo "Cassandra-devbranch-${targetArchName})
cassandra: `git log -1 --pretty=format:'%h %an %ad %s'`" >
Cassandra-devbranch-${targetArchName}.head ;
""")
- if (arch == "-arm64") {
- shell("""
- # docker image has to be built on arm64 (they are
not currently published to dockerhub)
- cd cassandra-builds/docker/testing ;
- docker build -t \$DOCKER_IMAGE:latest -f
ubuntu2004_j11.docker .
- """)
- }
shell("""
./cassandra-builds/build-scripts/cassandra-dtest-pytest-docker.sh \$REPO
\$BRANCH \$DTEST_REPO \$DTEST_BRANCH ${buildsRepo} ${buildsBranch}
\$DOCKER_IMAGE ${targetName} \${split}/${splits} ;
wget --retry-connrefused --waitretry=1
"\${BUILD_URL}/timestamps/?time=HH:mm:ss&timeZone=UTC&appendLog" -qO - >
console.log.xz || echo wget failed
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
