[
https://issues.apache.org/jira/browse/CASSANDRA-21547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18101702#comment-18101702
]
Dmitry Konstantinov edited comment on CASSANDRA-21547 at 8/4/26 1:13 PM:
-------------------------------------------------------------------------
+1, thank you for taking care of it.
so, dropped from 13.5 GiB to 2 GiB of compressed image size now (35 to 4GB
uncompressed)
||layer||uncompressed||compressed||
|{{apt install}} - 5 CPythons, JDK 8/11/17, ant, tooling|1.68 GB|0.634 GiB|
|{{COPY /home/cassandra}} - ccm repo, venvs, pip cache|1.31 GB|0.951 GiB|
|{{COPY /home/image-cache}} - maven repo + gradle|540 MB|0.322 GiB|
|{{adduser cassandra-tmp}}|292 MB|0.3 MiB|
|{{apt}} bootstrap|122 MB|0.042 GiB|
|{{ubuntu:22.04}} base|78.1 MB|0.028 GiB|
|everything else (~30 layers)|~22 MB|small|
nit: we can save another 292 MB uncompressed space (but small amount of
compressed):
279 MiB of sparse log files from the rare uid (new, cheap fix)
The {{adduser}} layer is 292 MB uncompressed. Unpacking it:
{noformat}
251.1 MiB var/log/lastlog
27.5 MiB var/log/faillog
{noformat}
Both are sparse files indexed by UID, and the Dockerfile uses {{--uid 901743}}.
{{lastlog}} is 292 bytes/uid, {{faillog}} 32 bytes/uid. They are holes, so they
compress to
0.27 MiB and never show up in registry-side numbers, but Docker's tar
extraction writes the
zeros as real blocks - roughly 279 MiB of actual disk on every agent, 7% of the
uncompressed
image.
The rare uid is deliberate, so keep it and delete the logs in the same layer:
{code}
RUN adduser --disabled-login --uid 901743 --lastuid 901743 --gecos cassandra
cassandra-tmp \
&& rm -f /var/log/lastlog /var/log/faillog
{code}
was (Author: dnk):
+1, thank you for taking care of it.
so, dropped from 13.5 GiB to 2 GiB of compressed image size now (35 to 4GB
uncompressed)
> CI: reduce build and test docker image sizes using multi-stage builds
> ---------------------------------------------------------------------
>
> Key: CASSANDRA-21547
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21547
> Project: Apache Cassandra
> Issue Type: Improvement
> Components: CI
> Reporter: Michael Semb Wever
> Assignee: Michael Semb Wever
> Priority: Normal
> Fix For: 5.0.x, 7.x, 6.0.x
>
>
> cassandra-ubuntu-test is currently 37GB and slows the CI pipeline down a lot.
> Background: CASSANDRA-21541 and CASSANDRA-21523
> Do the following (to all the dockerfiles) to reduce the docker image sizes.
> 1) Use multi-stages in dockerfiles (to avoid accumulating layer sizes)
> Restructure cassandra-ubuntu-test (only) into three stages:
> - *base* - all system setup (apt packages, JDKs, python, locales,
> alternatives, java.security edits and the cassandra-tmp build user), written
> once.
> - *builder* - derives from base and does the throw-away heavy work:
> prepopulate the Maven cache and build the five python virtualenvs and the ccm
> repository.
> - *final* - re-derives from the clean base and only COPY --from=builder
> the generated caches (/home/cassandra and /home/image-cache).
> 2) Clean apt caches in-layer in same RUN layer
> 3) Fold venv chmod/cleanup into same RUN layers
> Each Python virtualenv was built in one layer, then a later `RUN chmod -R
> og+wx` and `RUN rm` touched the whole tree in separate layers. Under overlay2
> a chmod copy-up rewrites every touched file into the new layer while the
> originals still occupy the earlier layer, so the ~2GB of virtualenvs were
> effectively stored twice; the separate `rm` of the cassandra-driver .git dirs
> reclaimed nothing. Consolidate each env's virtualenv creation, pip install,
> cassandra-driver/.git removal and og+wx chmod into a single RUN so the tree
> exists in exactly one layer with the correct permissions. No functional
> change: same tools, versions and permissions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
