[
https://issues.apache.org/jira/browse/SOLR-15250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris M. Hostetter updated SOLR-15250:
--------------------------------------
Attachment: SOLR-15250.patch
Status: Open (was: Open)
I started experimenting with this idea based on the "tepid" (at best) initial
response from the docker-library team in
[https://github.com/docker-solr/docker-solr/issues/368] (and the complete lack
of response to my follow up questions seeking more clarifications).
My impression is that we're not going to be able to continue to pursue the
approach advocated in SOLR-15129 for "official" {{_/solr}} docker images.
The attached patch starts with a hybrid of a couple of existing ideas:
* From SOLR-15129: put all the docker specific scripts needed in the final
docker image into the released solr.tgz
** Anything/Everything solr related that is needed in the final docker image
comes from solr.tgz
* From the existing docker-solr Dockerfile.template:
** add in SHA512 validation of the solr.tgz, based on a build ARG
** add in GPG validation logic, but with the option to skip it based on some
build ARGs
** loop over a configured set of "download urls" until we find the solr.tgz in
our context
* Similar to SOLR-15127: if the solr.tgz is already in the build context we
can skip this download loop
** when building from gradle, use a build context that does in fact include
the locally built solr.tgz – so no download is attempted
Completely new additions:
* gradle task implementing the "templating" logic for building the
{{Dockerfile.official}}
** looks for a few specific {{ARG}} declarations with blank values, and fills
them in with the expected values based on current build properties
** fails the build if the expected {{ARG}} lines aren't found exactly once
----
If people like this approach, the next steps would be:
* get sign off from docker-library folks on using multistage builds in our
official Dockerfile
** the core ideas in this jira don't _require_ using a multistage build, but
since the Dockerfile on the main branch already uses multi-stage builds to keep
the images size down I left it in when incorporating the additions from the
exsting docker-solr {{Dockerfile-varsolr.template}}
** If that's a deal breaker with docker-library we should rip it out sooner
rather then later
* add new gradle tasks/scripts to incorporate validation of the "generated"
{{Dockerfile.official}} using a mock download server (see nocommits in
build.gradle)
* decide if we want to keep docker-solr's current approach of including the
Release Manager's public key ID directly in the Dockerfile and fetching that
key from third party keyserver directories – OR – if we want to make the GPG
validation based on the canonical {{KEYS}} file for the Solr project
** I'm in favor of switching to downloading our {{KEYS}} file – not only
because I like reducing our dependency on third-party services, but also
because it would help us simplify how much templating we have to do (the
{{KEYS}} url is fixed and wouldn't change per release like the RM's public key
ID)
** If we want to keep docker-solr's current approach, then the templating
logic needs to know what the RM's GPG key ID is ... which would probably be
blocked until Solr's overall gradle build gets updated with release helper
tasks (similar to what the ant build has on branch_8x) for signing the release
artifacts – so we can have our docker templating task depend on those and look
at the key used.
*** I suspect the only reason that docker-solr works that way in the 8.x
releases is because for a long time we had a unique {{KEYS}} file per release –
but since that shouldn't be an issue for 9.x releases we can switch to a single
canonical URL for all pulic keys of all Release Managers
* regardless of what we decide regarding how we get the keys for GPG
validation, we once the gradle build supports signing solr.tgz, we can have our
Dockerfile validation tasks check all the GPG logic in the Dockerfile as well
** again by using mock server to override the solr.tgz.asc url (which would
need to become a new ARG)
** this validation could be run manualy by Release Managers – but if we have a
way to generate a 'mock key' then even Jenkins/CI builds can validate the logic
* decide on & resolve a few other misc nocommits: better name for new gradle
task(s); possible support for stripping _some_ specific comments in the
Dockerfile template that won't be relevant in the final
{{Dockerfile.official}}; if we should rename the Dockerfile(s) for clarity; and
where exactly the docker-scripts should live in the solr.tgz (I'm not a fan of
where SOLR-15129 suggests putting them)
> Dockerfile for local builds that can also serve as template for 'official'
> docker images
> ----------------------------------------------------------------------------------------
>
> Key: SOLR-15250
> URL: https://issues.apache.org/jira/browse/SOLR-15250
> Project: Solr
> Issue Type: Sub-task
> Reporter: Chris M. Hostetter
> Assignee: Chris M. Hostetter
> Priority: Major
> Attachments: SOLR-15250.patch
>
>
> This issue tracks PoC work experimenting with the idea of the following
> workflow:
> For Users:
> * a {{Dockerfile}} (or {{Dockerfile.local}}) in our git repo that can be
> used (directly or via gradle) to build docker images directly from a local
> solr.tgz (in the docker build context)
> For Release Manager:
> * The exact same {{Dockerfile}} serves as a "template" that gradle tasks use
> to generate a {{build/Dockerfile.official}} via some very simple
> substitutions to fill in ARG defaults based on the "official"
> solr-VERSION.tgz for this release
> * This {{Dockerfile.official}} can then be committed to the docker-solr
> github repo (or some similar new 9.x+ repo) and can be build with a a
> completely empty build context – in which it downloads (and validates) the
> official solr-VERSION.tgz (based on the ARG values that were filled in during
> the release)
> * Automated tests can help us "validate" that the generated
> {{Dockerfile.official}} will work _prior_ to officially publishing release
> artifacts, by using a "mock" download server to host the local
> solr-VERSION.tgz file
> The driving goal being that the Dockerfile used for official {{_/solr}}
> builds should be as close as possible to identical to the Dockerfile used for
> "local" builds by users – given the constraints put on us by the
> docker-library team.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
