Hello,
We're at the point of implementing the tarballs.openstack.org
publishing changes from [1], and I would just like to propose some
low-level plans for feedback that exceeded the detail in the spec.
We currently have tarballs.opendev.org which publishes content from
/afs/openstack.org/project/opendev.org/tarballs. This is hosted on
the physical server files02.openstack.org and managed by puppet [2].
1) I propose we move tarballs.opendev.org to be served by
static01.opendev.org and configured via ansible
Because:
* it's one less thing running on a Xenial host with puppet we don't
want to maintain.
* it will be alongside tarballs.openstack.org per below
The /afs/openstack.org/project/opendev.org directory is currently a
single AFS volume "projet.opendev" and contains subdirectories:
docs tarballs www
opendev.org jobs currently write their tarball content into the AFS
location, which is periodically "vos released" by [3].
2) I propose we make a separate volume, with separate quota, and
mount it at /afs/openstack.org/project/tarballs.opendev.org. We
copy the current data to that location and modify the opendev.org
tarball publishing jobs to use that location, and setup the same
periodic release.
Because:
* Although currently the volume is tiny (<100mb), it will become
quite large when combined with ~140gb of openstack repos
* this seems distincly separate from docs and www data
* we have content for other hosts at /afs/openstack.org/project like
this, it fits logically.
The next steps are described in the spec; with this in place, we copy
the current openstack tarballs from
static.openstack.org:/srv/static/tarballs to
/afs/openstack.org/project/tarballs.opendev.org/openstack/
We then update the openstack tarball publishing jobs to publish to
this new location via AFS (we should be able to make this happen in
parallel, initially).
Finally, we need to serve these files.
3) I propose we make tarballs.openstack.org a vhost on
static.opendev.org that serves the
/afs/openstack.org/project/tarballs.opendev.org/openstack/
directory.
Because
* This is transparent for tarballs.openstack.org; all URLs work with
no redirection, etc.
* anyone hitting tarballs.opendev.org will see top-level project
directories (openstack, zuul, airship, etc.) which makes sense.
I think this will get us where we want to be.
Any feedback welcome, thanks. We will keep track of things in [4].
[1] https://docs.opendev.org/opendev/infra-specs/latest/specs/retire-static.html
[2]
https://opendev.org/opendev/system-config/src/branch/master/manifests/site.pp#L441
[3]
https://opendev.org/opendev/system-config/src/branch/master/modules/openstack_project/files/openafs/release-volumes.py
[4] https://storyboard.openstack.org/#!/story/2006598
_______________________________________________
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
