On 2016-02-24 15:47:12 +0000 (+0000), Steven Dake (stdake) wrote: [...] > Where we are stumped is should we push to registry.openstack.org > which I would prefer, or push to the docker registry.
If we use a Jenkins publisher the files get pulled from the slave where they were generated to the Jenkins master server and then copied from there to the archive location. This might present some significant disk utilization issues on our Jenkins masters given the file sizes you've quoted, so I wouldn't recommend that as a solution. Another possible publication model is what we did with log uploads to Swift containers, where the Jenkins slave is given temporary constrained write credentials assigned by Zuul and directly uploads the files to the container, though index generation becomes a challenge if you need any sort of hierarchical organization (it's not like a POSIX filesystem so emulating Apache autoindex is challenging). Another major challenge for either of these is bandwidth. It's entirely possible your images are built on slaves in France and then shipped across the Atlantic to Dallas for archiving. I would not expect this to be a fast operation. > Another issue is getting the authentication credentials in the > gate securely - somehow it needs to be injected into our post jobs > without us having to check our credentials into the repository. If uploading to a third party service, the way we do that is to upload the files from the single-use slave first to a location under our control (by way of one of the mechanisms described above), and then run another job dependent on a trusted long-lived slave to retrieve the file and upload it to the final destination using preinstalled credentials for that remote service. The storage and bandwidth needs for this use case make me suspect our CI system is a poor place to attempt to implement such a solution, but hopefully others have ideas which simply aren't occurring to me. -- Jeremy Stanley __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
