On 08/11/2015 09:42 AM, Brian Rosmaita wrote:
On 8/7/15, 1:07 PM, "Jay Pipes" <jaypi...@gmail.com> wrote:
So, here's the crux of the issue. Nova and Cinder **do not want to speak
the Glance REST API** to either upload or download image bits from
storage. Streaming image bits through the Glance API endpoint is a
needless and inefficient step, and Nova and Cinder would like to
communicate directly with the backend storage systems.
Exactly why do you want to communicate directly with the backend storage
systems? Streaming image bits through Glance appears to be "needless and
inefficient", but if an end-user is booting 1K instances from some custom
image, Glance's image cache makes an enormous difference in delivery time.
Nova's image cache makes a bigger difference overall in those cases,
since the image will most likely be cached on compute nodes themselves
and won't need to be copied at all.
Having the image bits streaming through an unrelated endpoint (the
Glance API server) is just not required if the logic for grabbing the
"closest" image from a set of locations is all in the glance_store
library and different nova-compute daemons can just efficiently grab the
image from multiple locations if the image is stored in multiple
locations in backend storage.
In addition, Glance's image cache, while excellent for improving
performance of first-pull of images from slower backend storage like
Swift, also requires the operator to have a bunch of disk space used on
the controller nodes that run glance-api. In many deployments that I
know of, the controller nodes do not run stateful services (DB and MQ
are on separate node clusters entirely from nova-api, glance-api,
cinder-api, etc), and because of this, don't have a large root disk
(sometimes nothing more than a small SSD for the OS kernel and some
swap). Setting up Glance's image cache on these types of nodes means you
need to be careful not to run out of local disk space, since a single
popular Windows image can easily be 20-40+ GB. In addition to that, each
glance-api server is going to have its own image cache, not all with the
same images in them, since different requests will be routed to
different glance-api servers, and each image cache is its own LRU layout.
Having the image cache local to the compute nodes themselves gives the
best performance overall, and with glance_store, means that glance-api
isn't needed at all, and Glance can become just a metadata repository,
which would be awesome, IMHO.
Best,
-jay
So I'm curious about what exactly the use cases for direct backend storage
communication are, and why Glance can't meet them.
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
