In addition to zone ids being opaque, they have to be anonymized. On AWS, they randomize the assignment of availability zones per account because "us-east-1a" datacenter was melting the floor tiles due to load while "us-east-1c" datacenter was idling. This causes problems for those of us who manage development and deployment accounts. Unexpected consequences of lazy cloud application developers in a hurry. As one of those lazy developers, I don't really want visibility into or have to understand the provider's physical infrastructure or even logical organization for that matter. The only value I see in zones is that spreading across them gives me fault tolerance and selecting one gives me somewhat better network performance (or saves bandwidth charges).
Exposing your topology can be a bad thing. Brian Schott bfsch...@gmail.com On Feb 11, 2011, at 3:01 PM, Vishvananda Ishaya wrote: > Yes, i think that cross-project should be requests across an admin api. I > don't think high level zone info is enough information for the scheduler > to decide where the instance should go. Especially with things like swift, > where there are three copies that may be in different zones altogether. > > Vish > On Feb 11, 2011, at 11:55 AM, Eric Day wrote: > >> Hey Vish! >> >> On Fri, Feb 11, 2011 at 12:10:14PM -0600, Vishvananda Ishaya wrote: >>> I think that providers may not wish to expose the internal structure of >>> their network to the degree that you are suggesting. I prefer the idea of >>> near other object with an internal lookup into zone. >> >> Providers can always obscure the naming to hide topology, or >> services can have options to only expose a certain depth of >> the hierarchy (ie, truncate at the 'datacenter level', hide all >> room/rack/switch zone level details). So even though volume-001 >> may be in rack12.room2.dc1.dfw.rackspace.com, the API only returns >> dc1.dfw.rackspace.com. We can of course allow for both too when the >> granularity is needed within a project: >> >> Within nova requests, allow: >> >> near-volume:volume-001 >> >> But across projects: >> in-zone: dc1.dfw.rackspace.com >> >> If we don't expose object zones at all, how would you support >> cross-project lookups? If nova gets a request 'near:swift-12345', >> would it be configured to query swift to get the zone via an admin API? >> >> We could also expose detailed object zones but keep them completely >> opaque, and instead require each service to allow pluggable >> scheduler/zone mapping. For example, a volume may return zone >> '87af1b7c', which internally all services could lookup and translate >> to where that maps in the deployment hierarchy. I was hoping DNS names >> were abstract enough to reduce complexity, but perhaps they are not. :) >> >> -Eric >> >>> On Feb 11, 2011 9:38 AM, "Eric Day" <e...@oddments.org> wrote: >>>> Hi Sandy, >>>> >>>> I agree with using tags for full scheduler selection, it's something >>>> I've been pushing for from the start. The request contains any number >>>> of k/v pairs, the services provide any number of k/v pairs, and the >>>> scheduler performs a match (some required, some optional, ...). I see >>>> the URI/zone as one of those tags, not something we need to overload >>>> to contain all of the capabilities. It should only be a hierarchical >>>> "location", which may be geographic location, organizational location >>>> (dept, ...), or some other type (however you decide to construct >>>> your zones). >>>> >>>> For example, imagine a dynamic del.icio.us tag that allowed for domain >>>> name filtering on bookmarks (give me all bookmarks with tags [book >>>> review] [domain:slashdot.org]). For Nova, this means issuing requests >>>> like "create instance with [GPU] [Fast disk] [zone:dc1.example.com]". >>>> >>>> The important thing is that this is not a tag specific to a particular >>>> service. For example, Swift would never care or need to understand a >>>> 'GPU' tag, but it can share and understand zone tags. >>>> >>>> -Eric >>>> >>>> On Fri, Feb 11, 2011 at 12:40:44PM +0000, Sandy Walsh wrote: >>>>> Heh, hate to be the one to bust up the URI love-fest :) >>>>> >>>>> The issue I have with a single URI being used as the heuristic for node >>> selection is that it is very rigid. >>>>> >>>>> Different business units have different views on the network: >>>>> * Operations may view it as geography/data centers. >>>>> * Consumers may view it as technical ability (gpu's, fast disk, good >>> inter-server speed, etc) >>>>> * Sales/marketing may view it as the number of martinis they can buy ;) >>>>> >>>>> Trees become unmanageable/hard to visualize for users beyond a couple >>> hundred nodes. We are lucky that our geographical/DC-based hierarchy is >>> relatively flat. This is why I was initially pushing for a tag-based >>> system for selection (aka Zone/Host Capabilities). >>>>> >>>>> Consider the way delicio.us works. They manage many millions of URL's >>> and tags are an effective way to slice & dice your way through the data: >>>>> "Show me all the URL's on [OpenStack] [Python] [Zones] [Scheduler]" ... >>> blam. >>>>> >>>>> This is also the way the old Trader services worked: >>>>> "I want a [wax transfer] [color] printer that can has [30ppm] and >>> [300dpi] on [Floor 2]" >>>>> >>>>> "Near" simply has to mean the distance in zones from the most-optimal >>> zones, based on the tags. >>>>> >>>>> "I want a new instance with [GPU] and [Fast Disk] [Good inter-instance >>> network speed] [near] [DRW] [DC1]" >>>>> * where "[near]" implies "as close as possible to" in zone distance. >>>>> >>>>> Personally I don't like overloading the zone name to have a >>> "meaningful" URI when we can get the same functionality with >>> Capabilities/Tags already. And we already know we need Capability support >>> anyway. Especially if it means enforcing a rigid hierarchy. >>>>> >>>>> $0.02 >>>>> >>>>> -S >>>>> >>>>> >>>>> >>>>> ________________________________________ >>>>> From: openstack-bounces+sandy.walsh=rackspace....@lists.launchpad.net >>> [openstack-bounces+sandy.walsh=rackspace....@lists.launchpad.net] on >>> behalf of Eric Day [e...@oddments.org] >>>>> Sent: Friday, February 11, 2011 4:30 AM >>>>> To: Justin Santa Barbara >>>>> Cc: openstack@lists.launchpad.net; Devin Carlen >>>>> Subject: Re: [Openstack] Allowing clients to pass capability requests >>> through tags? >>>>> >>>>> The main reason I was proposing full location/zone of objects is to >>>>> allow this type of 'near' scheduling to happen without understanding >>>>> what the actual object is. For example, imagine we want to start an >>>>> instance near a particular swift object. We could query the swift >>>>> object and in the metadata there could be a 'zone' tag (well, three, >>>>> one for each copy). For example: >>>>> >>>>> get swift-12345: zone=rack12.room2.dc1.dfw.rackspace.com >>>>> >>>>> I can now use that zone name to: >>>>> >>>>> create_instance: openstack:near=rack12.room2.dc1.dfw.rackspace.com >>>>> >>>>> The deployment can decide what 'near' is (perhaps a measure of link >>>>> speed or latency). This way a particular deployment that uses the >>>>> same URI/zone names across projects can account for locality without >>>>> knowing what objects from different services are. If it were just >>>>> 'near=swift-12345', it would need to understand what a swift object >>>>> was and perform that lookup to find out where it is. >>>>> >>>>> So you can still grab a zone tag from a volume you created: >>>>> >>>>> get vol-000001: rack4.room2.dc1.dfw.rackspace.com >>>>> >>>>> and use the zone to launch an instance with: >>>>> >>>>> create_instance: openstack:near=rack4.room2.dc1.dfw.rackspace.com >>>>> >>>>> We can also write schedulers/tools for a particular deployment >>>>> that understands the zones to just say 'always prefer in >>>>> dc1.dfw.rackspace.com', because power is cheaper there right now, or >>>>> 'test.dc1.dfw.rackspace.com' because that is my test zone (perhaps >>>>> only enabled for certain accounts in the scheduler too). >>>>> >>>>> -Eric >>>>> >>>>> On Thu, Feb 10, 2011 at 03:38:42PM -0800, Justin Santa Barbara wrote: >>>>>> I think the blueprint was largely complementary to the multi-zone >>> stuff; >>>>>> this is more about how the client _requests_ a particular >>>>>> location/capability through the API. The multi-zone blueprint seems >>> to be >>>>>> more about how nova would satisfy those requests (in a non-trivial >>> zone >>>>>> structure.) >>>>>> The root motivator is indeed getting a 'good' connection to a storage >>>>>> volume. I'm thinking of iSCSI SAN storage here, so in my case this >>>>>> probably means the SAN device with the least number of switches in >>>>>> between. There could well be SAN devices in each rack (e.g. Solaris >>>>>> volume nodes), or the devices could even be running on the host >>> nodes, and >>>>>> I don't believe that zones in the EC2 sense are sufficient here. >>>>>> But I guess that if the zone hierarchy went all the way down to the >>> rack >>>>>> (or machine), that would work. So I could create a volume and it >>> would >>>>>> come back with a location of "rack4.room2.dc1.dfw.rackspace.com" and >>> I >>>>>> could then request allocation of machines in that same rack? Is that >>> the >>>>>> vision of the nested zones? >>>>>> I do have a concern that long-term if we _only_ use zones, that's >>> trying >>>>>> to multiplex a lot of information into the zone hierarchy, and we can >>>>>> really only put one attribute in there. I also like the flexibility >>> of >>>>>> the 'openstack:near=vol-000001' request, because then the cloud can >>> decide >>>>>> how near to place the instance based on its knowledge of the >>> topology, and >>>>>> the clients can be oblivious to the storage system and arrangement. >>> But, >>>>>> my immediate requirement would indeed be satisfied if the zones went >>> down >>>>>> to the rack/machine level. >>>>>> An alternative way to look at zones and instance-types is that >>> they're >>>>>> actually just fail-if-not-satisfiable tags of the creation request >>>>>> (openstack:+zone=us-east-1a and openstack:+instancetype=m1.large) >>> They're >>>>>> only distinguished attributes because AWS doesn't have an >>>>>> extensibility mechanism, which this blueprint would give us. >>>>>> Justin >>>>>> >>>>>> On Thu, Feb 10, 2011 at 3:12 PM, Devin Carlen <devcam...@me.com> >>> wrote: >>>>>> >>>>>> I haven't totally digested this blueprint yet but it seems like there >>> is >>>>>> some overlap with what is being discussed with the multi zone >>> metadata >>>>>> stuff. One approach might be to handle this awt the scheduler level >>>>>> though and try to ensure things are always in the same zone when >>>>>> appropriate. >>>>>> I think the bigger question you raise is how to request local volumes >>>>>> when possible, yes? >>>>>> >>>>>> Devin >>>>>> On Feb 10, 2011, at 3:37 PM, Justin Santa Barbara >>> <jus...@fathomdb.com> >>>>>> wrote: >>>>>> >>>>>> Does anyone have any thoughts/objections on the blueprint I posted >>> for >>>>>> allowing clients to pass capability-requests through tags? I'm >>>>>> planning on starting implementation soon, so if people think this is >>> a >>>>>> bad idea I'd rather know before I start coding! >>>>>> Blueprint: >>> >>> https://blueprints.launchpad.net/nova/+spec/use-metadata-tags-for-capabilities >>>>>> Wiki: >>> >>> https://blueprints.launchpad.net/nova/+spec/use-metadata-tags-for-capabilities >>>>>> And a quick TLDR: >>>>>> API clients need a way to request e.g. placement of machines near >>> each >>>>>> other / near volumes, or that a volume be created with a particular >>>>>> RAID level, or that a machine be created in a HIPAA compliant >>>>>> environment. (This is complementary to the work on hierarchical zones >>>>>> & URL naming, I believe) >>>>>> I propose using the instance tags for this, e.g. specifying >>>>>> openstack:near=vol-000001 when creating an instance to request >>>>>> locating the instance 'close to' that volume. >>>>>> By default these requests would be best-effort and >>> ignored-if-unknown; >>>>>> if the client wants to specify that something is required and should >>>>>> fail if not understood or not satisfiable, they could use a "+" e.g. >>>>>> openstack:+location=*.dc1.north.rackspace.com >>>>>> Controversially (?), this would not be supported for clients using >>> the >>>>>> AWS API, because tags can only be specified once the instance has >>>>>> already been created. >>>>>> Feedback appreciated! >>>>>> Justin >>>>>> >>>>>> _______________________________________________ >>>>>> Mailing list: https://launchpad.net/~openstack >>>>>> Post to : openstack@lists.launchpad.net >>>>>> Unsubscribe : https://launchpad.net/~openstack >>>>>> More help : https://help.launchpad.net/ListHelp >>>>> >>>>>> _______________________________________________ >>>>>> Mailing list: https://launchpad.net/~openstack >>>>>> Post to : openstack@lists.launchpad.net >>>>>> Unsubscribe : https://launchpad.net/~openstack >>>>>> More help : https://help.launchpad.net/ListHelp >>>>> >>>>> >>>>> _______________________________________________ >>>>> Mailing list: https://launchpad.net/~openstack >>>>> Post to : openstack@lists.launchpad.net >>>>> Unsubscribe : https://launchpad.net/~openstack >>>>> More help : https://help.launchpad.net/ListHelp >>>>> >>>>> >>>>> Confidentiality Notice: This e-mail message (including any attached or >>>>> embedded documents) is intended for the exclusive and confidential use >>> of the >>>>> individual or entity to which this message is addressed, and unless >>> otherwise >>>>> expressly indicated, is confidential and privileged information of >>> Rackspace. >>>>> Any dissemination, distribution or copying of the enclosed material is >>> prohibited. >>>>> If you receive this transmission in error, please notify us immediately >>> by e-mail >>>>> at ab...@rackspace.com, and delete the original message. >>>>> Your cooperation is appreciated. >>>> >>>> _______________________________________________ >>>> Mailing list: https://launchpad.net/~openstack >>>> Post to : openstack@lists.launchpad.net >>>> Unsubscribe : https://launchpad.net/~openstack >>>> More help : https://help.launchpad.net/ListHelp > > > _______________________________________________ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp _______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
