I think that providers may not wish to expose the internal structure of their network to the degree that you are suggesting. I prefer the idea of near other object with an internal lookup into zone. On Feb 11, 2011 9:38 AM, "Eric Day" <e...@oddments.org> wrote: > Hi Sandy, > > I agree with using tags for full scheduler selection, it's something > I've been pushing for from the start. The request contains any number > of k/v pairs, the services provide any number of k/v pairs, and the > scheduler performs a match (some required, some optional, ...). I see > the URI/zone as one of those tags, not something we need to overload > to contain all of the capabilities. It should only be a hierarchical > "location", which may be geographic location, organizational location > (dept, ...), or some other type (however you decide to construct > your zones). > > For example, imagine a dynamic del.icio.us tag that allowed for domain > name filtering on bookmarks (give me all bookmarks with tags [book > review] [domain:slashdot.org]). For Nova, this means issuing requests > like "create instance with [GPU] [Fast disk] [zone:dc1.example.com]". > > The important thing is that this is not a tag specific to a particular > service. For example, Swift would never care or need to understand a > 'GPU' tag, but it can share and understand zone tags. > > -Eric > > On Fri, Feb 11, 2011 at 12:40:44PM +0000, Sandy Walsh wrote: >> Heh, hate to be the one to bust up the URI love-fest :) >> >> The issue I have with a single URI being used as the heuristic for node selection is that it is very rigid. >> >> Different business units have different views on the network: >> * Operations may view it as geography/data centers. >> * Consumers may view it as technical ability (gpu's, fast disk, good inter-server speed, etc) >> * Sales/marketing may view it as the number of martinis they can buy ;) >> >> Trees become unmanageable/hard to visualize for users beyond a couple hundred nodes. We are lucky that our geographical/DC-based hierarchy is relatively flat. This is why I was initially pushing for a tag-based system for selection (aka Zone/Host Capabilities). >> >> Consider the way delicio.us works. They manage many millions of URL's and tags are an effective way to slice & dice your way through the data: >> "Show me all the URL's on [OpenStack] [Python] [Zones] [Scheduler]" ... blam. >> >> This is also the way the old Trader services worked: >> "I want a [wax transfer] [color] printer that can has [30ppm] and [300dpi] on [Floor 2]" >> >> "Near" simply has to mean the distance in zones from the most-optimal zones, based on the tags. >> >> "I want a new instance with [GPU] and [Fast Disk] [Good inter-instance network speed] [near] [DRW] [DC1]" >> * where "[near]" implies "as close as possible to" in zone distance. >> >> Personally I don't like overloading the zone name to have a "meaningful" URI when we can get the same functionality with Capabilities/Tags already. And we already know we need Capability support anyway. Especially if it means enforcing a rigid hierarchy. >> >> $0.02 >> >> -S >> >> >> >> ________________________________________ >> From: >> openstack-bounces+sandy.walsh=rackspace....@lists.launchpad.net[openstack-bounces+sandy.walsh= rackspace....@lists.launchpad.net] on behalf of Eric Day [e...@oddments.org] >> Sent: Friday, February 11, 2011 4:30 AM >> To: Justin Santa Barbara >> Cc: openstack@lists.launchpad.net; Devin Carlen >> Subject: Re: [Openstack] Allowing clients to pass capability requests through tags? >> >> The main reason I was proposing full location/zone of objects is to >> allow this type of 'near' scheduling to happen without understanding >> what the actual object is. For example, imagine we want to start an >> instance near a particular swift object. We could query the swift >> object and in the metadata there could be a 'zone' tag (well, three, >> one for each copy). For example: >> >> get swift-12345: zone=rack12.room2.dc1.dfw.rackspace.com >> >> I can now use that zone name to: >> >> create_instance: openstack:near=rack12.room2.dc1.dfw.rackspace.com >> >> The deployment can decide what 'near' is (perhaps a measure of link >> speed or latency). This way a particular deployment that uses the >> same URI/zone names across projects can account for locality without >> knowing what objects from different services are. If it were just >> 'near=swift-12345', it would need to understand what a swift object >> was and perform that lookup to find out where it is. >> >> So you can still grab a zone tag from a volume you created: >> >> get vol-000001: rack4.room2.dc1.dfw.rackspace.com >> >> and use the zone to launch an instance with: >> >> create_instance: openstack:near=rack4.room2.dc1.dfw.rackspace.com >> >> We can also write schedulers/tools for a particular deployment >> that understands the zones to just say 'always prefer in >> dc1.dfw.rackspace.com', because power is cheaper there right now, or >> 'test.dc1.dfw.rackspace.com' because that is my test zone (perhaps >> only enabled for certain accounts in the scheduler too). >> >> -Eric >> >> On Thu, Feb 10, 2011 at 03:38:42PM -0800, Justin Santa Barbara wrote: >> > I think the blueprint was largely complementary to the multi-zone stuff; >> > this is more about how the client _requests_ a particular >> > location/capability through the API. The multi-zone blueprint seems to be >> > more about how nova would satisfy those requests (in a non-trivial zone >> > structure.) >> > The root motivator is indeed getting a 'good' connection to a storage >> > volume. I'm thinking of iSCSI SAN storage here, so in my case this >> > probably means the SAN device with the least number of switches in >> > between. There could well be SAN devices in each rack (e.g. Solaris >> > volume nodes), or the devices could even be running on the host nodes, and >> > I don't believe that zones in the EC2 sense are sufficient here. >> > But I guess that if the zone hierarchy went all the way down to the rack >> > (or machine), that would work. So I could create a volume and it would >> > come back with a location of "rack4.room2.dc1.dfw.rackspace.com" and I >> > could then request allocation of machines in that same rack? Is that the >> > vision of the nested zones? >> > I do have a concern that long-term if we _only_ use zones, that's trying >> > to multiplex a lot of information into the zone hierarchy, and we can >> > really only put one attribute in there. I also like the flexibility of >> > the 'openstack:near=vol-000001' request, because then the cloud can decide >> > how near to place the instance based on its knowledge of the topology, and >> > the clients can be oblivious to the storage system and arrangement. But, >> > my immediate requirement would indeed be satisfied if the zones went down >> > to the rack/machine level. >> > An alternative way to look at zones and instance-types is that they're >> > actually just fail-if-not-satisfiable tags of the creation request >> > (openstack:+zone=us-east-1a and openstack:+instancetype=m1.large) They're >> > only distinguished attributes because AWS doesn't have an >> > extensibility mechanism, which this blueprint would give us. >> > Justin >> > >> > On Thu, Feb 10, 2011 at 3:12 PM, Devin Carlen <devcam...@me.com> wrote: >> > >> > I haven't totally digested this blueprint yet but it seems like there is >> > some overlap with what is being discussed with the multi zone metadata >> > stuff. One approach might be to handle this awt the scheduler level >> > though and try to ensure things are always in the same zone when >> > appropriate. >> > I think the bigger question you raise is how to request local volumes >> > when possible, yes? >> > >> > Devin >> > On Feb 10, 2011, at 3:37 PM, Justin Santa Barbara <jus...@fathomdb.com> >> > wrote: >> > >> > Does anyone have any thoughts/objections on the blueprint I posted for >> > allowing clients to pass capability-requests through tags? I'm >> > planning on starting implementation soon, so if people think this is a >> > bad idea I'd rather know before I start coding! >> > Blueprint: https://blueprints.launchpad.net/nova/+spec/use-metadata-tags-for-capabilities >> > Wiki: https://blueprints.launchpad.net/nova/+spec/use-metadata-tags-for-capabilities >> > And a quick TLDR: >> > API clients need a way to request e.g. placement of machines near each >> > other / near volumes, or that a volume be created with a particular >> > RAID level, or that a machine be created in a HIPAA compliant >> > environment. (This is complementary to the work on hierarchical zones >> > & URL naming, I believe) >> > I propose using the instance tags for this, e.g. specifying >> > openstack:near=vol-000001 when creating an instance to request >> > locating the instance 'close to' that volume. >> > By default these requests would be best-effort and ignored-if-unknown; >> > if the client wants to specify that something is required and should >> > fail if not understood or not satisfiable, they could use a "+" e.g. >> > openstack:+location=*.dc1.north.rackspace.com >> > Controversially (?), this would not be supported for clients using the >> > AWS API, because tags can only be specified once the instance has >> > already been created. >> > Feedback appreciated! >> > Justin >> > >> > _______________________________________________ >> > Mailing list: https://launchpad.net/~openstack >> > Post to : openstack@lists.launchpad.net >> > Unsubscribe : https://launchpad.net/~openstack >> > More help : https://help.launchpad.net/ListHelp >> >> > _______________________________________________ >> > Mailing list: https://launchpad.net/~openstack >> > Post to : openstack@lists.launchpad.net >> > Unsubscribe : https://launchpad.net/~openstack >> > More help : https://help.launchpad.net/ListHelp >> >> >> _______________________________________________ >> Mailing list: https://launchpad.net/~openstack >> Post to : openstack@lists.launchpad.net >> Unsubscribe : https://launchpad.net/~openstack >> More help : https://help.launchpad.net/ListHelp >> >> >> Confidentiality Notice: This e-mail message (including any attached or >> embedded documents) is intended for the exclusive and confidential use of the >> individual or entity to which this message is addressed, and unless otherwise >> expressly indicated, is confidential and privileged information of Rackspace. >> Any dissemination, distribution or copying of the enclosed material is prohibited. >> If you receive this transmission in error, please notify us immediately by e-mail >> at ab...@rackspace.com, and delete the original message. >> Your cooperation is appreciated. > > _______________________________________________ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp
_______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
