Hi Salvatore and Dan,
Thanks for the great discussion. (Apologize for joining the discussion late. My outlook filter has some problem recently. First several messages on this thread went to a wrong folder.) I prefer for having both mechanism available. That means, Plugin can decide whether it throws RESOURCENOTREADY error or not. There can be various actions associated with a resource. For example, Action A can be done before the resource is ready. Action B is not. In plugin, we can throw RESOURCENOTREADY exception for calling action B but not for action A. The client side's logic is simple. After the creation resource and get an identifier, the client will do whatever he wants. If he gets RESOURCENOTREADY exception, he will wait and check resource status and then make the call. Best, Ying From: netstack-bounces+yinliu2=cisco....@lists.launchpad.net [mailto:netstack-bounces+yinliu2=cisco....@lists.launchpad.net] On Behalf Of Dan Wendlandt Sent: Monday, August 08, 2011 12:00 PM To: Salvatore Orlando Cc: netstack@lists.launchpad.net Subject: Re: [Netstack] Quantum API resource status proposal Thanks for the explanation Salvatore, very helpful to understand where you are coming from. I think we're approaching this from slightly different angles, but aren't too far off. My thinking was that most (all?) plugins will operate in a way that they store the data resulting from a web service call before returning (e.g., receive the call, update the database, return). The corresponding "wiring" work to configure switch could then likely happen asynchronously. This model gives you the nice properties that once I've done a create, I can always use the identifiers returned in the create response in subsequent calls without having to do any polling. I wonder if it is possible that we instead just say that the API should not return an identifier from a create call until it is ready to accept subsequent calls using the returned identifier (note: this says nothing about whether that functionality is "wired", which may happen asynchronously, just whether the existence of an API object has been stored). It seems like a plugin should be able to store and return the data with minimal effort, but perhaps I'm missing a scenario where this wouldn't be the case. I think this would simplify clients, as the client logic is simpler if it does not have to have to be potentially catching a RESOURCENOTREADY error (though I assume such an error could probably be hidden by a wrapper class). I'm curious, does the nova API have an equivalent of RESOURCENOTREADY? Dan On Fri, Aug 5, 2011 at 8:34 AM, Salvatore Orlando <salvatore.orla...@eu.citrix.com> wrote: I agree we are definitely on the same page. Please see my reply to your comments below. Salvatore From: Dan Wendlandt [mailto:d...@nicira.com] Sent: 05 August 2011 16:14 To: Salvatore Orlando Cc: netstack@lists.launchpad.net Subject: Re: [Netstack] Quantum API resource status proposal Great Salvatore, I think we're largely on the same page, a few comments below. dan On Fri, Aug 5, 2011 at 2:34 AM, Salvatore Orlando <salvatore.orla...@eu.citrix.com> wrote: Thanks for your comments Dan. Replies inline. Cheers, Salvatore From: Dan Wendlandt [mailto:d...@nicira.com] Sent: 05 August 2011 01:25 To: Salvatore Orlando Cc: netstack@lists.launchpad.net Subject: Re: [Netstack] Quantum API resource status proposal Great write-up Salvatore, comments inline. dan On Tue, Aug 2, 2011 at 4:34 PM, Salvatore Orlando <salvatore.orla...@eu.citrix.com> wrote: Following the discussion on synchronous vs asynchronous behaviour in the last Netstack meeting, we agreed to send around a proposal for introducing the concept of "Resource Status". Goal: * Ensure consistent behaviour of the API w.r.t. users, meaning that the API should always behave in the same way regardless of the plugin used. Assumptions: * Plugins can implement any kind of behaviour: synchronous, or asynchronous. It is also possible to have plugins for which some operations are synchronous and other asynchronous; * Both the API and the plugin interface should be kept as simple as possible; we do not want unnecessary complexity; * API users need a mechanism to know whether an operation is completed or not. * A resource can "exist" but not be available yet, as the plugin might asynchronously perform all the operations required to provide it. Example (synchronous plugin) "Create network" on the OVS plugin adds a vlan binding into its own database and then return id and name of the newly created network Example (asynchronous plugin) Just guessing here... A plugin for 802.1qbh will probably have a more complex provisioning process, as it will involve configuring virtual switches, IV modules, and physical switches. The "Create network" operation will create an entry into the plugin db, start the provisioning process, and immediately return. The network will not yet be available. To further emphasize the point, even in the rather simple OVS plugin the operation of "attaching" an interface to a port is asynchronous. The plugin creates a new port row in the database, and the agent on the hypervisor monitors the DB for changes and asynchronously puts the port on that VLAN. I suspect that pretty much all plugins will have some amount of asynchronous behavior. Proposal: The proposal simply consists of adding an extra "status" attribute to all the resource managed by the Quantum API, namely: network, port, and attachment. The status attribute will be of an enumeration type describing the current state of the resource, similarly to the homonymous attribute for the "Server" resource in the Openstack API. The actual values for the enumeration could be different for each resource. Possible Status enumeration values: NETWORK: "PROVISIONING","AVAILABLE","ERROR". PORT: "PROVISIONING","DOWN", "ACTIVE","ERROR". ATTACHMENT: "PROVISIONING", "AVAILABLE", "ERROR" Note: the "Port" resource already has a "state" attribute, which describes the administrative state of the port. The proposed enumeration merges this attribute into the "status" attribute. Adding the "status" attribute will enable users to be aware of the current provisioning state of the resource. Client application can then query the resource (e.g.: by polling it with GET requests), and check for its status. When the resource is available, they can perform operations on it. A synchronous plugin should directly mark the resource as available when it returns. I am in favor of a status field. I want to make sure we're thinking about more than just provisioning though. Provisioning is one case in which the "logical" connectivity described in the API has not yet been mapped to the real world, but there can be other cases as well. For example, if your solution has an agent on the hypervisor and that agent no longer is running, how do you indicate the fact that this port is no longer "controlled" by quantum. I've spent a lot of time working with systems like this, and in my experience these types of situations are actually pretty common. A similar case is how do we indicate if someone has specified an interface-id, but the system does not see any ports with that interface id? An abstraction that I like for this is that of a "logical link status". If this status is "up", the logical connectivity described via the API is the connectivity the VM interface sees. If it is down, there is a mismatch. The plugin could then return a "link_status_string" indicating why the status is down (e.g., "interface-id not found"). [SALVATORE]: I find your comment really interesting. In my previous experience I too always separated the concepts of "administrative" and "operational" states. I quite mixed them up in my proposal in order to keep API changes to a minimum, but I agree that there's probably a case for having this distinction, at least for the "Port" resource. For a port I would see an administrative state as either 'ACTIVE' or 'DOWN' (as it is today), and then another attribute, logical or operational state which is somewhat similar to the "link" led on a physical network switch. Yup, this is exactly how I was thinking about it. The operation state could be one, IMHO, one of the following: BUILD - the port is being provisioned UPDATING - the port's configuration is being updated ERROR - something wrong with the port and connectivity is not guaranteed. This could be for one of the cases you listed. An error string returned by the plugin (e.g.: link_status_string) could also be returned by the API READY_LINKDOWN - port correctly configured and working properly, no attachment plugged READY_LINKUP - port correctly configured and working properly, attachment plugged, connectivity present. These make sense. An alternative scheme would be having three states: ADMINISTRATIVE, OPERATIONAL, LOGICALLINK. The API will reject operations on resources which are not available, unless they are GET operations. As I mentioned in the IRC chat, I think this one requires some more discussion. Would I be able to create a port, attach an interface to it, and configure a QoS policy right away, even if provisioning took a while? This seems potentially cumbersome for the client. If the client wanted to wait until the port was in a particular state before applying additional configuration, it would seem like the client code to do that would be fairly simple, but I suspect other clients would prefer just to fire off the configuration without waiting. [SALVATORE]: Good point. Also in this case the choice is between enforcing the behaviour at the API layer, or leaving it to the plugin. I want to be consistent with the other choices we made in the past, and I'll say I want the decision to be left the plugin, but the API has to be consistent wrt client applications. So I'd say that each operation could throw a "RESOURCENOTREADY" error. When a client receives this error, it means it has to poll the resource until its status becomes "READY" before submitting the request. I agree that the API should be consistent across plugins, so in this case I was not actually advocating that we leave the decision up to the plugin so much as wel leave it up to the API client. For example, if the API client didn't want to configure a filtering policy until a port was done being provisioned, it could poll on the port link status until that status was 'AVAILABLE', then submit its port operation. It seems to me that such polling is equivalent to what would happen if the port operations returned 'RESOURCENOTREADY', as the client would essentially have to keep retrying until the port operation did not return an error. Stepping back, maybe the issue is that I don't understand your underlying motivation for not allowing an API client to perform an operation on a temporarily unavailable port (as long as the API gives the client a means of detecting that the port is unavailable such that the client could perform this check itself if it likes). I suspect that if we first get on the same page about that, the rest of the design decisions will fall out cleanly :) By assuming the API would reject operations in resources which were not ready I was implicitly enforcing the behaviour at the API layer. I was assuming that a plugin would not be able to perform an operation on a resource if that resource was not ready (ready = logical link status is 'UP'). However, I realized this is probably a simplistic assumption as a plugin might have the capability of queuing requests for resources temporarily not available. This why in my second proposal I agreed not to block API requests anymore, but simply introduce the RESOURCENOTREADY error which CAN be raised by an API operation. As you said, retrying the operation until you don't get a RESOURCENOTREADY and polling until the status of the resource is AVAILABLE are exactly the same thing. I don't think it would be harmful to have both mechanisms. In this way, if the plugin accepts operations on resources not yet available, the client will never see the RESOURCENOTREADY error; On the other hand, if the plugin is unable to accept this kind of operation (e.g.: plugging an interface on a port still being provisioned), then the client will receive RESOURCENOTREADY error, and react accordingly. Regards, Salvatore -- Mailing list: https://launchpad.net/~netstack Post to : netstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~netstack More help : https://help.launchpad.net/ListHelp -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~ Dan Wendlandt Nicira Networks, Inc. www.nicira.com | www.openvswitch.org Sr. Product Manager cell: 650-906-2650 ~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~ Dan Wendlandt Nicira Networks, Inc. www.nicira.com | www.openvswitch.org Sr. Product Manager cell: 650-906-2650 ~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~ Dan Wendlandt Nicira Networks, Inc. www.nicira.com | www.openvswitch.org Sr. Product Manager cell: 650-906-2650 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- Mailing list: https://launchpad.net/~netstack Post to : netstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~netstack More help : https://help.launchpad.net/ListHelp
