I pushed some code for the api-filters feature into a github repo. You can find the code in the bp/api-filters here: git://github.com/salv-orlando/MyRepo.git
Please note this code is not final yet (unit tests will come, pep8 and pylint will be improved). I'm sharing this to give you a feel of the feature and receive feedback based not only on specifications but on actual code as well. Also note that the names assigned to filters are more or less a product of my fantasy and probably need to be improved. Salvatore From: netstack-bounces+salvatore.orlando=eu.citrix....@lists.launchpad.net [mailto:netstack-bounces+salvatore.orlando=eu.citrix....@lists.launchpad.net] On Behalf Of Salvatore Orlando Sent: 09 January 2012 15:04 To: Dan Wendlandt Cc: netstack@lists.launchpad.net Subject: Re: [Netstack] Specification for filters in quantum api Hi, First of all, apologies for a much-delayed reply. I'm officially back in office today, and I'm slowly catching up with my todo list. Please see my replies inline. The next time in my todo list is to push a preview of the code (I still need to add unit tests to make it gerrit-pushable). Salvatore From: Dan Wendlandt [mailto:d...@nicira.com] Sent: 03 January 2012 01:45 To: Salvatore Orlando Cc: netstack@lists.launchpad.net Subject: Re: [Netstack] Specification for filters in quantum api One other related point is that these filters are going to be in 1.1 only, which seems reasonable. However, if we want QuantumManager to be able to take advantage of the filters (which we definitely do), then we will need QuantumManager to be using 1.1 soon, which would mean we need to finalize 1.1 even sooner. Are there changes other than tags that are still outstanding for the 1.1 API? We have a few outstanding changes which I'll regard as minor, and re the following: - Pagination for list responses - ATOM links in responses These are features already present in the Openstack API, and adding them by E-4 should be absolutely feasible. The real question is: do we actually feel a need for these features? My opinion is that pagination is fairly important, whereas ATOM links are maybe somewhat unnecessary. Dan On Mon, Jan 2, 2012 at 5:42 PM, Dan Wendlandt <d...@nicira.com<mailto:d...@nicira.com>> wrote: << working through quantum email backlog, in roughly chronological order >> Hi Salvatore, As always, great write-up. Very clear. Everything in terms of the implementation section makes perfect sense and is very much in line with my thoughts. A few comments below. Dan - My main short-term driver for filtering is various queries made by the Quantum Manager. For example, in https://github.com/openstack/nova/blob/master/nova/network/quantum/quantum_connection.py there is a get_port_by_attachment and a get_attached_ports call. The former in particular is very expensive. It seems like these calls would be handled by the with-interface and has-interface queries, which is great. Yeah, the filters currently implemented will definitely allow to perform these operations. - For string matching, as we assuming exact match only, or support for regex/glob? At the moment I have exact matching only. Moving to regex should not be too much of an issue, though. I'd like to have some more feedback from the community on this topic. For instance, do we want to use different querystring parameters for exact matching and regex-based filters? (e.g.: GET ..../networks?name=test99 and GET .../networks?name_regex=t.st9%5B9%5D ) - Another interesting item to query on may be to find the net-id for a given port-id. This is a bit wonky, as a port-id is technically scoped by a net-id, but I think in many circumstances it could be convient to only have to store the port-id, with the ability to call up the network when needed. Good point, I will add a filter for that. - Are there planned changes to the client lib (e.g., be able to take in a dict of key-value pairs, which is converted into params on the query string). Would this be a separate issue? If so, let's file. I'm not adding anything to the client lib at the moment; however, we definitely need somebody to take a look at it and bring it up-to-date with API v1.1. - Looks like we can just copy how the Nova API docs indicate the set of supported filters. One downside with that format though is that there no good typing (e.g., look at the server list command linked from the spec, which specifies a "dateTime" value but doesn't describe the format). I think we should follow the style of docs for the nova API, as it gives the user an immediate synopsis of supported filters. However, I'd also add a section discussing, for each resource, which filters are supported and how they should be specified, starting from the tables defined in the blueprint specification. - The issue of what to do with plugins that don't implement the filters is a tough question. I think that by updating our common sqlalchemy DB layer to handle filters, we will provide a good default datastore that any plugin can use. Plugins that choose to implement their own datastore would then figure out how to do their own filtering (filtering is likely to be very datastore specific, as it really comes down to what fields that datastore lets you index over), but that seems reasonable, since implementing a different datastore itself is a pretty big complexity leap. With that in mind, I am find with a fairly harsh failure if Quantum receives an API request with a filter, but the plugin does not support filters. Stated another way, I don't think its a good use of time to be building a datastore-agnostic filtering mechanism on top of the plugin layer, as any real plugin will want datastore-aware filtering. This has been my dilemma over the holydays as well. I decided not to add filtering support in the sqlalchemy layer; by doing so I would have ensured only that the FakePlugin supported filters; indeed, since plugins are not required to use that db layer, adding filtering support there does not ensure plugin will support filtering at all. Therefore I decided for the following approach: - Plugins expose two methods (can_filter_networks and can_filter_ports) for expressing their capabilities. If these methods are not defined on the plugin, then Quantum assumes the plugin does not support filtering; - If the plugin does not support filtering, then filtering is done in the API layer by using "standard" plugin operations. This can turn out to be inefficient and expensive, but at least will still ensure that plugins that do not support filtering will not break the functionality. - On a related note, do you consider implementing filtering in the sqlalchemy layer part of this deliverable, or do we need a separate issue? I see it as something separate at the moment, for the very reason you mentioned earlier on: our sqlalchemy layer at the moment is not a "mandatory" component of Quantum, but rather a set of classes and routines that a plugin *might* use or not. - I believe Ying was going to look at implementing key-value tags in the API. Before we "freeze" the set of filters supported for 1.1, it would probably be a good idea to make sure we have a filter mechanism for searching on tags. We had a meeting with Ying before Christmas. Right now I do not remember whether we decided to add the concept of tag to the core API, or implement tags through extensions. Nevertheless, I'm currently implementing filters in a way that, if the plugin supports filters, all querystring parameters are passed to the plugin. On Tue, Dec 13, 2011 at 8:58 AM, Salvatore Orlando <salvatore.orla...@eu.citrix.com<mailto:salvatore.orla...@eu.citrix.com>> wrote: Hi, A full spec is now available for the API-filters blueprint. You can find it at http://wiki.openstack.org/quantum-api-filters or checking the blueprint itself. Regards, Salvatore -- Mailing list: https://launchpad.net/~netstack Post to : netstack@lists.launchpad.net<mailto:netstack@lists.launchpad.net> Unsubscribe : https://launchpad.net/~netstack More help : https://help.launchpad.net/ListHelp -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~ Dan Wendlandt Nicira Networks: www.nicira.com<http://www.nicira.com> twitter: danwendlandt ~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~ Dan Wendlandt Nicira Networks: www.nicira.com<http://www.nicira.com> twitter: danwendlandt ~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- Mailing list: https://launchpad.net/~netstack Post to : netstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~netstack More help : https://help.launchpad.net/ListHelp
