On Jul 25, 2011, at 12:30 PM, Sumit Naiksatam (snaiksat) wrote: Thank Ryu, very nice explanation, and does fall in line with what I could make from studying the current implementation. I will let Dan also respond in case he has any more insight. Meanwhile, a few comments/questions:
(1) Has anyone felt the need for Quantum to get notified when a particular host joins or leaves a nova cluster? I can see this being needed in the case where Quantum manages networking constructs (like bridges, or vnics in the case of 802.1Qbh) on a per host basis. Any thoughts around how this can be achieved (there is more than one way of doing it, but I just want to if any work/thought has gone into this)? I think this will be a requirement. We plan to look at some options because, as you note, there are multiple ways to do this. My preference which will need to be tested, would be to have Quantum listen to the notifications from Nova. I like this because it keeps things asynchronous. Another option is to have Nova explicitly manage this by issuing "detach" commands as part of a VM cleanup. If detach is implemented as an async call (quickly returns and then goes and detaches), this could also work at scale. We haven't had a change to fully vet either of these ideas (and there are other options also.) (2) I am still not very clear as to how the “user” and “project” in nova are related to the “tenant” in Quantum. Any clarifications would help. My understanding is that these are all converging as Nova integrates with Keystone. There is the concept of tenant which will replace the idea of projects going forward. (3) Regarding nova communicating with Quantum, I agree with both the options you have mentioned. I think we should keep the possibility open for using either option. Even if we implement it in the VIF driver, I guess it’s fine if that particular VIF driver is Quantum-specific, because you can always write another driver which relies on a different network service, or source of information. (4) This last one is probably not specific to nova-refactoring – Dan mentioned the use of Quantum Client library as a possible way to communicate between nova and Quantum. Other nova services use AMQP to exchange messages. I am just wondering if that is an option as well in communicating between nova and Quantum. Thanks, ~Sumit. From: Ishimoto, Ryu [mailto:r...@midokura.jp] Sent: Monday, July 25, 2011 4:58 AM To: Sumit Naiksatam (snaiksat) Cc: Dan Wendlandt; netstack@lists.launchpad.net<mailto:netstack@lists.launchpad.net> Subject: Re: "network-refactoring-l2" branch, and Quantum Hi Sumit, Comments inline Thanks Dan for your detailed response. My question was indeed based on the understanding that the current nova network managers (in the network service) do perform some L3/DHCP/VPN configurations apart from L2 configuration. As of today’s implementation in the network-refactoring-l2 branch, one would need to run the nova-network service configured with either one of Flat/DHCP/VLANManagers even when running Quantum as a separate service, right? If that is the case, how does one correlate an entry in the network table in the nova database, with a network created in Quantum? (I do realize that one could write a different network manager within nova, but my question is more in the context of what is available today.) Because Nova still relies on certain features provided by the Nova network managers(like IP management), you have to run one of Flat/DHCP/VlanManagers as a network service, or define your own manager. I understand the confusion about the correlation between Networks in Nova and Quantum, and I don't think this topic has been adequately discussed, so might as well start here :-) The confusion comes from the fact that Nova Network includes both L3 and L2 information. As far as I understand it, when integrating with Quantum, the L2 data is ignored. However, since currently the IP management is still being done by the network managers, for Quantum to work with Nova L3 networking, these Network records must still exist to act as subnets in which the VIFs get their IP addresses from. This means that Networks in Quantum and Nova are managed separately, and mean different things. Dan, please comment if you were thinking something different. I am guessing that the current approach is to use the Flat/DHCP/VLANManger to configure the L3/DHCP/VPN artifacts, and to use Quantum to configure the L2 network (in cases where it’s not a Linux bridge). The network host, when you run one of the existing Nova network managers, still uses Linux bridge and VLAN to set up L2 connectivity. This should be eventually refactored, but for now, one could extend the NetworkManager class to setup L2 networking on the network host to match that of the VIF plugins. But, I did also want to ask, as to what is the thinking around nova communicating with Quantum (if required). You have rightly pointed out “reporting the interface binding” as one of the cases when this might need to be done. Another case wherein I think this might be relevant is if the VIF driver (within nova virt layer) needs some information about the network from Quantum. There would need to be a communication channel between the VIF driver and Quantum to achieve this. I believe your suggestion is that in any of these cases, the Quantum client library can be used to communicate with the Quantum API (either core or extensions). I think we can do either: * Have the VIF driver directly communicate with Quantum to get the relevant data at the time of 'plugging' in the VIF in the virt layer. * At the time in which the compute manager gets network information from the network manager(allocate_for_instance), have the extended Quantum-aware network manager get the relevant data from Quantum, and return it to the compute manager, which in turn passes this data down to the virt layer(network_info dictionary). The first one is easier to implement, but you will end up with a dependency of a particular VIF type(802.1Qbh) to a Quantum service. As for the part on APIs which was not clear in my email, yes I was referring to the “nova-api (i.e. the OpenStack API) will expose interface-ids as an API extension”, as you point out. I believe, the extension referred to here is on the nova API side (and not on the Quantum side as my email seemed to suggest), right? Right, the exposure of interface IDs will be done through Nova's OpenStack API extension. I will get this extension in for the next milestone. Hope that helped! Thanks, Ryu Thanks, ~Sumit. From: Dan Wendlandt [mailto:d...@nicira.com<mailto:d...@nicira.com>] Sent: Sunday, July 24, 2011 10:46 PM To: Sumit Naiksatam (snaiksat) Cc: Ishimoto, Ryu; netstack@lists.launchpad.net<mailto:netstack@lists.launchpad.net> Subject: Re: "network-refactoring-l2" branch, and Quantum Hi Sumit, Very good questions. I'll give you my take inline, and Ryu should jump in as well :) Dan On Sun, Jul 24, 2011 at 12:50 PM, Sumit Naiksatam (snaiksat) <snaik...@cisco.com<mailto:snaik...@cisco.com>> wrote: Hi Ryu, Dan, and others involved, I had some questions regarding how the code base in the "network-refactoring-l2" branch currently works (or is supposed to) with Quantum. The network-refactoring-l2 branch itself doesn't actually interact with Quantum. It is really just a first step to providing more flexibility within nova for how VM interfaces are created and plugged into a "switch". This will be very helpful for many people using Quantum, as the existing nova code was hard-coded to use the Linux bridge. As I understand, eventually we want to get to a point where one should be able to run the Quantum service instead of the nova-network service. I actually don't think that's necessarily true. The existing nova-network service does many things beyond L2 networking, namely: - IP address management (melange project is targeting providing an improved version of this) - Network Node capabilities, including DHCP, L3 gateway, VPN, metadata server, floating IP forwarding, etc (such functionality may grow to be their own independent services, or an addon to Quantum in the future). - Network orchestration, in the sense that it automatically attaches VIFs to networks based on config (vlan vs. flat manager) and project assignment (donabe aims to become the primary mechanism for this type of orchestration, though some simple behavior equivalent to the existing flat/flatdhcp/vlan models may make sense to stay in nova for a while). There are some obvious next steps for D4: - Ryu will continue the work to merge his work that makes sure that other nova services retrieve network info via the API (not the DB). This is a critical step. - We'll expose interface-ids via the nova API (likely as an extension) - Troy's team will continue integrating the melange code with nova. - We'll probably create an alternate NetworkManager class (and corresponding manager utility) that uses melange for IPAM, quantum for L2 networks, and supports several different orchestration options. However, we are not there yet. Currently one still needs to run the nova-network service in order to be able create the network on the nova side of things. If this understanding is correct, here are few thoughts and questions - (1) How are you reconciling the network created within nova, with that created in Quantum? Current nova networks are a bit tricky, as they represent both L2 networks and IP subnets. A new network manager class + utility will give us the flexibility to handle this however we want. (2) The blueprint for this branch indicates that OpenStack APIs will be implemented as extension APIs in Quantum. Do we have any documentation on this API? (Also, I did not find any implementation in the Quantum trunk, I am guessing we haven't done that yet, right?) I'm not sure I follow what you're saying here. I suspect you are referring to the fact that nova-api (i.e. the OpenStack API) will expose interface-ids as an API extension? Those changes have not gone into nova yet, but are on the list I mentioned above. (3) There was earlier some talk about nova using an "administrative API" to communicate with Quantum. Is that still the plan? In general, what is the thinking around how nova would communicate with Quantum? There are possible uses that I have mentioned for such an admin API: - Reporting "interface bindings" from nova to a quantum plugin. Right now, since the nature of the binding is specific to the nova vif-plugin itself, we're leaving the communication of this binding info up to the plugin itself, so you're free to do whatever you want (your quantum plugin can expose additional APIs if needed). If there is code to share here, we'll probably just do it by sharing a code library (I suspect the new network manager we build will also require the quantum client library, which I would see an a nova dependency if using quantum). - Communicating permission to plug a vif. There have been a couple emails back and forth about this on the list. I am hoping we can use keystone for this, which in case no direct nova/quantum communication is needed. Otherwise, we'll probably create an admin API for this. Thanks, ~Sumit. -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~ Dan Wendlandt Nicira Networks, Inc. www.nicira.com<http://www.nicira.com> | www.openvswitch.org<http://www.openvswitch.org> Sr. Product Manager cell: 650-906-2650<tel:650-906-2650> ~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- Mailing list: https://launchpad.net/~netstack Post to : netstack@lists.launchpad.net<mailto:netstack@lists.launchpad.net> Unsubscribe : https://launchpad.net/~netstack More help : https://help.launchpad.net/ListHelp This email may include confidential information. If you received it in error, please delete it.
-- Mailing list: https://launchpad.net/~netstack Post to : netstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~netstack More help : https://help.launchpad.net/ListHelp
