Hello Stackers! Sorry to not back on this topic last week, too many things to do...
So, instead of trying this and that, reply this, reply again... I made a video about this problem, I hope that helps more than those e-mails that I'm writing! =P Honestly, I don't know the source of this problem, if it is with OpenStack / Neutron, or with "Linux / Namespace / OVS"... It would be great to test it alone, Ubuntu Linux + Namespace + OVS (without Neutron), to see if the problem persist but, I have no idea about how to setup everything, just like Neutron does. Maybe, I just need to reproduce the "Namespace and OVS bridges / ports / VXLAN - as is", without Neutron?! I can try that... Also, my Grizzly setup is gone, I deleted it... Sorry about that... I know it works because it is the first time I'm seeing this problem... I had used Grizzly for ~5 months with only 1 problem (related to MTU 1400) but, this problem with Havana is totally different... Video: OpenStack Havana L3 Router problem - Ubuntu 12.04.3 LTS: http://www.youtube.com/watch?v=jVjiphMuuzM * After 5 minutes, I inserted a new video, showing how I "fixed" it, by running Squid within the Tenant router. You guys can see that, using the default Tenant router (10:30), it will take about 1 hour to finish the "apt-get download" and, with Squid (09:27), it goes down to about 3 minutes (no, it is still not cached, I clean it for each test). Sorry about the size of the video, it is about 12 minutes and high-res (to see the screen details) but, it is a serious problem and I think it worth watching it... NOTE: Sorry about my English! It is very hard to "speak" a non-native language, handling an Android phone and typing the keyboard... :-) Best! Thiago On 28 October 2013 07:00, Darragh O'Reilly <dara2002-openst...@yahoo.com>wrote: > Thiago, > > some more answers below. > > Btw: I saw the problem with a "qemu-nbd -c" process using all the cpu on > the compute. It happened just once - must be a bug in it. You can disable > libvirt injection if you don't want it by setting "libvirt_inject_partition > = -2" in nova.conf. > > > On Saturday, 26 October 2013, 16:58, Martinx - ジェームズ < > thiagocmarti...@gmail.com> wrote: > > Hi Darragh, > > > > > >Yes, on the same net-node machine, Grizzly works, Havana don't... But, > for Grizzly, I have Ubuntu 12.04 with Linux 3.2 and >OVS 1.4.0-1ubuntu1.6. > > > so we don't know if the problem is due to Neutron, the Ubuntu kernel or > OVS. I suspect the kernel as it implements the routing/natting, interfaces > and namespaces. I don't think Neutron Havana changes how these things are > setup too much. > > Can you try running Havana on a network node with the Linux 3.2 kernel? > > > > > > > >If I replace the Havana net-node hardware entirely, the problem persist > (i.e. it "follows" Havana net-node), so, I think, it can not be related to > the hardware. > > > > > >I tried Havana with both OVS 1.10.2 (from Cloud Archive) and with OVS > 1.11.0 (compiled and installed by myself using dpkg-buildpackage / dpkg). > > > > > >My logs (including Open vSwitch) right after starting an Instance > (nothing at OVS logs): > > > > > >http://paste.openstack.org/show/49870/ > > > > > > > >I tried everything, including installing the Network Node on top of a KVM > virtual machine or directly on a dedicated server, same result, the problem > follows Hanava node (virtual or physical). Grizzly Network Node works both > on a KVM VM or on a dedicated server. > > > > > >Regards, > >Thiago > > > > > > > >On 26 October 2013 06:28, Darragh OReilly wrote: > > > >Hi Thiago, > >> > >>so just to confirm - on the same netnode machine, with the same OS, > kernal and OVS versions - Grizzly is ok and Havana is not? > >> > >>Also, on the network node, are there any errors in the neutron logs, the > syslog, or /var/log/openvswitch/* ? > >> > >> > >> > >>Re, Darragh. > >> > >> > >> > >> > >>On Saturday, 26 October 2013, 5:25, Martinx - ジェームズ < > thiagocmarti...@gmail.com> wrote: > >> > >>LOL... One day, Internet via "Quantum Entanglement"! Oops, Neutron! > =P > >>> > >>> > >>> > >>>I'll ignore the problems related to the "performance between two > instances on different hypervisors" for now. My priority is the > connectivity issue with the External networks... At least, internal is slow > but it works. > >>> > >>> > >>>I'm about to remove the L3 Agent / Namespaces entirely from my > topology... It is a shame because it is pretty cool! With Grizzly I had no > problems at all. Plus, I need to put Havana into production ASAP! :-/ > >>> > >>> > >>>Why I'm giving it up (of L3 / NS) for now? Because I tried: > >>> > >>> > >>>The option "tenant_network_type" with gre, vxlan and vlan (range > physnet1:206:256 configured at the 3Com switch as tagged). > >>> > >>> > >>>From the instances, the connection with External network is always > slow, no matter if I choose for Tenants, GRE, VXLAN or VLAN. > >>> > >>> > >>>For example, right now, I'm using VLAN, same problem. > >>> > >>> > >>>Don't you guys think that this can be a problem with the bridge "br-ex" > and its internals ? Since I swapped the "Tenant Network Type" 3 times, same > result... But I still did not removed the br-ex from the scene. > >>> > >>> > >>>If someone wants to debug it, I can give the root password, no problem, > it is just a lab... =) > >>> > >>> > >>>Thanks! > >>>Thiago > >>> > >>> > >>>On 25 October 2013 19:45, Rick Jones <rick.jon...@hp.com> wrote: > >>> > >>>On 10/25/2013 02:37 PM, Martinx - ジェームズ wrote: > >>>> > >>>>WOW!! Thank you for your time Rick! Awesome answer!! =D > >>>>> > >>>>>I'll do this tests (with ethtool GRO / CKO) tonight but, do you think > >>>>>that this is the main root of the problem?! > >>>>> > >>>>> > >>>>>I mean, I'm seeing two distinct problems here: > >>>>> > >>>>>1- Slow connectivity to the External network plus SSH lags all over > the > >>>>>cloud (everything that pass trough L3 / Namespace is problematic), > and; > >>>>> > >>>>>2- Communication between two Instances on different hypervisors (i.e. > >>>>>maybe it is related to this GRO / CKO thing). > >>>>> > >>>>> > >>>>>So, two different problems, right?! > >>>>> > >>>> > One or two problems I cannot say. Certainly if one got the benefit of > stateless offloads in one direction and not the other, one could see > different performance limits in each direction. > >>>> > >>>>All I can really say is I liked it better when we were called Quantum, > because then I could refer to it as "Spooky networking at a distance." > Sadly, describing Neutron as "Networking with no inherent charge" doesn't > work as well :) > >>>> > >>>>rick jones > >>>> > >>>> > >>> > >>> > >>> > > > > > > >
_______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
