Hey Sheng, Thanks for testing! I must admit I didn't test thoroughly with vlan as most of my networks are Nicira based.
I've done some research and I can fix it by using fake bridges (in openvswitch) when libvirt is too old. That requires some modifications to the plug routines, but nothing that can't be done. I've created two issues based on both our test results : CLOUDSTACK-2326 and CLOUDSTACK-2327. If you have more relevant input feel free to update the tickerts. I'll start working on them right away. Cheers, Hugo From: Sheng Yang [mailto:sh...@yasker.org] Sent: Friday, May 03, 2013 12:39 AM To: Hugo Trippaers Cc: <dev@cloudstack.apache.org> Subject: Re: OVS on KVM After upgrade to Ubuntu 13.04(libvirt 1.0.2), vlan tag works well. --Sheng On Thu, May 2, 2013 at 11:18 AM, Sheng Yang <sh...@yasker.org<mailto:sh...@yasker.org>> wrote: After searching I found this: http://libvirt.org/formatnetwork.html <quote> Setting VLAN tag (on supported network types only) ... <devices> <interface type='bridge'> <vlan trunk='yes'> <tag id='42'/> <tag id='47'/> </vlan> <source bridge='ovsbr0'/> <virtualport type='openvswitch'> <parameters interfaceid='09b11c53-8b5c-4eeb-8f00-d84eaa0aaa4f'/> </virtualport> </interface> <devices> ... If (and only if) the network type supports vlan tagging transparent to the guest, an optional <vlan> element can specify one or more vlan tags to apply to the traffic of all guests using this network **Since 0.10.0**. (openvswitch and type='hostdev' SR-IOV networks do support transparent vlan tagging of guest traffic; everything else, including standard linux bridges and libvirt's own virtual networks, do not support it. 802.1Qbh (vn-link) and 802.1Qbg (VEPA) switches provide their own way (outside of libvirt) to tag guest traffic onto specific vlans.) As expected, the tag attribute specifies which vlan tag to use. If a network has more than one <vlan> element defined, it is assumed that the user wants to do VLAN trunking using all the specified tags. In the case that vlan trunking with a single tag is desired, the optional attribute trunk='yes' can be added to the vlan element. </quote> I am using 0.9.13(with ubuntu 12.10). Does that means we need newer version? --Sheng On Thu, May 2, 2013 at 10:55 AM, Sheng Yang <sh...@yasker.org<mailto:sh...@yasker.org>> wrote: I DO SEE the tag on VM profile when agent start, but I didn't see them on OVS ports. 2013-05-01 18:04:44,702{GMT} DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-5:) starting v-2-VM: <domain type='kvm'> <name>v-2-VM</name> <uuid>1422832d-be18-352a-a08a-9bbff40e0d14</uuid> <description>Debian GNU/Linux 5.0 (32-bit)</description> <clock offset='utc'> </clock> <features> <pae/> <apic/> <acpi/> </features> <devices> <emulator>/usr/bin/kvm</emulator> <interface type='bridge'> <source bridge='cloud0'/> <mac address='0e:00:a9:fe:02:45'/> <model type='virtio'/> <virtualport type='openvswitch'> </virtualport> </interface> <interface type='bridge'> <source bridge='cloudbr0'/> <mac address='06:f7:5c:00:00:06'/> <model type='virtio'/> <virtualport type='openvswitch'> </virtualport> </interface> <interface type='bridge'> <source bridge='cloudbr0'/> <mac address='06:4c:12:00:00:1a'/> <model type='virtio'/> <virtualport type='openvswitch'> </virtualport> <vlan trunk='no'> <tag id='1610'/> <----------- here </vlan></interface> <serial type='pty'> <target port='0'/> </serial> <graphics type='vnc' autoport='yes' listen='' /> <disk device='disk' type='file'> <driver name='qemu' type='qcow2' cache='none' /> <source file='/mnt/20ad978d-a581-3a08-95fd-c2a45417513c/2f12ce26-4e4b-4d6e-b77e-1c45afff58e9'/> <target dev='vda' bus='virtio'/> </disk> <disk device='cdrom' type='file'> <driver name='qemu' type='raw' cache='none' /> <source file='/usr/share/cloudstack-common/vms/systemvm.iso'/> <target dev='hdc' bus='ide'/> </disk> <console type='pty'> <target port='0'/> </console> <input type='tablet' bus='usb'/> <channel type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/v-2-VM.agent'/> <target type='virtio' name='v-2-VM.vport'/> <address type='virtio-serial'/> </channel> </devices> <memory>1048576</memory> <vcpu>1</vcpu> <os> <type arch='x86_64' machine='pc'>hvm</type> <boot dev='cdrom'/> <boot dev='hd'/> </os> <cputune> <shares>500</shares> </cputune> <on_reboot>restart</on_reboot> <on_poweroff>destroy</on_poweroff> <on_crash>destroy</on_crash> </domain> After this, vnet2 should be tagged with 1610, but: root@yasker-box1:~# ovs-vsctl list port vnet2 _uuid : 012a6140-bd87-4917-84cc-7190829c695a bond_downdelay : 0 bond_fake_iface : false bond_mode : [] bond_updelay : 0 external_ids : {} fake_bridge : false interfaces : [95bcf67b-12c1-44e5-87da-5663c6644da3] lacp : [] mac : [] name : "vnet2" other_config : {} qos : [] statistics : {} status : {} tag : [] trunks : [] vlan_mode : [] So it cannot access the public network. After: root@yasker-box1:~# ovs-vsctl set port vnet2 tag=1610 root@yasker-box1:~# ovs-vsctl list port vnet2 _uuid : 012a6140-bd87-4917-84cc-7190829c695a bond_downdelay : 0 bond_fake_iface : false bond_mode : [] bond_updelay : 0 external_ids : {} fake_bridge : false interfaces : [95bcf67b-12c1-44e5-87da-5663c6644da3] lacp : [] mac : [] name : "vnet2" other_config : {} qos : [] statistics : {} status : {} tag : 1610 trunks : [] vlan_mode : [] It can access the public network with vlan 1610. --Sheng On Thu, May 2, 2013 at 4:34 AM, Hugo Trippaers <htrippa...@schubergphilis.com<mailto:htrippa...@schubergphilis.com>> wrote: Hey Sheng, The tagging is done by libvirt. Can you check your agent.log? I would have expected an entry in the log file looking like this 's_logger.debug("creating a vlan dev and bridge for public traffic per traffic label " + trafficLabel);' Also the XML document for the vif sent to libvirt should have the following tag '<vlan trunk='no'>\n<tag id='" + _vlanTag + "'/>\n</vlan>"' What are your traffic labels set to for kvm? Could you share your agent.properties? Cheers, Hugo From: Sheng Yang [mailto:sh...@yasker.org<mailto:sh...@yasker.org>] Sent: Thursday, May 02, 2013 3:17 AM To: Hugo Trippaers; <dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>> Subject: OVS on KVM Hi Hugo, I am trying to use OVS on KVM now, but I found all public ports are not tagged with public vlan as it supposed to be, so any public traffic cannot goes out. I've verified that I am using OvsVifDriver. Here is the output of ovs-vsctl show: <quote> root@yasker-box1:~/kvm-agent# ovs-vsctl show 02281b72-131c-4b24-b191-fb1bb7fe186d Bridge "cloud0" Port "cloud0" Interface "cloud0" type: internal Port "vnet3" Interface "vnet3" Port "vnet0" Interface "vnet0" Bridge "cloudbr0" Port "vnet2" Interface "vnet2" Port "vnet6" Interface "vnet6" Port "vnet4" Interface "vnet4" Port "vnet9" Interface "vnet9" Port "vnet10" Interface "vnet10" Port "vnet1" Interface "vnet1" Port "cloudbr0" Interface "cloudbr0" type: internal Port "eth0" Interface "eth0" Port "vnet5" Interface "vnet5" ovs_version: "1.4.3" </quote> I've checked the Installation guide, it use different bridge for different vlan. But would that be the only way to work? Because we can have different public vlans. Maybe I got some setup wrong... Any comments? Thanks! --Sheng
