Thanks Hugo, I think these two things are exactly what we need. But, I am thinking of if we need to support libvirt version below 0.10.x? I don't know how much work it would bring to get 0.9.11 ~ 0.10.x working, and I doubt fake bridge would works well enough(e.g. https://www.redhat.com/archives/libvirt-users/2013-April/msg00074.html ). Probably we can just require libvirt version more than 0.10.0(or 0.10.1? according to http://www.siliconloons.com/?p=305)?
--Sheng On Fri, May 3, 2013 at 8:46 AM, Hugo Trippaers < htrippa...@schubergphilis.com> wrote: > Hey Sheng,**** > > ** ** > > Thanks for testing! I must admit I didn’t test thoroughly with vlan as > most of my networks are Nicira based. **** > > ** ** > > I’ve done some research and I can fix it by using fake bridges (in > openvswitch) when libvirt is too old. That requires some modifications to > the plug routines, but nothing that can’t be done.**** > > ** ** > > I’ve created two issues based on both our test results : CLOUDSTACK-2326 > and CLOUDSTACK-2327. If you have more relevant input feel free to update > the tickerts. I’ll start working on them right away.**** > > ** ** > > Cheers,**** > > ** ** > > Hugo**** > > ** ** > > ** ** > > ** ** > > ** ** > > *From:* Sheng Yang [mailto:sh...@yasker.org] > *Sent:* Friday, May 03, 2013 12:39 AM > *To:* Hugo Trippaers > *Cc:* <dev@cloudstack.apache.org> > *Subject:* Re: OVS on KVM**** > > ** ** > > After upgrade to Ubuntu 13.04(libvirt 1.0.2), vlan tag works well.**** > > ** ** > > --Sheng**** > > ** ** > > On Thu, May 2, 2013 at 11:18 AM, Sheng Yang <sh...@yasker.org> wrote:**** > > After searching I found this:**** > > ** ** > > http://libvirt.org/formatnetwork.html**** > > ** ** > > <quote>**** > > Setting VLAN tag (on supported network types only)**** > > ...**** > > <devices>**** > > <interface type='bridge'>**** > > <vlan trunk='yes'>**** > > <tag id='42'/>**** > > <tag id='47'/>**** > > </vlan>**** > > <source bridge='ovsbr0'/>**** > > <virtualport type='openvswitch'>**** > > <parameters interfaceid='09b11c53-8b5c-4eeb-8f00-d84eaa0aaa4f'/>** > ** > > </virtualport>**** > > </interface>**** > > <devices>**** > > ...**** > > If (and only if) the network type supports vlan tagging transparent to the > guest, an optional <vlan> element can specify one or more vlan tags to > apply to the traffic of all guests using this network **Since 0.10.0**. > (openvswitch and type='hostdev' SR-IOV networks do support transparent vlan > tagging of guest traffic; everything else, including standard linux bridges > and libvirt's own virtual networks, do not support it. 802.1Qbh (vn-link) > and 802.1Qbg (VEPA) switches provide their own way (outside of libvirt) to > tag guest traffic onto specific vlans.) As expected, the tag attribute > specifies which vlan tag to use. If a network has more than one <vlan> > element defined, it is assumed that the user wants to do VLAN trunking > using all the specified tags. In the case that vlan trunking with a single > tag is desired, the optional attribute trunk='yes' can be added to the vlan > element.**** > > </quote>**** > > ** ** > > I am using 0.9.13(with ubuntu 12.10). Does that means we need newer > version?**** > > ** ** > > --Sheng**** > > ** ** > > ** ** > > On Thu, May 2, 2013 at 10:55 AM, Sheng Yang <sh...@yasker.org> wrote:**** > > I DO SEE the tag on VM profile when agent start, but I didn't see them on > OVS ports.**** > > ** ** > > 2013-05-01 18:04:44,702{GMT} DEBUG [kvm.resource.LibvirtComputingResource] > (agentRequest-Handler-5:) starting v-2-VM: <domain type='kvm'>**** > > <name>v-2-VM</name>**** > > <uuid>1422832d-be18-352a-a08a-9bbff40e0d14</uuid>**** > > <description>Debian GNU/Linux 5.0 (32-bit)</description>**** > > <clock offset='utc'>**** > > </clock>**** > > <features>**** > > <pae/>**** > > <apic/>**** > > <acpi/>**** > > </features>**** > > <devices>**** > > <emulator>/usr/bin/kvm</emulator>**** > > <interface type='bridge'>**** > > <source bridge='cloud0'/>**** > > <mac address='0e:00:a9:fe:02:45'/>**** > > <model type='virtio'/>**** > > <virtualport type='openvswitch'>**** > > </virtualport>**** > > </interface>**** > > <interface type='bridge'>**** > > <source bridge='cloudbr0'/>**** > > <mac address='06:f7:5c:00:00:06'/>**** > > <model type='virtio'/>**** > > <virtualport type='openvswitch'>**** > > </virtualport>**** > > </interface>**** > > <interface type='bridge'>**** > > <source bridge='cloudbr0'/>**** > > <mac address='06:4c:12:00:00:1a'/>**** > > <model type='virtio'/>**** > > <virtualport type='openvswitch'>**** > > </virtualport>**** > > <vlan trunk='no'>**** > > <tag id='1610'/> <----------- here**** > > </vlan></interface>**** > > <serial type='pty'>**** > > <target port='0'/>**** > > </serial>**** > > <graphics type='vnc' autoport='yes' listen='' />**** > > <disk device='disk' type='file'>**** > > <driver name='qemu' type='qcow2' cache='none' />**** > > <source > file='/mnt/20ad978d-a581-3a08-95fd-c2a45417513c/2f12ce26-4e4b-4d6e-b77e-1c45afff58e9'/> > **** > > <target dev='vda' bus='virtio'/>**** > > </disk>**** > > <disk device='cdrom' type='file'>**** > > <driver name='qemu' type='raw' cache='none' />**** > > <source file='/usr/share/cloudstack-common/vms/systemvm.iso'/>**** > > <target dev='hdc' bus='ide'/>**** > > </disk>**** > > <console type='pty'>**** > > <target port='0'/>**** > > </console>**** > > <input type='tablet' bus='usb'/>**** > > <channel type='unix'>**** > > <source mode='bind' path='/var/lib/libvirt/qemu/v-2-VM.agent'/>**** > > <target type='virtio' name='v-2-VM.vport'/>**** > > <address type='virtio-serial'/>**** > > </channel>**** > > </devices>**** > > <memory>1048576</memory>**** > > <vcpu>1</vcpu>**** > > <os>**** > > <type arch='x86_64' machine='pc'>hvm</type>**** > > <boot dev='cdrom'/>**** > > <boot dev='hd'/>**** > > </os>**** > > <cputune>**** > > <shares>500</shares>**** > > </cputune>**** > > <on_reboot>restart</on_reboot>**** > > <on_poweroff>destroy</on_poweroff>**** > > <on_crash>destroy</on_crash>**** > > </domain>**** > > ** ** > > After this, vnet2 should be tagged with 1610, but:**** > > ** ** > > root@yasker-box1:~# ovs-vsctl list port vnet2**** > > _uuid : 012a6140-bd87-4917-84cc-7190829c695a**** > > bond_downdelay : 0**** > > bond_fake_iface : false**** > > bond_mode : []**** > > bond_updelay : 0**** > > external_ids : {}**** > > fake_bridge : false**** > > interfaces : [95bcf67b-12c1-44e5-87da-5663c6644da3]**** > > lacp : []**** > > mac : []**** > > name : "vnet2"**** > > other_config : {}**** > > qos : []**** > > statistics : {}**** > > status : {}**** > > tag : []**** > > trunks : []**** > > vlan_mode : []**** > > ** ** > > So it cannot access the public network.**** > > ** ** > > After:**** > > ** ** > > root@yasker-box1:~# ovs-vsctl set port vnet2 tag=1610**** > > root@yasker-box1:~# ovs-vsctl list port vnet2**** > > _uuid : 012a6140-bd87-4917-84cc-7190829c695a**** > > bond_downdelay : 0**** > > bond_fake_iface : false**** > > bond_mode : []**** > > bond_updelay : 0**** > > external_ids : {}**** > > fake_bridge : false**** > > interfaces : [95bcf67b-12c1-44e5-87da-5663c6644da3]**** > > lacp : []**** > > mac : []**** > > name : "vnet2"**** > > other_config : {}**** > > qos : []**** > > statistics : {}**** > > status : {}**** > > tag : 1610**** > > trunks : []**** > > vlan_mode : []**** > > ** ** > > It can access the public network with vlan 1610.**** > > ** ** > > --Sheng**** > > ** ** > > ** ** > > On Thu, May 2, 2013 at 4:34 AM, Hugo Trippaers < > htrippa...@schubergphilis.com> wrote:**** > > Hey Sheng,**** > > **** > > The tagging is done by libvirt. Can you check your agent.log?**** > > **** > > I would have expected an entry in the log file looking like this > ‘s_logger.debug("creating a vlan dev and bridge for public traffic per > traffic label " + trafficLabel);’**** > > **** > > Also the XML document for the vif sent to libvirt should have the > following tag ‘<vlan trunk='no'>\n<tag id='" + _vlanTag + "'/>\n</vlan>"’* > *** > > **** > > What are your traffic labels set to for kvm? Could you share your > agent.properties?**** > > **** > > Cheers,**** > > **** > > Hugo**** > > **** > > *From:* Sheng Yang [mailto:sh...@yasker.org] > *Sent:* Thursday, May 02, 2013 3:17 AM > *To:* Hugo Trippaers; <dev@cloudstack.apache.org> > *Subject:* OVS on KVM**** > > **** > > Hi Hugo,**** > > **** > > I am trying to use OVS on KVM now, but I found all public ports are not > tagged with public vlan as it supposed to be, so any public traffic cannot > goes out. I've verified that I am using OvsVifDriver. **** > > **** > > Here is the output of ovs-vsctl show:**** > > **** > > <quote>**** > > root@yasker-box1:~/kvm-agent# ovs-vsctl show**** > > 02281b72-131c-4b24-b191-fb1bb7fe186d**** > > Bridge "cloud0"**** > > Port "cloud0"**** > > Interface "cloud0"**** > > type: internal**** > > Port "vnet3"**** > > Interface "vnet3"**** > > Port "vnet0"**** > > Interface "vnet0"**** > > Bridge "cloudbr0"**** > > Port "vnet2"**** > > Interface "vnet2"**** > > Port "vnet6"**** > > Interface "vnet6"**** > > Port "vnet4"**** > > Interface "vnet4"**** > > Port "vnet9"**** > > Interface "vnet9"**** > > Port "vnet10"**** > > Interface "vnet10"**** > > Port "vnet1"**** > > Interface "vnet1"**** > > Port "cloudbr0"**** > > Interface "cloudbr0"**** > > type: internal**** > > Port "eth0"**** > > Interface "eth0"**** > > Port "vnet5"**** > > Interface "vnet5"**** > > ovs_version: "1.4.3"**** > > </quote>**** > > **** > > I've checked the Installation guide, it use different bridge for different > vlan. But would that be the only way to work? Because we can have different > public vlans. Maybe I got some setup wrong...**** > > **** > > Any comments?**** > > **** > > Thanks!**** > > **** > > --Sheng**** > > ** ** > > ** ** > > ** ** >
