'cloudVirBr' is hard-coded in
/usr/lib[64]/cloud/agent/scripts/vm/network/vnet/modifyvlan.sh in function
addVlan().
What I don't understand is how the script is even working. It does
vconfig add ethN V
brctl addbr brV
brctl addif brV ethN.V
Furthermore, in scripts/storage/qcow2/modifyvlan.sh (anyone looked at why
there are 3 different copies of this shell script?!?) it does the same
thing but using bonds.
But when I run thru the sequence by hand (eth or bond) it doesn't pass
traffic. The only way I get it to work is:
brctl addbr br
brctl addif br ethN
vconfig add ethN V
ifconfig ethN.V <ipaddr> <netmask>
And I can define as many ethN.[1..9] as I like with all different VLAN IDs
and attach them to the single bridge. My way also works when adding a bond
to the bridge and adding as many VLAN tagged bonds as I like.
But the guest isn't receiving response packets to things like ARP. When I
sniff bond0 on the hypervisor I see the VLAN tag in the frame AS LONG AS
in the guest I configure ethN.V; as would be logical. The ARP reply also
gets as far as bond0 on the way back. Using tcpdump on interface 'vnet0'
doesn't show any packets going in either direction though they are
transiting bond0 just fine. I can see the ARP replies on bond0.N which is
again as expected.
If I can sniff bond0, how come I can't sniff vnet0?
There is something odd in the 'brctl showmacs <bridge>' output which
probably explains why vnet0 sees no packets.
The guest's MAC is 52:54:00:c9:8d:d1. It's on port #2, marked local=no and
eventually ages out. But there is another MAC on port 2 that is the same
except it starts with 'fe'. This one is marked local=yes and doesn't age
out. It seems this is the MAC given to the host-side of the tap. So I can
understand it being semi-permanent. So then why is the the guest's
(?private?) MAC address showing up in the bridge? If the ARP response
packets have the wrong (guest's private MAC) in them (and they do) that
would explain why it can't return to the guest since it "missed" the
on-ramp that is the hypervisor's end of the TUN interface.
Looks like a mistake in the tap code that's forgetting to rewrite the MAC
value? Maybe because there is a VLAN tag in the frame?
At first there were no "QEMU networks" defined since I had undefined the
default one since I wasn't interested in NAT.
[root@kvm1b ~]# virsh net-list
Name State Autostart
-----------------------------------------
But seeing as that might be a problem I defined one for bridging.
<network>
<name>bridge-shared</name>
<uuid>0824e08b-bb0e-452a-e2c9-dcca52af2341</uuid>
<forward mode='bridge'/>
<bridge name='shared' />
</network>
Now I have a network
[root@kvm1b networks]# virsh net-list
Name State Autostart
-----------------------------------------
bridge-shared active yes
This is the interface section from my guest's XML.
<interface type='bridge'>
<mac address='52:54:00:c9:8d:d1'/>
<source bridge='shared'/>
<model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03'
function='0x0'/>
</interface>
[root@kvm1b ~]# virsh domiflist test
Interface Type Source Model MAC
-------------------------------------------------------
vnet0 bridge shared virtio 52:54:00:c9:8d:d1
[root@kvm1b networks]# virsh iface-list
Name State MAC Address
--------------------------------------------
bond0.39 active 00:25:90:4c:ba:92
bond0.398 active 00:25:90:4c:ba:92
eth2 active 90:e2:ba:04:f6:28
eth3 active 90:e2:ba:04:f6:29
lo active 00:00:00:00:00:00
shared active 00:25:90:4c:ba:92
and for good measure
[root@kvm1b networks]# virsh iface-dumpxml shared
<interface type='bridge' name='shared'>
<protocol family='ipv6'>
<ip address='fe80::225:90ff:fe4c:ba92' prefix='64'/>
</protocol>
<bridge>
<interface type='bond' name='bond0'>
<bond>
<interface type='ethernet' name='eth0'>
<mac address='00:25:90:4c:ba:92'/>
</interface>
<interface type='ethernet' name='eth1'>
<mac address='00:25:90:4c:ba:92'/>
</interface>
</bond>
</interface>
<interface type='ethernet' name='vnet0'>
<mac address='fe:54:00:c9:8d:d1'/>
</interface>
</bridge>
</interface>
[root@kvm1b networks]# virsh domifstat test vnet0
vnet0 rx_bytes 44592
vnet0 rx_packets 847
vnet0 rx_errs 0
vnet0 rx_drop 0
vnet0 tx_bytes 2150
vnet0 tx_packets 35
vnet0 tx_errs 0
vnet0 tx_drop 0
So you'd think something was getting back to the guest. I just don't know
what. tcpdump running inside the guest on eth0 shows nothing at all, just
like sniffing vnet0.
The MAC hitting the wire on the hypervisor's bond0 is still the guest's
private MAC and it seems the response packets don't know where to go once
they come back to the bridge.
On a different note I ran across this:
http://publib.boulder.ibm.com/infocenter/lnxinfo/v3r0m0/topic/liaat/liaatbptap.htm
So QEMU has it's own bastard redefinition of the term "VLAN"? WTF!
