Hello Mark, On Thu, Nov 21, 2013 at 01:01:17PM -0600, Mark Biggers wrote: > Hello Valentin, thanks for the reply. > > On 11/21/2013 03:30 AM, Valentin Bud wrote: > Hello Mark, > > Before pointing you to the problem I think your config has you should also > check that you have routing enabled in the machine > > Are you speaking of "ip route" routes? Or some sysconf variable? Or, > route(s) on the VMs themselves??
I was actually speaking about IP Forwarding, to be precise. That is attained via a sysctl. It should be set to 1. To make the changes persistent after reboot you have /etc/sysctl.conf. NOTE: Paths, configs might be different, I have once used OpenSuse, a billion years ago. # sysctl net.ipv4.ip_forward > > I have attempted "ip route", to route to the VMs 10.0.0.0/24 network.... > Where's 10.0.0.1 supposed be -- on the vbr0? Yes, 10.0.0.1 should be configured on the vbr0 interface and eth0 shouldn't be part of your Open vSwitch. > > and if you want Internet connectivity for VMs also NAT vbr0 over eth0. > > I want (1) internet connectivity to the VMs (VLANs) and connectivity "out" of > the VMs network. Not sure how to get this going, though it appears the > OVSwitch has all the MAC-addr info, for the VMs... Don't forget to NAT your 10/24 network over eth0 to have Internet connectivity available in the machines. Also you should have a DNS server running at 10.0.0.1 or change DNS from vnet to some recursive DNS server you have in your network. I suspect 192.168.1.1 is your DNS recursive server. If you want services from the VMs accessible from 192.168.1/24 network you can forward ports to specific VMs from your laptop (host) using iptables. Let's say you want to access a Web Server located on the VM with IP Address 10.0.0.100. You'd have to DNAT port 80 from eth0 IP Address to 10.0.0.100 port 80. And have your webserver available at http://eth0.ip.addr.ess. > > This might not be the case if your router has routes to 10/24 network through > your laptop's > eth0 interface. > Currently, it does not. Just attempted, and can't ping to 10.0.0.3 VM. > (output, below). If you want to access your VMs from the 192.168.1.0/24 you shouldn't NAT the 10/24 network over eth0 but configure your Netgear router with a route to 10/24 via your eth0's IP Address. I would recommend to enter a MAC address - IP binding in the router so your laptop receives the same IP Address on eth0 every time it connects to the network. > > Reading once again your config I see you've inserted eth0 in the vbr0 OvS > bridge and that it has an IP > address from the 192.168.1.0/24<http://192.168.1.0/24> network. I suspect > that is your local network. > > Yes, 192.168.1.0/24 is my "external" (laptop) network, including a Netgear > router at 192.168.1.1. > > Do you have connectivity between your VMs using this setup? You should from > what your setup tells me. > > The VMs, at 10.0.0.3 and 10.0.0.4 can ping each other, and ssh works between > them just fine. They can only "see" the 10.0.0.0/24 network, and can't ping > 10.0.0.1. They can't ping 10.0.0.1 because that IP Address hasn't been configured on any interface. Please follow the steps I have outlined in my previous E-Mail if you want NAT-ed setup. That should get you started in no time. Here they are: # ovs-vsctl del-port vbr0 eth0 # dhclient eth0 ( or set its IP address manually) # ovs-vsctl set Port vbr0 tag=0 # ip addr add 10.0.0.1/24 dev vbr0 # iptables -t nat -A POSTROUTING -s 10.0.0.0/24 -d 0.0.0.0/0 -o eth0 -j MASQUERADE > > thank you, > ----mark Hope it helps. Cheers and Goodwill, > > r...@sealion.ine.corp:~<mailto:r...@sealion.ine.corp:~> # route add -net > 10.0.0.0/24 gw 192.168.1.250 dev eth0 > SIOCADDRT: Network is unreachable > r...@sealion.ine.corp:~<mailto:r...@sealion.ine.corp:~> # route add -net > 10.0.0.0/24 gw 192.168.1.250 dev vbr0 > r...@sealion.ine.corp:~<mailto:r...@sealion.ine.corp:~> # ping 10.0.0.3 > PING 10.0.0.3 (10.0.0.3) 56(84) bytes of data. > From 192.168.1.250 icmp_seq=1 Destination Host Unreachable > From 192.168.1.250 icmp_seq=2 Destination Host Unreachable > From 192.168.1.250 icmp_seq=3 Destination Host Unreachable > From 192.168.1.250 icmp_seq=4 Destination Host Unreachable > ^C > --- 10.0.0.3 ping statistics --- > 4 packets transmitted, 0 received, +4 errors, 100% packet loss, time 2999ms > pipe 4 > r...@sealion.ine.corp:~<mailto:r...@sealion.ine.corp:~> # netstat -nr > Kernel IP routing table > Destination Gateway Genmask Flags MSS Window irtt Iface > 0.0.0.0 192.168.1.1 0.0.0.0 UG 0 0 0 vbr0 > 10.0.0.0 192.168.1.250 255.255.255.0 UG 0 0 0 vbr0 > 67.139.46.149 192.168.1.1 255.255.255.255 UGH 0 0 0 vbr0 > 127.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 lo > 127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo > 192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 vbr0 > r...@sealion.ine.corp:~<mailto:r...@sealion.ine.corp:~> # > > ------------------------------------------------------------------------------------------------------------------------------------------------- > On Thu, Nov 21, 2013 at 12:31 AM, Mark Biggers > <mbigg...@ine.com<mailto:mbigg...@ine.com>> wrote: > Hello ONE team, (RESENT -- dropped in bad "onevm show NN" data, earlier -- > corrected, below!) > > I have "passed" on the ebtables configuration for networking in 4.2 ONE. > We'll need OpenVSwitch anyways to manage the VMs VLANS, so I have moved on. > > I *think* I have an almost working OpenVSwitch configuration. Must I > manually create "flows" for each VM/MAC-addr to enable IP traffic, across the > OVS "vbridge" (vbr0), in this case? > > No, you shouldn't have to manually add flows for IP traffic to flow between > your machines and/or Internet. > > > The info on my new (OVS networking) setup, is included, at the end of this > message. Thank you. (The platform is still openSUSE 12.3 on a Thinkpad > W530...) > > > On 11/19/2013 05:43 AM, Jaime Melis wrote: > Hi Mark, > > I have the feeling the NAT policies are interfering with this. Can you try > without applying NAT rules? > > > On Wed, Nov 13, 2013 at 9:08 PM, Mark Biggers > <mbigg...@ine.com<mailto:mbigg...@ine.com>> wrote: > The subject says it all. I am available on IRC -- see my signature, and > Google chat. > > I can get no "networking across a bridge" working, for the ONE "ebtables" > model. > > =============== edited out.... > > -- > Jaime Melis > Project Engineer > OpenNebula - Flexible Enterprise Cloud Made Simple > www.OpenNebula.org<http://www.OpenNebula.org> | > jme...@opennebula.org<mailto:jme...@opennebula.org> > > Script started on Wed Nov 20 16:27:05 2013 > > r...@sealion.ine.corp:one<mailto:r...@sealion.ine.corp:one> # netstat -nr > Kernel IP routing table > Destination Gateway Genmask Flags MSS Window irtt Iface > 0.0.0.0 192.168.1.1 0.0.0.0 UG 0 0 0 vbr0 > 67.139.46.149 192.168.1.1 255.255.255.255 UGH 0 0 0 vbr0 > 127.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 lo > 127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo > 192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 vbr0 > > r...@sealion.ine.corp:one<mailto:r...@sealion.ine.corp:one> # ip addr > 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN > link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 > inet 127.0.0.1/8<http://127.0.0.1/8> brd 127.255.255.255 scope host lo > 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP > qlen 1000 > link/ether 3c:97:0e:ab:0a:de brd ff:ff:ff:ff:ff:ff > 3: wlan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000 > link/ether 6c:88:14:da:0b:44 brd ff:ff:ff:ff:ff:ff > 4: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN > link/ether 0a:0e:fd:bb:5a:8a brd ff:ff:ff:ff:ff:ff > 7: vbr0: <BROADCAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN > link/ether 3c:97:0e:ab:0a:de brd ff:ff:ff:ff:ff:ff > inet 192.168.1.250/24<http://192.168.1.250/24> scope global vbr0 > 12: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state > UNKNOWN qlen 500 > link/ether fe:00:0a:00:00:03 brd ff:ff:ff:ff:ff:ff > 13: vnet1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state > UNKNOWN qlen 500 > link/ether fe:00:0a:00:00:04 brd ff:ff:ff:ff:ff:ff > > r...@sealion.ine.corp:one<mailto:r...@sealion.ine.corp:one> # BRIDGE_DEV=vbr0 > r...@sealion.ine.corp:one<mailto:r...@sealion.ine.corp:one> # sudo ovs-ofctl > dump-desc $BRIDGE_DEV > OFPST_DESC reply (xid=0x2): > Manufacturer: Nicira, Inc. > Hardware: Open vSwitch > Software: 1.11.0 > Serial Num: None > DP Description: None > > r...@sealion.ine.corp:one<mailto:r...@sealion.ine.corp:one> # sudo ovs-vsctl > show > 001119d6-32d7-4db8-8015-229b271cca6a > Bridge "vbr0" > Controller "ptcp:" > fail_mode: standalone > Port "vnet0" > tag: 0 > Interface "vnet0" > Port "vnet1" > tag: 0 > Interface "vnet1" > Port "eth0" > Interface "eth0" > Port "vbr0" > Interface "vbr0" > type: internal > ovs_version: "1.11.0" > > Open vSwitch is a multilayer switch meaning that each VLAN has to have a > Layer 3 interface with IP address from the > network you plan to use in order to connect to other networks. > > Your VLAN 0 doesn't have such an interface. > > Port "vbr0" > *tag: 0* > Interface "vbr0" > type: internal > > Also vbr0 should have an IP address from 10/24 network, .1 to be exact so > your VMs can have Internet connectivity. > > Another thing I would like to point out is the use of VLAN 0. That is a > special kind of VLAN, accepts and processes tagged > and un-tagged traffic. I would advise you to use another VLAN and keep the > eth0 interface out of that bridge. > > In that case your machine would act as a router between 10/24 and the > Internet via eth0. > > I don't know if it works but you could also try to add another internal > interface to your bridge and assign it > 10.0.0.1/24<http://10.0.0.1/24> and tag 0. Then you'd have to MASQUERADE over > vbr0 interface. > > The following set of commands would get your OvS networks up and running. > > # ovs-vsctl del-port vbr0 eth0 > > # dhclient eth0 ( or set its IP address manually) > > # ovs-vsctl set Port vbr0 tag=0 > > # ip addr add 10.0.0.1/24<http://10.0.0.1/24> dev vbr0 > > # iptables -A INPUT -o eth0 -j MASQUERADE > > You should also have a running resolver if you want your machines to resolve > DNS names because > you've passed 10.0.0.1 as DNS in the network defined inside OpenNebula. > > I hop the above makes sense and helps you to some extent. > > Cheers and Goodwill, > > > r...@sealion.ine.corp:one<mailto:r...@sealion.ine.corp:one> # sudo ovs-ofctl > show $BRIDGE_DEV > OFPT_FEATURES_REPLY (xid=0x2): dpid:00003c970eab0ade > n_tables:254, n_buffers:256 > capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP > actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC SET_DL_DST > SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE > 1(eth0): addr:3c:97:0e:ab:0a:de > config: 0 > state: STP_FORWARD > current: 1GB-FD COPPER AUTO_NEG > advertised: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD COPPER AUTO_NEG > supported: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD COPPER AUTO_NEG > speed: 1000 Mbps now, 1000 Mbps max > 2(vnet0): addr:fe:00:0a:00:00:03 > config: 0 > state: 0 > current: 10MB-FD COPPER > speed: 10 Mbps now, 0 Mbps max > 3(vnet1): addr:fe:00:0a:00:00:04 > config: 0 > state: 0 > current: 10MB-FD COPPER > speed: 10 Mbps now, 0 Mbps max > LOCAL(vbr0): addr:3c:97:0e:ab:0a:de > config: 0 > state: 0 > speed: 0 Mbps now, 0 Mbps max > OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0 > > r...@sealion.ine.corp:one<mailto:r...@sealion.ine.corp:one> # sudo ovs-ofctl > dump-flows $BRIDGE_DEV > NXST_FLOW reply (xid=0x4): > cookie=0x0, duration=8382.092s, table=0, n_packets=4, n_bytes=240, > idle_age=8381, priority=40000,in_port=2,dl_src=02:00:0a:00:00:03 > actions=NORMAL > cookie=0x0, duration=6882.175s, table=0, n_packets=4, n_bytes=240, > idle_age=6881, priority=40000,in_port=3,dl_src=02:00:0a:00:00:04 > actions=NORMAL > cookie=0x0, duration=6882.159s, table=0, n_packets=0, n_bytes=0, > idle_age=6882, priority=39000,in_port=3 actions=drop > cookie=0x0, duration=8382.072s, table=0, n_packets=0, n_bytes=0, > idle_age=8382, priority=39000,in_port=2 actions=drop > cookie=0x0, duration=13024.731s, table=0, n_packets=157158, > n_bytes=44610731, idle_age=0, priority=0 actions=NORMAL > > r...@sealion.ine.corp:one<mailto:r...@sealion.ine.corp:one> # sudo -u > oneadmin onehost show 7 > HOST 7 INFORMATION > ID : 7 > NAME : sealion.ine.corp > CLUSTER : - > STATE : MONITORED > IM_MAD : kvm > VM_MAD : kvm > VN_MAD : ovswitch > LAST MONITORING TIME : 11/20 16:39:02 > > HOST SHARES > TOTAL MEM : 31G > USED MEM (REAL) : 1.6G > USED MEM (ALLOCATED) : 1.5G > TOTAL CPU : 800 > USED CPU (REAL) : 7 > USED CPU (ALLOCATED) : 200 > RUNNING VMS : 2 > > MONITORING INFORMATION > ARCH="x86_64" > CPUSPEED="3000" > FREECPU="792.8" > FREEMEMORY="30893828" > HOSTNAME="sealion.ine.corp" > HYPERVISOR="kvm" > MODELNAME="Intel(R) Core(TM) i7-3940XM CPU @ 3.00GHz" > NETRX="0" > NETTX="0" > TOTALCPU="800" > TOTALMEMORY="32557228" > USEDCPU="7.2000000000000455" > USEDMEMORY="1663400" > > VIRTUAL MACHINES > > ID USER GROUP NAME STAT UCPU UMEM HOST TIME > 42 oneadmin oneadmin vyatta-router 32mrunn 0 768M > sealion.in<http://sealion.in> 0d 16h48 > 43 oneadmin oneadmin vyatta-router-0 32mrunn 0 768M > sealion.in<http://sealion.in> 0d 16h48 > > > oneadmin@sealion:~ > onevm list > ID USER GROUP NAME STAT UCPU UMEM HOST > TIME > 41 oneadmin oneadmin one-vr42 stop 1 768M 6d > 00h24 > 42 oneadmin oneadmin vyatta-router runn 0 768M > sealion.in<http://sealion.in> 0d 16h48 > 43 oneadmin oneadmin vyatta-router-0 runn 0 768M > sealion.in<http://sealion.in> 0d 16h48 > > oneadmin@sealion:~ > onevnet show 9 > VIRTUAL NETWORK 9 INFORMATION > ID : 9 > NAME : ovsnet_0_0 > USER : oneadmin > GROUP : oneadmin > CLUSTER : - > TYPE : RANGED > BRIDGE : vbr0 > VLAN : Yes > VLAN ID : 0 > USED LEASES : 3 > > PERMISSIONS > OWNER : um- > GROUP : --- > OTHER : --- > > VIRTUAL NETWORK TEMPLATE > DNS="10.0.0.1" > GATEWAY="10.0.0.1" > NETWORK_ADDRESS="10.0.0.0/24<http://10.0.0.0/24>" > NETWORK_MASK="255.255.255.0" > > RANGE > IP_START : 10.0.0.1 > IP_END : 10.0.0.254 > > USED LEASES > LEASE=[ MAC="02:00:0a:00:00:02", IP="10.0.0.2", > IP6_LINK="fe80::400:aff:fe00:2", USED="1", VID="41" ] > LEASE=[ MAC="02:00:0a:00:00:03", IP="10.0.0.3", > IP6_LINK="fe80::400:aff:fe00:3", USED="1", VID="42" ] > LEASE=[ MAC="02:00:0a:00:00:04", IP="10.0.0.4", > IP6_LINK="fe80::400:aff:fe00:4", USED="1", VID="43" ] > > VIRTUAL MACHINES > > ID USER GROUP NAME STAT UCPU UMEM HOST > TIME > 41 oneadmin oneadmin one-vr42 stop 1 768M 6d > 00h26 > 42 oneadmin oneadmin vyatta-router runn 0 768M > sealion.in<http://sealion.in> 0d 16h48 > 43 oneadmin oneadmin vyatta-router-0 runn 0 768M > sealion.in<http://sealion.in> 0d 16h48 > > Script done on Wed 20 Nov 2013 04:59:17 PM EST > Script started on Wed 20 Nov 2013 05:23:22 PM EST > > oneadmin@sealion:~ > onevm show 42 > VIRTUAL MACHINE 42 INFORMATION > ID : 42 > NAME : vyatta-router > USER : oneadmin > GROUP : oneadmin > STATE : ACTIVE > LCM_STATE : RUNNING > RESCHED : No > HOST : sealion.ine.corp > START TIME : 11/14 16:55:09 > END TIME : 11/15 09:43:24 > DEPLOY ID : one-42 > > VIRTUAL MACHINE MONITORING > USED MEMORY : 768M > USED CPU : 0 > NET_TX : 0K > NET_RX : 533K > > PERMISSIONS > OWNER : um- > GROUP : --- > OTHER : --- > > VM DISKS > ID TARGET IMAGE TYPE SAVE SAVE_AS > 0 vda Vyatta Core 6.5R1 - kvm file NO - > > VM NICS > ID NETWORK VLAN BRIDGE IP MAC > 0 ovsnet_0_0 yes vbr0 10.0.0.3 02:00:0a:00:00:03 > fe80::400:aff:fe00:3 > > VIRTUAL MACHINE HISTORY > SEQ HOST ACTION REAS START TIME > PROLOG > 0 sealion.ine.cor stop user 11/14 16:55:10 0d 00h14m > 0h00m23s > 1 sealion.ine.cor none erro 11/15 09:37:31 0d 00h00m > 0h00m00s > 2 sealion.ine.cor none erro 11/15 09:43:01 0d 00h00m > 0h00m23s > 3 sealion.ine.cor stop user 11/15 14:16:01 0d 03h15m > 0h00m22s > 4 sealion.ine.cor stop user 11/20 11:27:59 0d 02h40m > 0h00m00s > 5 sealion.ine.cor none none 11/20 14:08:59 0d 03h14m > 0h00m00s > > USER TEMPLATE > ERROR="Fri Nov 15 09:43:24 2013 : Error executing image transfer script: > Error creating ISO symbolic link" > > VIRTUAL MACHINE TEMPLATE > CONTEXT=[ > DISK_ID="1", > HOSTNAME="MAINHOST", > IMAGE_UNAME="oneadmin", > IP_GEN="192.168.122.42", > TARGET="vdb" ] > CPU="1" > GRAPHICS=[ > LISTEN="0.0.0.0", > PORT="5942", > TYPE="vnc" ] > MEMORY="768" > OS=[ > ARCH="i686" ] > TEMPLATE_ID="44" > VMID="42" > > oneadmin@sealion:~ > onevm show 43 > VIRTUAL MACHINE 43 INFORMATION > ID : 43 > NAME : vyatta-router-02 > USER : oneadmin > GROUP : oneadmin > STATE : ACTIVE > LCM_STATE : RUNNING > RESCHED : No > HOST : sealion.ine.corp > START TIME : 11/14 16:55:54 > END TIME : 11/15 09:43:54 > DEPLOY ID : one-43 > > VIRTUAL MACHINE MONITORING > USED MEMORY : 768M > USED CPU : 0 > NET_TX : 0K > NET_RX : 464K > > PERMISSIONS > OWNER : um- > GROUP : --- > OTHER : --- > > VM DISKS > ID TARGET IMAGE TYPE SAVE SAVE_AS > 0 vda Vyatta Core 6.5R1 - kvm file NO - > > VM NICS > ID NETWORK VLAN BRIDGE IP MAC > 0 ovsnet_0_0 yes vbr0 10.0.0.4 02:00:0a:00:00:04 > fe80::400:aff:fe00:4 > > VIRTUAL MACHINE HISTORY > SEQ HOST ACTION REAS START TIME > PROLOG > 0 sealion.ine.cor stop user 11/14 16:56:10 0d 00h14m > 0h00m21s > 1 sealion.ine.cor none erro 11/15 09:38:01 0d 00h00m > 0h00m00s > 2 sealion.ine.cor none erro 11/15 09:43:31 0d 00h00m > 0h00m22s > 3 sealion.ine.cor stop user 11/15 14:17:01 0d 03h14m > 0h00m24s > 4 sealion.ine.cor stop user 11/20 11:28:29 0d 02h39m > 0h00m00s > 5 sealion.ine.cor none none 11/20 14:33:59 0d 02h49m > 0h00m00s > > USER TEMPLATE > ERROR="Fri Nov 15 09:43:53 2013 : Error executing image transfer script: > Error creating ISO symbolic link" > > VIRTUAL MACHINE TEMPLATE > CONTEXT=[ > DISK_ID="1", > HOSTNAME="MAINHOST", > IMAGE_UNAME="oneadmin", > IP_GEN="192.168.122.43", > TARGET="vdb" ] > CPU="1" > GRAPHICS=[ > LISTEN="0.0.0.0", > PORT="5943", > TYPE="vnc" ] > MEMORY="768" > OS=[ > ARCH="i686" ] > TEMPLATE_ID="44" > VMID="43" > > oneadmin@sealion:~ > exit > exit > > Script done on Wed 20 Nov 2013 05:23:33 PM EST > 1 > > _______________________________________________ > Users mailing list > Users@lists.opennebula.org<mailto:Users@lists.opennebula.org> > http://lists.opennebula.org/listinfo.cgi/users-opennebula.org > > > > > -- > Valentin Bud > http://databus.pro | valen...@databus.pro<mailto:valen...@databus.pro> > _______________________________________________ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
