[ https://issues.apache.org/jira/browse/CLOUDSTACK-9974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641195#comment-16641195 ]
Rohit Yadav commented on CLOUDSTACK-9974: ----------------------------------------- Hi [~rhuang] , can you test latest 4.11.1.0 release and advise if you're able to reproduce the error? Jira usage is deprecated, please report future comments/issues/bugs here: https://github.com/apache/cloudstack/issues > VR got 4 nics instead of 3; public IP assigned on eth3 instead of eth2; SNAT > not working > ---------------------------------------------------------------------------------------- > > Key: CLOUDSTACK-9974 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9974 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: cloudstack-agent, KVM, Virtual Router > Affects Versions: 4.9.2.0 > Environment: Cloudstack 4.9.2.0 > 1 cluster Ubuntu 16.04 with KVM > 1 cluster XenServer 6.5SP1 > Zone with Advanced Network > Reporter: Francois Scheurer > Priority: Critical > > Dear Cloudstack contributors > We are currently testing CS 4.9.2 with XenServer (6.5) and KVM (Ubuntu 16.04). > We noticed that the VR on KVM are created with 4 nics (eth0-eth3) instead of > 3. > The bad effect is that SNAT is not working, so the VM's cannot access > internet... > As you can see, the public ip 217.71.92.215 ha an SNAT on eth2 but is > allocated to eth3: > root@r-4862-VM:~# ip a > 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN > link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 > inet 127.0.0.1/8 scope host lo > 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast > state UP qlen 1000 > link/ether 02:00:70:65:00:02 brd ff:ff:ff:ff:ff:ff > inet 172.16.0.1/24 brd 172.16.0.255 scope global eth0 > 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast > state UP qlen 1000 > link/ether 0e:00:a9:fe:03:04 brd ff:ff:ff:ff:ff:ff > inet 169.254.3.4/16 brd 169.254.255.255 scope global eth1 > 4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast > state UP qlen 1000 > link/ether 06:29:76:00:02:45 brd ff:ff:ff:ff:ff:ff > 5: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast > state UP qlen 1000 > link/ether 06:1f:96:00:02:45 brd ff:ff:ff:ff:ff:ff > inet 217.71.92.215/27 brd 217.71.92.223 scope global eth3 > root@r-4862-VM:~# iptables -t nat -L -n -v > Chain PREROUTING (policy ACCEPT 45661 packets, 3361K bytes) > pkts bytes target prot opt in out source > destination > Chain INPUT (policy ACCEPT 12476 packets, 844K bytes) > pkts bytes target prot opt in out source > destination > Chain OUTPUT (policy ACCEPT 19244 packets, 1285K bytes) > pkts bytes target prot opt in out source > destination > Chain POSTROUTING (policy ACCEPT 19240 packets, 1284K bytes) > pkts bytes target prot opt in out source > destination > 4 304 SNAT all -- * eth2 0.0.0.0/0 > 0.0.0.0/0 to:217.71.92.215 > eth3 should not exist at all... > Rebooting the VR several sometimes "fixes" the issue and allocates the public > ip to eth2, but eth3 still exists. > Let's look why eth3 is created: > root@r-4862-VM:~# grep -r eth3 /etc > /etc/cloudstack/ips.json: "eth3": [ > /etc/cloudstack/ips.json: "device": "eth3", > /etc/iptables/iptables-secstorage:-A INPUT -i eth3 -m state --state > RELATED,ESTABLISHED -j ACCEPT > /etc/iptables/router_rules.v4:-A PREROUTING -i eth3 -m state --state > NEW -j CONNMARK --set-xmark 0x3/0xffffffff > /etc/iptables/router_rules.v4:-A INPUT -i eth3 -m state --state > RELATED,ESTABLISHED -j ACCEPT > /etc/iptables/router_rules.v4:-A FORWARD -i eth3 -o eth0 -m state > --state RELATED,ESTABLISHED -j ACCEPT > /etc/iptables/router_rules.v4:-A FORWARD -i eth0 -o eth3 -j > FW_OUTBOUND > /etc/iproute2/rt_tables:3 Table_eth3 > /etc/init.d/cloud-early-config: echo "auto eth3" >> > /etc/network/interfaces > /etc/init.d/cloud-early-config: [ -n "$MTU" ] && ifconfig eth3 mtu > $MTU && echo " mtu $MTU" >> /etc/network/interfaces > root@r-4862-VM:~# cat /etc/cloudstack/ips.json > { > "eth0": [ > { > "add": true, > "broadcast": "172.16.0.255", > "cidr": "172.16.0.1/24", > "device": "eth0", > "gateway": "None", > "netmask": "255.255.255.0", > "network": "172.16.0.0/24", > "nic_dev_id": "0", > "nw_type": "guest", > "one_to_one_nat": false, > "public_ip": "172.16.0.1", > "size": "24", > "source_nat": false > } > ], > "eth1": [ > { > "add": true, > "broadcast": "169.254.255.255", > "cidr": "169.254.3.4/16", > "device": "eth1", > "gateway": "None", > "netmask": "255.255.0.0", > "network": "169.254.0.0/16", > "nic_dev_id": "1", > "nw_type": "control", > "one_to_one_nat": false, > "public_ip": "169.254.3.4", > "size": "16", > "source_nat": false > } > ], > "eth2": [], > "eth3": [ > { > "add": true, > "broadcast": "217.71.92.223", > "cidr": "217.71.92.215/27", > "device": "eth3", > "first_i_p": true, > "gateway": "217.71.92.193", > "netmask": "255.255.255.224", > "network": "217.71.92.192/27", > "new_nic": true, > "nic_dev_id": 3, > "nw_type": "public", > "one_to_one_nat": false, > "public_ip": "217.71.92.215", > "size": "27", > "source_nat": true, > "vif_mac_address": "06:1f:96:00:02:45" > } > ], > "id": "ips" > If we look at the management-server.log we can find a mismatch between > the mac address of eth2: "mac":"06:29:76:00:02:45", > and the mac of the public ip: "vifMacAddress":"06:1f:96:00:02:45", > [root@ewcstack-man02-test]# less > /var/log/cloudstack/management/management-server.log > 2017-05-23 14:46:51,298 DEBUG > [c.c.n.r.VirtualNetworkApplianceManagerImpl] > (Work-Job-Executor-1:ctx-fbb16b26 job-30071/job-30072 ctx-0903cf9f) > (logid:418adedf) Found 1 ip(s) to apply as a part of domR > VM[DomainRouter|r-4862-VM] start. > 2017-05-23 14:46:51,309 DEBUG [c.c.n.r.CommandSetupHelper] > (Work-Job-Executor-1:ctx-fbb16b26 job-30071/job-30072 ctx-0903cf9f) > (logid:418adedf) @fsc: createRedundantAssociateIPCommands > nic=Nic[44643-4862-null-217.71.92.215] nic.getMacAddress()=06:29:76:00:02:45 > 2017-05-23 14:46:51,313 DEBUG [c.c.n.r.CommandSetupHelper] > (Work-Job-Executor-1:ctx-fbb16b26 job-30071/job-30072 ctx-0903cf9f) > (logid:418adedf) @fsc: createRedundantAssociateIPCommands > nic=Nic[44643-4862-null-217.71.92.215] baseMac=06:29:76:00:02:45 > 2017-05-23 14:46:51,326 DEBUG [c.c.n.r.CommandSetupHelper] > (Work-Job-Executor-1:ctx-fbb16b26 job-30071/job-30072 ctx-0903cf9f) > (logid:418adedf) @fsc: createRedundantAssociateIPCommands A vifMacAddress... > 2017-05-23 14:46:51,326 DEBUG [c.c.n.r.CommandSetupHelper] > (Work-Job-Executor-1:ctx-fbb16b26 job-30071/job-30072 ctx-0903cf9f) > (logid:418adedf) @fsc: createRedundantAssociateIPCommands D > vifMacAddress=06:1f:96:00:02:45 NetUtils.generateMacOnIncrease(baseMac, > ipAddr.getVlanId())=06:29:84:00:02:45 > 2017-05-23 14:46:51,326 DEBUG [c.c.n.r.CommandSetupHelper] > (Work-Job-Executor-1:ctx-fbb16b26 job-30071/job-30072 ctx-0903cf9f) > (logid:418adedf) @fsc: createRedundantAssociateIPCommands E... > 2017-05-23 14:46:51,339 DEBUG > [c.c.n.r.VirtualNetworkApplianceManagerImpl] > (Work-Job-Executor-1:ctx-fbb16b26 job-30071/job-30072 ctx-0903cf9f) > (logid:418adedf) Resending ipAssoc, port forwarding, load balancing rules as > a part of Virtual router start > 2017-05-23 14:46:51,356 DEBUG > [c.c.n.r.VirtualNetworkApplianceManagerImpl] > (Work-Job-Executor-1:ctx-fbb16b26 job-30071/job-30072 ctx-0903cf9f) > (logid:418adedf) Egress policy for the Network 414 is already defined as > Deny. So, no need to default the rule to Allow. > 2017-05-23 14:46:51,356 DEBUG > [c.c.n.r.VirtualNetworkApplianceManagerImpl] > (Work-Job-Executor-1:ctx-fbb16b26 job-30071/job-30072 ctx-0903cf9f) > (logid:418adedf) Found 0 firewall Egress rule(s) to apply as a part of domR > VM[DomainRouter|r-4862-VM] start. > 2017-05-23 14:46:51,360 DEBUG > [c.c.n.r.VirtualNetworkApplianceManagerImpl] > (Work-Job-Executor-1:ctx-fbb16b26 job-30071/job-30072 ctx-0903cf9f) > (logid:418adedf) Found 0 static nat(s) to apply as a part of domR > VM[DomainRouter|r-4862-VM] start. > 2017-05-23 14:46:51,360 DEBUG > [c.c.n.r.VirtualNetworkApplianceManagerImpl] > (Work-Job-Executor-1:ctx-fbb16b26 job-30071/job-30072 ctx-0903cf9f) > (logid:418adedf) Found 0 firewall Ingress rule(s) to apply as a part of domR > VM[DomainRouter|r-4862-VM] start. > 2017-05-23 14:46:51,360 DEBUG > [c.c.n.r.VirtualNetworkApplianceManagerImpl] > (Work-Job-Executor-1:ctx-fbb16b26 job-30071/job-30072 ctx-0903cf9f) > (logid:418adedf) Found 0 port forwarding rule(s) to apply as a part of domR > VM[DomainRouter|r-4862-VM] start. > 2017-05-23 14:46:51,360 DEBUG > [c.c.n.r.VirtualNetworkApplianceManagerImpl] > (Work-Job-Executor-1:ctx-fbb16b26 job-30071/job-30072 ctx-0903cf9f) > (logid:418adedf) Found 0 static nat rule(s) to apply as a part of domR > VM[DomainRouter|r-4862-VM] start. > 2017-05-23 14:46:51,360 DEBUG > [c.c.n.r.VirtualNetworkApplianceManagerImpl] > (Work-Job-Executor-1:ctx-fbb16b26 job-30071/job-30072 ctx-0903cf9f) > (logid:418adedf) Found 0 vpn(s) to apply as a part of domR > VM[DomainRouter|r-4862-VM] start. > 2017-05-23 14:46:51,362 DEBUG > [c.c.n.r.VirtualNetworkApplianceManagerImpl] > (Work-Job-Executor-1:ctx-fbb16b26 job-30071/job-30072 ctx-0903cf9f) > (logid:418adedf) Found 0 load balancing rule(s) to apply as a part of domR > VM[DomainRouter|r-4862-VM] start. > 2017-05-23 14:46:51,366 DEBUG > [c.c.n.r.VirtualNetworkApplianceManagerImpl] > (Work-Job-Executor-1:ctx-fbb16b26 job-30071/job-30072 ctx-0903cf9f) > (logid:418adedf) Creating monitoring services on VM[DomainRouter|r-4862-VM] > start... > 2017-05-23 14:46:51,372 DEBUG > [c.c.n.r.VirtualNetworkApplianceManagerImpl] > (Work-Job-Executor-1:ctx-fbb16b26 job-30071/job-30072 ctx-0903cf9f) > (logid:418adedf) Reapplying dhcp entries as a part of domR > VM[DomainRouter|r-4862-VM] start... > 2017-05-23 14:46:51,375 DEBUG > [c.c.n.r.VirtualNetworkApplianceManagerImpl] > (Work-Job-Executor-1:ctx-fbb16b26 job-30071/job-30072 ctx-0903cf9f) > (logid:418adedf) Reapplying vm data (userData and metaData) entries as a part > of domR VM[DomainRouter|r-4862-VM] start... > 2017-05-23 14:46:51,402 DEBUG [c.c.a.t.Request] > (Work-Job-Executor-1:ctx-fbb16b26 job-30071/job-30072 ctx-0903cf9f) > (logid:418adedf) Seq 249-4889501820440805400: Sending { Cmd , MgmtId: > 345049101620, via: 249(ewcstack-vh023-test), Ver: v1, Flags: 100011, > [{"com.cloud.agent.api.StartCommand":{"vm":{"id":4862,"name":"r-4862-VM","type":"DomainRouter","cpus":2,"minSpeed":250,"maxSpeed":1000,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian > GNU/Linux 7(64-bit)","platformEmulator":"Debian GNU/Linux > 7(64-bit)","bootArgs":" template=domP > name=r-4862","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"4zIXt8gMEhpnJ4IpJXYMzA","params":{"memoryOvercommitRatio":"1.0","cpuOvercommitRatio":"4.0"},"uuid":"0b0fd71b-f0ed-46e4-8dd0-28583beb3937","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"fea09980-6a70-4003-9d51-e615dd898f1b","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"bca3ab3e-6bbe-3c92-b289-987be43a37b2","id":102,"poolType":"RBD","host":"mon.ceph","path":"cloudstack","port":6789,"url":"RBD://mon.ceph/cloudstack/?ROLE=Primary&STOREUUID=bca3ab3e-6bbe-3c92-b289-987be43a37b2"}},"name":"ROOT-4862","size":3145728000,"path":"fea09980-6a70-4003-9d51-e615dd898f1b","volumeId":11708,"vmName":"r-4862-VM","accountId":50,"format":"RAW","provisioningType":"THIN","id":11708,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"fea09980-6a70-4003-9d51-e615dd898f1b","type":"ROOT","_details":{"managed":"false","storagePort":"6789","storageHost":"mon.ceph","volumeSize":"3145728000"}}],"nics":[{"deviceId":2,"networkRateMbps":120,"defaultNic":true,"pxeDisable":true,"nicUuid":"7838cbe1-32b6-4bec-89e4-2079a2e400c3","uuid":"d76510c2-208f-4f91-a01f-39865fac7372","ip":"217.71.92.215","netmask":"255.255.255.224","gateway":"217.71.92.193","mac":"06:29:76:00:02:45","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Public","broadcastUri":"vlan://2099","isolationUri":"vlan://2099","isSecurityGroupEnabled":false,"name":"cloudbr0"},{"deviceId":0,"networkRateMbps":120,"defaultNic":false,"pxeDisable":true,"nicUuid":"8a517b63-d955-4800-9153-1fe0733a0e49","uuid":"3e7c7ae7-4f5e-4fd8-85dd-343c08ed39f3","ip":"172.16.0.1","netmask":"255.255.255.0","mac":"02:00:70:65:00:02","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://3913","isolationUri":"vlan://3913","isSecurityGroupEnabled":false,"name":"cloudbr0"},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"dc4d31cc-596c-41f7-ad87-a4d6e5372124","uuid":"9dab4d74-c260-4c55-9dce-2018ac5b1d6e","ip":"169.254.3.4","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:03:04","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false}]},"hostIp":"10.1.0.232","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.3.4","port":3922,"interval":6,"retries":100,"name":"r-4862-VM","wait":0}},{"com.cloud.agent.api.GetDomRVersionCmd":{"accessDetails":{"router.name":"r-4862-VM","router.ip":"169.254.3.4"},"wait":0}},{},{"com.cloud.agent.api.routing.AggregationControlCommand":{"action":"Start","accessDetails":{"router.guest.ip":"172.16.0.1","router.name":"r-4862-VM","router.ip":"169.254.3.4"},"wait":0}},{"com.cloud.agent.api.routing.IpAssocCommand":{"ipAddresses":[{"accountId":50,"publicIp":"217.71.92.215","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"broadcastUri":"vlan://2099","vlanGateway":"217.71.92.193","vlanNetmask":"255.255.255.224","vifMacAddress":"06:1f:96:00:02:45","networkRate":120,"trafficType":"Public","networkName":"cloudbr0","newNic":false}],"accessDetails":{"zone.network.type":"Advanced","router.name":"r-4862-VM","router.ip":"169.254.3.4","router.guest.ip":"172.16.0.1"},"wait":0}},{"com.cloud.agent.api.routing.SetMonitorServiceCommand":{"services":[{"id":0,"service":"dhcp","processname":"dnsmasq","serviceName":"dnsmasq","servicePath":"/var/run/dnsmasq/dnsmasq.pid","pidFile":"/var/run/dnsmasq/dnsmasq.pid","isDefault":false},{"id":0,"service":"loadbalancing","processname":"haproxy","serviceName":"haproxy","servicePath":"/var/run/haproxy.pid","pidFile":"/var/run/haproxy.pid","isDefault":false},{"id":0,"service":"ssh","processname":"sshd","serviceName":"ssh","servicePath":"/var/run/sshd.pid","pidFile":"/var/run/sshd.pid","isDefault":true},{"id":0,"service":"webserver","processname":"apache2","serviceName":"apache2","servicePath":"/var/run/apache2.pid","pidFile":"/var/run/apache2.pid","isDefault":true}],"accessDetails":{"router.name":"r-4862-VM","router.ip":"169.254.3.4","router.guest.ip":"172.16.0.1"},"wait":0}},{"com.cloud.agent.api.routing.AggregationControlCommand":{"action":"Finish","accessDetails":{"router.guest.ip":"172.16.0.1","router.name":"r-4862-VM","router.ip":"169.254.3.4"},"wait":0}}] > } > "nics":[ > { > "deviceId":2, > "networkRateMbps":120, > "defaultNic":true, > "pxeDisable":true, > "nicUuid":"7838cbe1-32b6-4bec-89e4-2079a2e400c3", > "uuid":"d76510c2-208f-4f91-a01f-39865fac7372", > "ip":"217.71.92.215", > "netmask":"255.255.255.224", > "gateway":"217.71.92.193", > "mac":"06:29:76:00:02:45", > "dns1":"8.8.8.8", > "dns2":"8.8.4.4", > "broadcastType":"Vlan", > "type":"Public", > "broadcastUri":"vlan://2099", > "isolationUri":"vlan://2099", > "isSecurityGroupEnabled":false, > "name":"cloudbr0" > }, > { > "deviceId":0, > "networkRateMbps":120, > "defaultNic":false, > "pxeDisable":true, > "nicUuid":"8a517b63-d955-4800-9153-1fe0733a0e49", > "uuid":"3e7c7ae7-4f5e-4fd8-85dd-343c08ed39f3", > "ip":"172.16.0.1", > "netmask":"255.255.255.0", > "mac":"02:00:70:65:00:02", > "dns1":"8.8.8.8", > "dns2":"8.8.4.4", > "broadcastType":"Vlan", > "type":"Guest", > "broadcastUri":"vlan://3913", > "isolationUri":"vlan://3913", > "isSecurityGroupEnabled":false, > "name":"cloudbr0" > }, > { > "deviceId":1, > "networkRateMbps":-1, > "defaultNic":false, > "pxeDisable":true, > "nicUuid":"dc4d31cc-596c-41f7-ad87-a4d6e5372124", > "uuid":"9dab4d74-c260-4c55-9dce-2018ac5b1d6e", > "ip":"169.254.3.4", > "netmask":"255.255.0.0", > "gateway":"169.254.0.1", > "mac":"0e:00:a9:fe:03:04", > "broadcastType":"LinkLocal", > "type":"Control", > "isSecurityGroupEnabled":false > } > ] > }, > "hostIp":"10.1.0.232", > "executeInSequence":false, > "wait":0 > } > }, > { > "com.cloud.agent.api.routing.IpAssocCommand":{ > "ipAddresses":[ > { > "accountId":50, > "publicIp":"217.71.92.215", > "sourceNat":true, > "add":true, > "oneToOneNat":false, > "firstIP":true, > "broadcastUri":"vlan://2099", > "vlanGateway":"217.71.92.193", > "vlanNetmask":"255.255.255.224", > "vifMacAddress":"06:1f:96:00:02:45", > "networkRate":120, > "trafficType":"Public", > "networkName":"cloudbr0", > "newNic":false > } > ], > "accessDetails":{ > "zone.network.type":"Advanced", > "router.name":"r-4862-VM", > "router.ip":"169.254.3.4", > "router.guest.ip":"172.16.0.1" > }, > "wait":0 > } > }, > If we look at the agent.log on the KVM host, we see the line: > 2017-05-23 14:47:27,712 DEBUG [kvm.resource.OvsVifDriver] > (agentRequest-Handler-1:null) (logid:418adedf) plugging > nic=[Nic:Public-null-vlan://2099] > We suspect that the cloudstack-agent on KVM cannot find a matching nic with > "06:1f:96:00:02:45" and therefore add a new nic eth3. > root@ewcstack-vh023-test:~# less /var/log/cloudstack/agent/agent.log > 2017-05-23 14:46:51,432 DEBUG [cloud.agent.Agent] > (agentRequest-Handler-1:null) (logid:418adedf) Request:Seq > 249-4889501820440805400: { Cmd , MgmtId: 345049101620, via: 249, Ver: v1, > Flags: 100011, > [{"com.cloud.agent.api.StartCommand":{"vm":{"id":4862,"name":"r-4862-VM","type":"DomainRouter","cpus":2,"minSpeed":250,"maxSpeed":1000,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian > GNU/Linux 7(64-bit)","platformEmulator":"Debian GNU/Linux > 7(64-bit)","bootArgs":" template=domP > name=r-4862","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"4zIXt8gMEhpnJ4IpJXYMzA","params":{"memoryOvercommitRatio":"1.0","cpuOvercommitRatio":"4.0"},"uuid":"0b0fd71b-f0ed-46e4-8dd0-28583beb3937","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"fea09980-6a70-4003-9d51-e615dd898f1b","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"bca3ab3e-6bbe-3c92-b289-987be43a37b2","id":102,"poolType":"RBD","host":"mon.ceph","path":"cloudstack","port":6789,"url":"RBD://mon.ceph/cloudstack/?ROLE=Primary&STOREUUID=bca3ab3e-6bbe-3c92-b289-987be43a37b2"}},"name":"ROOT-4862","size":3145728000,"path":"fea09980-6a70-4003-9d51-e615dd898f1b","volumeId":11708,"vmName":"r-4862-VM","accountId":50,"format":"RAW","provisioningType":"THIN","id":11708,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"fea09980-6a70-4003-9d51-e615dd898f1b","type":"ROOT","_details":{"managed":"false","storagePort":"6789","storageHost":"mon.ceph","volumeSize":"3145728000"}}],"nics":[{"deviceId":2,"networkRateMbps":120,"defaultNic":true,"pxeDisable":true,"nicUuid":"7838cbe1-32b6-4bec-89e4-2079a2e400c3","uuid":"d76510c2-208f-4f91-a01f-39865fac7372","ip":"217.71.92.215","netmask":"255.255.255.224","gateway":"217.71.92.193","mac":"06:29:76:00:02:45","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Public","broadcastUri":"vlan://2099","isolationUri":"vlan://2099","isSecurityGroupEnabled":false,"name":"cloudbr0"},{"deviceId":0,"networkRateMbps":120,"defaultNic":false,"pxeDisable":true,"nicUuid":"8a517b63-d955-4800-9153-1fe0733a0e49","uuid":"3e7c7ae7-4f5e-4fd8-85dd-343c08ed39f3","ip":"172.16.0.1","netmask":"255.255.255.0","mac":"02:00:70:65:00:02","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://3913","isolationUri":"vlan://3913","isSecurityGroupEnabled":false,"name":"cloudbr0"},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"dc4d31cc-596c-41f7-ad87-a4d6e5372124","uuid":"9dab4d74-c260-4c55-9dce-2018ac5b1d6e","ip":"169.254.3.4","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:03:04","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false}]},"hostIp":"10.1.0.232","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.3.4","port":3922,"interval":6,"retries":100,"name":"r-4862-VM","wait":0}},{"com.cloud.agent.api.GetDomRVersionCmd":{"accessDetails":{"router.ip":"169.254.3.4","router.name":"r-4862-VM"},"wait":0}},{},{"com.cloud.agent.api.routing.AggregationControlCommand":{"action":"Start","accessDetails":{"router.guest.ip":"172.16.0.1","router.ip":"169.254.3.4","router.name":"r-4862-VM"},"wait":0}},{"com.cloud.agent.api.routing.IpAssocCommand":{"ipAddresses":[{"accountId":50,"publicIp":"217.71.92.215","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"broadcastUri":"vlan://2099","vlanGateway":"217.71.92.193","vlanNetmask":"255.255.255.224","vifMacAddress":"06:1f:96:00:02:45","networkRate":120,"trafficType":"Public","networkName":"cloudbr0","newNic":false}],"accessDetails":{"router.guest.ip":"172.16.0.1","zone.network.type":"Advanced","router.ip":"169.254.3.4","router.name":"r-4862-VM"},"wait":0}},{"com.cloud.agent.api.routing.SetMonitorServiceCommand":{"services":[{"id":0,"service":"dhcp","processname":"dnsmasq","serviceName":"dnsmasq","servicePath":"/var/run/dnsmasq/dnsmasq.pid","pidFile":"/var/run/dnsmasq/dnsmasq.pid","isDefault":false},{"id":0,"service":"loadbalancing","processname":"haproxy","serviceName":"haproxy","servicePath":"/var/run/haproxy.pid","pidFile":"/var/run/haproxy.pid","isDefault":false},{"id":0,"service":"ssh","processname":"sshd","serviceName":"ssh","servicePath":"/var/run/sshd.pid","pidFile":"/var/run/sshd.pid","isDefault":true},{"id":0,"service":"webserver","processname":"apache2","serviceName":"apache2","servicePath":"/var/run/apache2.pid","pidFile":"/var/run/apache2.pid","isDefault":true}],"accessDetails":{"router.guest.ip":"172.16.0.1","router.ip":"169.254.3.4","router.name":"r-4862-VM"},"wait":0}},{"com.cloud.agent.api.routing.AggregationControlCommand":{"action":"Finish","accessDetails":{"router.guest.ip":"172.16.0.1","router.ip":"169.254.3.4","router.name":"r-4862-VM"},"wait":0}}] > } > 2017-05-23 14:46:51,432 DEBUG [cloud.agent.Agent] > (agentRequest-Handler-1:null) (logid:418adedf) Processing command: > com.cloud.agent.api.StartCommand > 2017-05-23 14:46:51,444 DEBUG [kvm.resource.LibvirtConnection] > (agentRequest-Handler-1:null) (logid:418adedf) Looking for libvirtd > connection at: qemu:///system > 2017-05-23 14:46:51,445 INFO [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-1:null) (logid:418adedf) Trying to fetch storage pool > bca3ab3e-6bbe-3c92-b289-987be43a37b2 from libvirt > 2017-05-23 14:46:51,445 DEBUG [kvm.resource.LibvirtConnection] > (agentRequest-Handler-1:null) (logid:418adedf) Looking for libvirtd > connection at: qemu:///system > 2017-05-23 14:46:51,448 DEBUG [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-1:null) (logid:418adedf) Succesfully refreshed pool > bca3ab3e-6bbe-3c92-b289-987be43a37b2 Capacity: 20605776691200 Used: > 3247413177954 Available: 10646356393984 > 2017-05-23 14:46:51,449 DEBUG [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-1:null) (logid:418adedf) Could not find volume > fea09980-6a70-4003-9d51-e615dd898f1b: Storage volume not found: no storage > vol with matching name 'fea09980-6a70-4003-9d51-e615dd898f1b' > 2017-05-23 14:46:51,449 DEBUG [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-1:null) (logid:418adedf) Refreshing storage pool > bca3ab3e-6bbe-3c92-b289-987be43a37b2 > 2017-05-23 14:46:52,080 DEBUG [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-1:null) (logid:418adedf) Found volume > fea09980-6a70-4003-9d51-e615dd898f1b in storage pool > bca3ab3e-6bbe-3c92-b289-987be43a37b2 after refreshing the pool > 2017-05-23 14:46:52,142 INFO [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-1:null) (logid:418adedf) Trying to fetch storage pool > bca3ab3e-6bbe-3c92-b289-987be43a37b2 from libvirt > 2017-05-23 14:46:52,142 DEBUG [kvm.resource.LibvirtConnection] > (agentRequest-Handler-1:null) (logid:418adedf) Looking for libvirtd > connection at: qemu:///system > 2017-05-23 14:46:52,145 DEBUG [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-1:null) (logid:418adedf) Succesfully refreshed pool > bca3ab3e-6bbe-3c92-b289-987be43a37b2 Capacity: 20605776691200 Used: > 3247413177954 Available: 10646356799488 > 2017-05-23 14:46:52,145 DEBUG [kvm.resource.OvsVifDriver] > (agentRequest-Handler-1:null) (logid:418adedf) plugging > nic=[Nic:Guest-172.16.0.1-vlan://3913] > 2017-05-23 14:46:52,145 DEBUG [kvm.resource.OvsVifDriver] > (agentRequest-Handler-1:null) (logid:418adedf) creating a vlan dev and bridge > for guest traffic per traffic label cloudbr0 > 2017-05-23 14:46:52,145 DEBUG [kvm.resource.OvsVifDriver] > (agentRequest-Handler-1:null) (logid:418adedf) plugging > nic=[Nic:Control-169.254.3.4-null] > 2017-05-23 14:46:52,145 DEBUG [utils.script.Script] > (agentRequest-Handler-1:null) (logid:418adedf) Executing: /bin/bash -c ip > route | grep 169.254.0.0/16 > 2017-05-23 14:46:52,151 DEBUG [utils.script.Script] > (agentRequest-Handler-1:null) (logid:418adedf) Execution is successful. > 2017-05-23 14:46:52,152 DEBUG [utils.script.Script] > (agentRequest-Handler-1:null) (logid:418adedf) Executing: /bin/sh -c > ovs-vsctl br-exists cloud0 > 2017-05-23 14:46:52,159 DEBUG [utils.script.Script] > (agentRequest-Handler-1:null) (logid:418adedf) Execution is successful. > 2017-05-23 14:46:52,159 DEBUG [kvm.resource.OvsVifDriver] > (agentRequest-Handler-1:null) (logid:418adedf) plugging > nic=[Nic:Public-217.71.92.215-vlan://2099] > 2017-05-23 14:46:52,159 DEBUG [kvm.resource.OvsVifDriver] > (agentRequest-Handler-1:null) (logid:418adedf) creating a vlan dev and bridge > for public traffic per traffic label cloudbr0 > 2017-05-23 14:46:52,160 DEBUG > [resource.wrapper.LibvirtStartCommandWrapper] (agentRequest-Handler-1:null) > (logid:418adedf) starting r-4862-VM: <domain type='kvm'> > <name>r-4862-VM</name> > <uuid>0b0fd71b-f0ed-46e4-8dd0-28583beb3937</uuid> > <description>Debian GNU/Linux 7(64-bit)</description> > <cpu></cpu><sysinfo type='smbios'> > <system> > <entry name='manufacturer'>Apache Software Foundation</entry> > <entry name='product'>CloudStack KVM Hypervisor</entry> > <entry name='uuid'>0b0fd71b-f0ed-46e4-8dd0-28583beb3937</entry> > </system> > </sysinfo> > <os> > <type arch='x86_64' machine='pc'>hvm</type> > <boot dev='cdrom'/> > <boot dev='hd'/> > <smbios mode='sysinfo'/> > </os> > <on_reboot>restart</on_reboot> > <on_poweroff>destroy</on_poweroff> > <on_crash>destroy</on_crash> > <memory>524288</memory> > <devices> > <memballoon model='none'/> > </devices> > <vcpu>2</vcpu> > <features> > <pae/> > <apic/> > <acpi/> > </features> > <cputune> > <shares>500</shares> > </cputune> > <clock offset='utc'> > <timer name='kvmclock' > > </timer> > </clock> > <devices> > <emulator>/usr/bin/kvm-spice</emulator> > <interface type='bridge'> > <source bridge='cloudbr0'/> > <mac address='02:00:70:65:00:02'/> > <model type='virtio'/> > <bandwidth> > <inbound average='15360' peak='15360'/> > <outbound average='15360' peak='15360'/> > </bandwidth> > <virtualport type='openvswitch'> > </virtualport> > <vlan trunk='no'> > <tag id='3913'/> > </vlan></interface> > <interface type='bridge'> > <source bridge='cloud0'/> > <mac address='0e:00:a9:fe:03:04'/> > <model type='virtio'/> > <virtualport type='openvswitch'> > </virtualport> > </interface> > <interface type='bridge'> > <source bridge='cloudbr0'/> > <mac address='06:29:76:00:02:45'/> > <model type='virtio'/> > <bandwidth> > <inbound average='15360' peak='15360'/> > <outbound average='15360' peak='15360'/> > </bandwidth> > <virtualport type='openvswitch'> > </virtualport> > <vlan trunk='no'> > <tag id='2099'/> > </vlan></interface> > <console type='pty'> > <target port='0'/> > </console> > <disk device='disk' type='network'> > <driver name='qemu' type='raw' cache='none' /> > <source protocol='rbd' > name='cloudstack/fea09980-6a70-4003-9d51-e615dd898f1b'> > <host name='mon.ceph' port='6789'/> > </source> > <auth username='cloudstack'> > <secret type='ceph' uuid='bca3ab3e-6bbe-3c92-b289-987be43a37b2'/> > </auth> > <target dev='vda' bus='virtio'/> > <serial>fea099806a7040039d51</serial></disk> > <disk device='cdrom' type='file'> > <driver name='qemu' type='raw' cache='none' /> > <source file='/usr/share/cloudstack-common/vms/systemvm.iso'/> > <target dev='hdc' bus='ide'/> > </disk> > <channel type='unix'> > <source mode='bind' path='/var/lib/libvirt/qemu/r-4862-VM.agent'/> > <target type='virtio' name='r-4862-VM.vport'/> > <address type='virtio-serial'/> > </channel> > <serial type='pty'> > <target port='0'/> > </serial> > <graphics type='vnc' autoport='yes' listen='10.1.0.232' > passwd='4zIXt8gMEhpnJ4IpJXYMzA'/> > <input type='tablet' bus='usb'/> > </devices> > </domain> > 2017-05-23 14:46:53,240 DEBUG [kvm.resource.LibvirtComputingResource] > (agentRequest-Handler-1:null) (logid:418adedf) Executing: > /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/patchviasocket.py -n > r-4862-VM -p > %template=domP%name=r-4862-VM%vmpassword=h9bb3J4p%eth2ip=217.71.92.215%eth2mask=255.255.255.224%gateway=217.71.92.193%eth0ip=172.16.0.1%eth0mask=255.255.255.0%domain=cs32cloud.internal%cidrsize=24%dhcprange=172.16.0.1%eth1ip=169.254.3.4%eth1mask=255.255.0.0%type=router%disable_rp_filter=true%dns1=8.8.8.8%dns2=8.8.4.4 > > 2017-05-23 14:46:53,266 DEBUG [kvm.resource.LibvirtComputingResource] > (agentRequest-Handler-1:null) (logid:418adedf) Execution is successful. > 2017-05-23 14:46:53,266 DEBUG > [resource.virtualnetwork.VirtualRoutingResource] > (agentRequest-Handler-1:null) (logid:418adedf) Trying to connect to > 169.254.3.4 > 2017-05-23 14:46:56,339 DEBUG > [resource.virtualnetwork.VirtualRoutingResource] > (agentRequest-Handler-1:null) (logid:418adedf) Could not connect to > 169.254.3.4 > 2017-05-23 14:46:58,547 DEBUG [cloud.agent.Agent] > (agentRequest-Handler-2:null) (logid:e4e75b6b) Processing command: > com.cloud.agent.api.GetHostStatsCommand > 2017-05-23 14:47:00,419 DEBUG [cloud.agent.Agent] > (agentRequest-Handler-3:null) (logid:0b373a17) Processing command: > com.cloud.agent.api.GetVmStatsCommand > 2017-05-23 14:47:00,419 DEBUG [kvm.resource.LibvirtConnection] > (agentRequest-Handler-3:null) (logid:0b373a17) Looking for libvirtd > connection at: qemu:///system > 2017-05-23 14:47:00,426 WARN > [resource.wrapper.LibvirtGetVmStatsCommandWrapper] > (agentRequest-Handler-3:null) (logid:0b373a17) Can't get vm stats: > org.libvirt.LibvirtException: internal error: cannot find statistics for > device 'ide0-1-0', continue > 2017-05-23 14:47:01,339 DEBUG > [resource.virtualnetwork.VirtualRoutingResource] > (agentRequest-Handler-1:null) (logid:418adedf) Trying to connect to > 169.254.3.4 > 2017-05-23 14:47:04,403 DEBUG > [resource.virtualnetwork.VirtualRoutingResource] > (agentRequest-Handler-1:null) (logid:418adedf) Could not connect to > 169.254.3.4 > 2017-05-23 14:47:09,403 DEBUG > [resource.virtualnetwork.VirtualRoutingResource] > (agentRequest-Handler-1:null) (logid:418adedf) Unable to logon to 169.254.3.4 > 2017-05-23 14:47:09,403 DEBUG [kvm.resource.LibvirtComputingResource] > (agentRequest-Handler-1:null) (logid:418adedf) Executing: > /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/patchviasocket.py -n > r-4862-VM -p > %template=domP%name=r-4862-VM%vmpassword=h9bb3J4p%eth2ip=217.71.92.215%eth2mask=255.255.255.224%gateway=217.71.92.193%eth0ip=172.16.0.1%eth0mask=255.255.255.0%domain=cs32cloud.internal%cidrsize=24%dhcprange=172.16.0.1%eth1ip=169.254.3.4%eth1mask=255.255.0.0%type=router%disable_rp_filter=true%dns1=8.8.8.8%dns2=8.8.4.4 > > 2017-05-23 14:47:09,430 DEBUG [kvm.resource.LibvirtComputingResource] > (agentRequest-Handler-1:null) (logid:418adedf) Execution is successful. > 2017-05-23 14:47:09,431 DEBUG > [resource.virtualnetwork.VirtualRoutingResource] > (agentRequest-Handler-1:null) (logid:418adedf) Trying to connect to > 169.254.3.4 > 2017-05-23 14:47:09,431 DEBUG > [resource.virtualnetwork.VirtualRoutingResource] > (agentRequest-Handler-1:null) (logid:418adedf) Could not connect to > 169.254.3.4 > 2017-05-23 14:47:09,850 DEBUG [cloud.agent.Agent] > (agentRequest-Handler-4:null) (logid:02de0e65) Processing command: > com.cloud.agent.api.GetStorageStatsCommand > 2017-05-23 14:47:09,850 INFO [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-4:null) (logid:02de0e65) Trying to fetch storage pool > 18f0e010-2d14-45e4-ab09-3cfd0796abeb from libvirt > 2017-05-23 14:47:09,850 DEBUG [kvm.resource.LibvirtConnection] > (agentRequest-Handler-4:null) (logid:02de0e65) Looking for libvirtd > connection at: qemu:///system > 2017-05-23 14:47:09,852 INFO [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-4:null) (logid:02de0e65) Asking libvirt to refresh > storage pool 18f0e010-2d14-45e4-ab09-3cfd0796abeb > 2017-05-23 14:47:09,854 DEBUG [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-4:null) (logid:02de0e65) Succesfully refreshed pool > 18f0e010-2d14-45e4-ab09-3cfd0796abeb Capacity: 20698107904 Used: 9596616704 > Available: 11101491200 > 2017-05-23 14:47:09,898 DEBUG [cloud.agent.Agent] > (agentRequest-Handler-5:null) (logid:02de0e65) Processing command: > com.cloud.agent.api.GetStorageStatsCommand > 2017-05-23 14:47:09,898 INFO [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-5:null) (logid:02de0e65) Trying to fetch storage pool > bca3ab3e-6bbe-3c92-b289-987be43a37b2 from libvirt > 2017-05-23 14:47:09,898 DEBUG [kvm.resource.LibvirtConnection] > (agentRequest-Handler-5:null) (logid:02de0e65) Looking for libvirtd > connection at: qemu:///system > 2017-05-23 14:47:09,900 INFO [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-5:null) (logid:02de0e65) Asking libvirt to refresh > storage pool bca3ab3e-6bbe-3c92-b289-987be43a37b2 > 2017-05-23 14:47:10,503 DEBUG [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-5:null) (logid:02de0e65) Succesfully refreshed pool > bca3ab3e-6bbe-3c92-b289-987be43a37b2 Capacity: 20605776691200 Used: > 3247572561523 Available: 10645898035200 > 2017-05-23 14:47:14,431 DEBUG > [resource.virtualnetwork.VirtualRoutingResource] > (agentRequest-Handler-1:null) (logid:418adedf) Trying to connect to > 169.254.3.4 > 2017-05-23 14:47:17,459 DEBUG > [resource.virtualnetwork.VirtualRoutingResource] > (agentRequest-Handler-1:null) (logid:418adedf) Could not connect to > 169.254.3.4 > 2017-05-23 14:47:22,459 DEBUG > [resource.virtualnetwork.VirtualRoutingResource] > (agentRequest-Handler-1:null) (logid:418adedf) Unable to logon to 169.254.3.4 > 2017-05-23 14:47:22,459 DEBUG [kvm.resource.LibvirtComputingResource] > (agentRequest-Handler-1:null) (logid:418adedf) Executing: > /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/patchviasocket.py -n > r-4862-VM -p > %template=domP%name=r-4862-VM%vmpassword=h9bb3J4p%eth2ip=217.71.92.215%eth2mask=255.255.255.224%gateway=217.71.92.193%eth0ip=172.16.0.1%eth0mask=255.255.255.0%domain=cs32cloud.internal%cidrsize=24%dhcprange=172.16.0.1%eth1ip=169.254.3.4%eth1mask=255.255.0.0%type=router%disable_rp_filter=true%dns1=8.8.8.8%dns2=8.8.4.4 > > 2017-05-23 14:47:22,486 DEBUG [kvm.resource.LibvirtComputingResource] > (agentRequest-Handler-1:null) (logid:418adedf) Execution is successful. > 2017-05-23 14:47:22,487 DEBUG > [resource.virtualnetwork.VirtualRoutingResource] > (agentRequest-Handler-1:null) (logid:418adedf) Trying to connect to > 169.254.3.4 > 2017-05-23 14:47:22,488 DEBUG > [resource.virtualnetwork.VirtualRoutingResource] > (agentRequest-Handler-1:null) (logid:418adedf) Could not connect to > 169.254.3.4 > 2017-05-23 14:47:27,488 DEBUG > [resource.virtualnetwork.VirtualRoutingResource] > (agentRequest-Handler-1:null) (logid:418adedf) Trying to connect to > 169.254.3.4 > 2017-05-23 14:47:27,488 DEBUG [cloud.agent.Agent] > (agentRequest-Handler-1:null) (logid:418adedf) Processing command: > com.cloud.agent.api.check.CheckSshCommand > 2017-05-23 14:47:27,490 DEBUG > [resource.wrapper.LibvirtOvsVpcRoutingPolicyConfigCommandWrapper] > (agentRequest-Handler-1:null) (logid:418adedf) Ping command port, > 169.254.3.4:3922 > 2017-05-23 14:47:27,490 DEBUG > [resource.virtualnetwork.VirtualRoutingResource] > (agentRequest-Handler-1:null) (logid:418adedf) Trying to connect to > 169.254.3.4 > 2017-05-23 14:47:27,491 DEBUG > [resource.wrapper.LibvirtOvsVpcRoutingPolicyConfigCommandWrapper] > (agentRequest-Handler-1:null) (logid:418adedf) Ping command port succeeded > for vm r-4862-VM > 2017-05-23 14:47:27,491 DEBUG [cloud.agent.Agent] > (agentRequest-Handler-1:null) (logid:418adedf) Processing command: > com.cloud.agent.api.GetDomRVersionCmd > 2017-05-23 14:47:27,492 DEBUG [kvm.resource.LibvirtComputingResource] > (agentRequest-Handler-1:null) (logid:418adedf) Executing: > /usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh > get_template_version.sh 169.254.3.4 > 2017-05-23 14:47:27,590 DEBUG [kvm.resource.LibvirtComputingResource] > (agentRequest-Handler-1:null) (logid:418adedf) Execution is successful. > 2017-05-23 14:47:27,591 DEBUG [cloud.agent.Agent] > (agentRequest-Handler-1:null) (logid:418adedf) Processing command: > com.cloud.agent.api.NetworkUsageCommand > 2017-05-23 14:47:27,591 DEBUG [kvm.resource.LibvirtComputingResource] > (agentRequest-Handler-1:null) (logid:418adedf) Executing: > /usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh netusage.sh > 169.254.3.4 -c > 2017-05-23 14:47:27,708 DEBUG [kvm.resource.LibvirtComputingResource] > (agentRequest-Handler-1:null) (logid:418adedf) Execution is successful. > 2017-05-23 14:47:27,709 DEBUG [cloud.agent.Agent] > (agentRequest-Handler-1:null) (logid:418adedf) Processing command: > com.cloud.agent.api.routing.AggregationControlCommand > 2017-05-23 14:47:27,709 DEBUG [cloud.agent.Agent] > (agentRequest-Handler-1:null) (logid:418adedf) Processing command: > com.cloud.agent.api.routing.IpAssocCommand > 2017-05-23 14:47:27,709 DEBUG [kvm.resource.LibvirtConnection] > (agentRequest-Handler-1:null) (logid:418adedf) Looking for libvirtd > connection at: qemu:///system > 2017-05-23 14:47:27,712 DEBUG [kvm.resource.OvsVifDriver] > (agentRequest-Handler-1:null) (logid:418adedf) plugging > nic=[Nic:Public-null-vlan://2099] > Thank you in advance for your help. > Best Regards > Francois Scheurer -- This message was sent by Atlassian JIRA (v7.6.3#76005)