Hi Li, Can you try your tests with this PR: https://github.com/apache/cloudstack/pull/4040 This one ^^ is enabling it on L2 networks, to be honest I'm not sure about L3 as I'm currently testing it.
But to be able to test I'll need to facilitate an L3 network (arping requires IP), and then specify the interface to the other L2 network attached to the same VM. Please note that this is 4.15+ and it's currently under testing, so do it in a testing lab. Thanks, On 26.05.20, 18:14, "li jerry" <[email protected]> wrote: Dear All Who has successfully deployed a PVLAN network on ACS 4.13.1? I created the PVLAN network through the following process, but it failed Server: CentOS7 CloudStack 4.13.1 Zone 1： l Physical Network1 ：Public and Guest , Isolation method=VLAN, KVM traffic label=br1 l Physical Network2 : Management , Isolation method=VLAN, KVM traffic label=br2 SystemVM Template (KVM): Version 4.11.3 Hypervisor: CentOS7.7 3.10.0-1062 Openvswitch-2.12.0 libvirt 4.5.0 QEMU 1.5.3 1. Create L3 Guest Network, name= Pvlan-Net-01, vlan=700, isolatedpvlan=1700 ( this is api command) command=createNetwork&zoneId=409b04ea-d128-48ac-8e33-4df700da89cc&networkOfferingId=9778a4ab-0de8-4440-9879-a488416e0572&physicalnetworkid=ca0768c8-f068-4d88-b7bd-2766414a6415&name=Pvlan-Net-01&displayText=Pvlan-Net-01&vlan=700&bypassVlanOverlapCheck=false&isolatedpvlan=1700&acltype=domain&gateway=172.17.0.1&netmask=255.255.0.0&startip=172.17.1.1&endip=172.17.1.254&networkdomain=hyperx.com&response=json&_=1590497900407 2. attache network to VM vm1 (this VM is running and has an isolated network 10.0.0.x / 24) 3. attache failed, management throws the following error: this is management log 2020-05-26 21:59:40,268 DEBUG [c.c.a.t.Request] (Work-Job-Executor-1:ctx-905dd7d7 job-44/job-45 ctx-4c296832) (logid:2009fc05) Seq 4-6374282322589515787: Received: { Ans: , MgmtId: 113349236140, via: 4(2222), Ver: v1, Flags: 10, { StartAnswer, CheckSshAnswer, GetDomRVersionAnswer, NetworkUsageAnswer, Answer, Answer, Answer, Answer, Answer } } 2020-05-26 21:59:40,280 DEBUG [o.a.c.n.t.AdvancedNetworkTopology] (Work-Job-Executor-1:ctx-905dd7d7 job-44/job-45 ctx-4c296832) (logid:2009fc05) SETUP DHCP PVLAN RULES 2020-05-26 21:59:40,292 DEBUG [c.c.n.r.NetworkHelperImpl] (Work-Job-Executor-1:ctx-905dd7d7 job-44/job-45 ctx-4c296832) (logid:2009fc05) Router requires upgrade. Unable to send command to router:5, router template version : null, minimal required version : 4.10.0 2020-05-26 21:59:40,294 WARN [o.a.c.n.t.AdvancedNetworkVisitor] (Work-Job-Executor-1:ctx-905dd7d7 job-44/job-45 ctx-4c296832) (logid:2009fc05) Timed Out com.cloud.exception.ResourceUnavailableException: Resource [VirtualRouter:5] is unreachable: Unable to send command. Router requires upgrade at com.cloud.network.router.NetworkHelperImpl.sendCommandsToRouter(NetworkHelperImpl.java:175) at org.apache.cloudstack.network.topology.AdvancedNetworkVisitor.visit(AdvancedNetworkVisitor.java:185) at com.cloud.network.rules.DhcpPvlanRules.accept(DhcpPvlanRules.java:61) at org.apache.cloudstack.network.topology.AdvancedNetworkTopology.setupDhcpForPvlan(AdvancedNetworkTopology.java:131) at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeStart(VirtualNetworkApplianceManagerImpl.java:2080) at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1170) at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:904) at com.cloud.network.router.NetworkHelperImpl.start(NetworkHelperImpl.java:277) at com.cloud.network.router.NetworkHelperImpl.startVirtualRouter(NetworkHelperImpl.java:356) at com.cloud.network.router.NetworkHelperImpl.startRouters(NetworkHelperImpl.java:341) at org.cloud.network.router.deployment.RouterDeploymentDefinition.deployVirtualRouter(RouterDeploymentDefinition.java:205) at com.cloud.network.element.VirtualRouterElement.prepare(VirtualRouterElement.java:278) at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepareElement(NetworkOrchestrator.java:1380) at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepareNic(NetworkOrchestrator.java:1715) at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.createNicForVm(NetworkOrchestrator.java:3767) at com.cloud.vm.VirtualMachineManagerImpl.orchestrateAddVmToNetwork(VirtualMachineManagerImpl.java:3504) at com.cloud.vm.VirtualMachineManagerImpl.orchestrateAddVmToNetwork(VirtualMachineManagerImpl.java:5264) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107) at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5326) at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:603) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:551) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2020-05-26 21:59:40,297 INFO [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-1:ctx-905dd7d7 job-44/job-45 ctx-4c296832) (logid:2009fc05) The guru did not like the answers so stopping VM[DomainRouter|r-5-VM] 2020-05-26 21:59:40,302 DEBUG [c.c.a.t.Request] (Work-Job-Executor-1:ctx-905dd7d7 job-44/job-45 ctx-4c296832) (logid:2009fc05) Seq 4-6374282322589515789: Sending { Cmd , MgmtId: 113349236140, via: 4(2222), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"checkBeforeCleanup":false,"controlIp":"169.254.188.7","forceStop":false,"volumesToDisconnect":[],"vmName":"r-5-VM","executeInSequence":false,"wait":0}}] } Agent error 20-05-26 22:53:44,801 DEBUG [kvm.resource.LibvirtComputingResource] (UgentTask-2:null) (logid:) Execution is successful. 2020-05-26 22:53:44,802 DEBUG [kvm.resource.LibvirtConnection] (UgentTask-2:null) (logid:) Looking for libvirtd connection at: qemu:///system 2020-05-26 22:53:44,807 DEBUG [cloud.agent.Agent] (UgentTask-2:null) (logid:) Sending ping: Seq 4-4: { Cmd , MgmtId: -1, via: 4, Ver: v1, Flags: 11, [{"com.cloud.agent.api.PingRoutingWithNwGroupsCommand":{"newGroupStates":{},"_hostVmStateReport":{"r-11-VM":{"state":"PowerOn","host":"2222"},"r-4-VM":{"state":"PowerOn","host":"2222"}},"_gatewayAccessible":true,"_vnetAccessible":true,"hostType":"Routing","hostId":4,"wait":0}}] } 2020-05-26 22:53:44,903 DEBUG [cloud.agent.Agent] (Agent-Handler-1:null) (logid:) Received response: Seq 4-4: { Ans: , MgmtId: 113349236140, via: 4, Ver: v1, Flags: 100010, [{"com.cloud.agent.api.PingAnswer":{"_command":{"hostType":"Routing","hostId":4,"wait":0},"result":true,"wait":0}}] } 2020-05-26 22:53:48,475 DEBUG [resource.virtualnetwork.VirtualRoutingResource] (agentRequest-Handler-3:null) (logid:03678ec5) Trying to connect to 169.254.208.61 2020-05-26 22:53:48,477 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null) (logid:03678ec5) Processing command: com.cloud.agent.api.check.CheckSshCommand 2020-05-26 22:53:48,477 DEBUG [resource.wrapper.LibvirtOvsVpcRoutingPolicyConfigCommandWrapper] (agentRequest-Handler-3:null) (logid:03678ec5) Ping command port, 169.254.208.61:3922 2020-05-26 22:53:48,477 DEBUG [resource.virtualnetwork.VirtualRoutingResource] (agentRequest-Handler-3:null) (logid:03678ec5) Trying to connect to 169.254.208.61 2020-05-26 22:53:48,477 DEBUG [resource.wrapper.LibvirtOvsVpcRoutingPolicyConfigCommandWrapper] (agentRequest-Handler-3:null) (logid:03678ec5) Ping command port succeeded for vm r-11-VM 2020-05-26 22:53:48,477 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null) (logid:03678ec5) Processing command: com.cloud.agent.api.GetDomRVersionCmd 2020-05-26 22:53:48,480 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:03678ec5) Executing: /usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh get_template_version.sh 169.254.208.61 2020-05-26 22:53:48,482 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:03678ec5) Executing while with timeout : 1800000 2020-05-26 22:53:48,986 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:03678ec5) Execution is successful. 2020-05-26 22:53:48,987 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:03678ec5) Executing script in VR: get_template_version.sh 2020-05-26 22:53:48,988 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null) (logid:03678ec5) Processing command: com.cloud.agent.api.NetworkUsageCommand 2020-05-26 22:53:48,988 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:03678ec5) Executing: /usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh netusage.sh 169.254.208.61 -c 2020-05-26 22:53:48,989 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:03678ec5) Executing while with timeout : 3600000 2020-05-26 22:53:49,571 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:03678ec5) Execution is successful. 2020-05-26 22:53:49,572 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null) (logid:03678ec5) Processing command: com.cloud.agent.api.routing.AggregationControlCommand 2020-05-26 22:53:49,572 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null) (logid:03678ec5) Processing command: com.cloud.agent.api.routing.SetMonitorServiceCommand 2020-05-26 22:53:49,573 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null) (logid:03678ec5) Processing command: com.cloud.agent.api.routing.AggregationControlCommand 2020-05-26 22:53:49,573 DEBUG [resource.virtualnetwork.VirtualRoutingResource] (agentRequest-Handler-3:null) (logid:03678ec5) Transforming com.cloud.agent.api.routing.SetMonitorServiceCommand to ConfigItems 2020-05-26 22:53:49,602 DEBUG [virtualnetwork.facade.AbstractConfigItemFacade] (agentRequest-Handler-3:null) (logid:03678ec5) Transformed filename: monitor_service.json to: monitor_service.json.e2aeaa96-5a74-4753-8edf-6a040717a8aa 2020-05-26 22:53:49,604 DEBUG [resource.virtualnetwork.VirtualRoutingResource] (agentRequest-Handler-3:null) (logid:03678ec5) Aggregate action timeout in seconds is 600 2020-05-26 22:53:49,605 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:03678ec5) Creating file in VR, with ip: 169.254.208.61, file: VR-1e88cb1c-f82d-4994-9a80-b8c5fc22fd9e.cfg 2020-05-26 22:53:50,236 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:03678ec5) Executing: /usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh vr_cfg.sh 169.254.208.61 -c /var/cache/cloud/VR-1e88cb1c-f82d-4994-9a80-b8c5fc22fd9e.cfg 2020-05-26 22:53:50,239 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:03678ec5) Executing while with timeout : 600600 2020-05-26 22:53:52,488 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:03678ec5) Execution is successful. 2020-05-26 22:53:52,488 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:03678ec5) Executing script in VR: vr_cfg.sh Note： 1. I used the same template to create the Isolated network and attache to the VM, everything works fine. 2. When I created the PVLAN VR, I used the virsh console to enter the VM and executed the get_template_version.sh script, which output cloudstack 4.11.3 I do n’t know what causes the PVLAN network to be created! Any help would be greatly appreciated! If you need more detailed information, please let me know Thank you -Jerry [email protected] www.shapeblue.com 3 London Bridge Street, 3rd floor, News Building, London SE1 9SGUK @shapeblue
