Hi Wido, I will test the patch and respond today. Thanks Özhan
On Mon, Jan 22, 2018 at 10:08 AM, Wido den Hollander <w...@widodh.nl> wrote: > > > On 01/22/2018 07:35 AM, Wido den Hollander wrote: > >> >> >> On 01/21/2018 11:23 AM, Rohit Yadav wrote: >> >>> Wido - Were you able to reproduce and fix the issue? Thanks. >>> >>> >> Still working on it! This weekend I was short on time and wasn't able to >> fix it yet. >> >> Today (Mon) and tomorrow (Tue) my time is limited as well. Trying to fix >> it asap. >> > > During my train ride this morning I wrote this patch: > https://github.com/apache/cloudstack/pull/2418 > > @ Özhan, could you test this patch? It's just a matter of replacing > security_group.py on your Hypervisor. > > Thanks, > > Wido > > > >> Wido >> >> >>> >>> - Rohit >>> >>> <https://cloudstack.apache.org> >>> >>> >>> >>> ________________________________ >>> From: Wido den Hollander <w...@widodh.nl> >>> Sent: Friday, January 19, 2018 10:12:45 PM >>> To: dev@cloudstack.apache.org >>> Subject: Re: [4.11] KVM Advanced Networking with SG Problem >>> >>> >>> >>> On 01/19/2018 02:03 PM, Özhan Rüzgar Karaman wrote: >>> >>>> Hi Daan; >>>> Wido or others will write a fix, i am not a developer, i do not have a >>>> fix, >>>> i just only want to report it to make it official thats all :) >>>> >>>> >>> I'll look into this asap. The Python script should parse these rules >>> properly and then it should be fixed. >>> >>> I hope to have a fix this weekend. >>> >>> Wido >>> >>> Thanks >>>> Özhan >>>> >>>> On Fri, Jan 19, 2018 at 3:59 PM, Daan Hoogland <daan.hoogl...@gmail.com >>>> > >>>> wrote: >>>> >>>> This is not a PR but a ticket, Özhan. Do you plan to make a pull >>>>> request on >>>>> github with your solution for it? >>>>> >>>>> On Fri, Jan 19, 2018 at 1:53 PM, Özhan Rüzgar Karaman < >>>>> oruzgarkara...@gmail.com> wrote: >>>>> >>>>> Hi Daan; >>>>>> Wido is the previous PR's owner, he will check it. By the way i have >>>>>> created a PR for this problem which is below: >>>>>> >>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-10242 >>>>>> >>>>>> I select its priority as blocker, if its wrong developers will update >>>>>> its >>>>>> priority. >>>>>> >>>>>> Thanks >>>>>> Özhan >>>>>> >>>>>> >>>>>> >>>>>> On Fri, Jan 19, 2018 at 3:25 PM, Daan Hoogland < >>>>>> daan.hoogl...@gmail.com> >>>>>> wrote: >>>>>> >>>>>> Özhan, this is sure to break ipv6. can you make it use another >>>>>>> >>>>>> delimiter? >>>>> >>>>>> >>>>>>> On Fri, Jan 19, 2018 at 1:12 PM, Özhan Rüzgar Karaman < >>>>>>> oruzgarkara...@gmail.com> wrote: >>>>>>> >>>>>>> Hi Rohit; >>>>>>>> This is a fresh install of 4.11 rc1 and we have only ipv4 setup on >>>>>>>> >>>>>>> our >>>>> >>>>>> test >>>>>>> >>>>>>>> environment no ipv6 addresses, our VR's are new 4.11 rc1 system vms. >>>>>>>> >>>>>>> Our >>>>>> >>>>>>> workaround is 4 lines of code to convert ";" character to ":" on >>>>>>>> security_group.py >>>>>>>> code to make it operational for ipv4 addresses but i am sure it will >>>>>>>> >>>>>>> break >>>>>>> >>>>>>>> Wido's "Add support for ipv6 address and subnets" PR. Workaround >>>>>>>> >>>>>>> works >>>>> >>>>>> only >>>>>>> >>>>>>>> for us because we have ipv4 only setup. >>>>>>>> >>>>>>>> If Wido could check parse_network_rules function on >>>>>>>> security_group.py >>>>>>>> >>>>>>> then >>>>>>> >>>>>>>> that could be great. After his check and possible code fix i like to >>>>>>>> >>>>>>> make >>>>>> >>>>>>> test again on our environment. >>>>>>>> >>>>>>>> @Rohit i will create a JIRA ticket to follow it easily by team. >>>>>>>> >>>>>>>> Thanks >>>>>>>> Özhan >>>>>>>> >>>>>>>> On Fri, Jan 19, 2018 at 2:51 PM, Rohit Yadav < >>>>>>>> >>>>>>> rohit.ya...@shapeblue.com> >>>>>> >>>>>>> wrote: >>>>>>>> >>>>>>>> Hi Ozhan, >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks for sharing. >>>>>>>>> >>>>>>>>> >>>>>>>>> I traced the change to the following PR that changes the delimiter >>>>>>>>> character to ';' than ":" to support ipv6 addresses: >>>>>>>>> >>>>>>>>> https://github.com/apache/cloudstack/pull/2028/files >>>>>>>>> >>>>>>>>> >>>>>>>>> Can you share with the workaround, if applicable send a pull >>>>>>>>> >>>>>>>> request? >>>>> >>>>>> >>>>>>>>> >>>>>>>>> Were you still using old 4.9.3 VRs post upgrade, does killing old >>>>>>>>> >>>>>>>> 4.9 >>>>> >>>>>> VRs >>>>>>> >>>>>>>> help fix the issue? /cc Wido >>>>>>>>> >>>>>>>>> >>>>>>>>> - Rohit >>>>>>>>> >>>>>>>>> <https://cloudstack.apache.org> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> ________________________________ >>>>>>>>> From: Özhan Rüzgar Karaman <oruzgarkara...@gmail.com> >>>>>>>>> Sent: Friday, January 19, 2018 3:38:19 PM >>>>>>>>> To: dev@cloudstack.apache.org >>>>>>>>> Subject: Re: [4.11] KVM Advanced Networking with SG Problem >>>>>>>>> >>>>>>>>> Hi; >>>>>>>>> We solved the bug there and write a small workaround today, the >>>>>>>>> >>>>>>>> problem >>>>>> >>>>>>> is >>>>>>>> >>>>>>>>> generally from the Java code which calls security_group.py. On >>>>>>>>> >>>>>>>> 4.9.3 >>>>> >>>>>> release it was using : character but from 4.11 release delimiter >>>>>>>>> >>>>>>>> changed >>>>>>> >>>>>>>> to >>>>>>>> >>>>>>>>> ; character but security_group.py expects : as delimeter so >>>>>>>>> security_group.py could not parse & send rules to the iptables. >>>>>>>>> >>>>>>>>> Afternoon i will create a JIRA ticket and if anyone could fix the >>>>>>>>> >>>>>>>> delimiter >>>>>>>> >>>>>>>>> character or code in the Java code for 4.11 release that would be >>>>>>>>> >>>>>>>> great >>>>>> >>>>>>> because without this code Security Groups are not operational for >>>>>>>>> >>>>>>>> 4.11. >>>>>> >>>>>>> >>>>>>>>> Also @Rohit do we need to check test codes for Security Groups? >>>>>>>>> >>>>>>>> Because i >>>>>>> >>>>>>>> do not understand how this bug passed our testing scenarios. >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> Özhan >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, Jan 19, 2018 at 12:00 PM, Rohit Yadav < >>>>>>>>> >>>>>>>> rohit.ya...@shapeblue.com >>>>>>> >>>>>>>> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Can anyone help look into this issue, reproduce it and if it's a >>>>>>>>>> >>>>>>>>> genuine >>>>>>>> >>>>>>>>> bug help fix it? >>>>>>>>>> >>>>>>>>>> Any takers - Wido, Wei, Mike and others who may be using KVM+SG? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> - Rohit >>>>>>>>>> >>>>>>>>>> <https://cloudstack.apache.org> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> ________________________________ >>>>>>>>>> From: Özhan Rüzgar Karaman <oruzgarkara...@gmail.com> >>>>>>>>>> Sent: Tuesday, January 16, 2018 9:53:59 PM >>>>>>>>>> To: dev@cloudstack.apache.org >>>>>>>>>> Subject: [4.11] KVM Advanced Networking with SG Problem >>>>>>>>>> >>>>>>>>>> Hi; >>>>>>>>>> We made a test with 4.11 rc over Ubuntu16.04 KVM hosts and we >>>>>>>>>> >>>>>>>>> noticed >>>>>> >>>>>>> that >>>>>>>>> >>>>>>>>>> there is a problem on setting & applying security group changes >>>>>>>>>> >>>>>>>>> on >>>>> >>>>>> KVM >>>>>>> >>>>>>>> host. >>>>>>>>>> >>>>>>>>>> All instances could ping vr and they could access internet but no >>>>>>>>>> >>>>>>>>> one >>>>>> >>>>>>> could >>>>>>>>> >>>>>>>>>> access to the instances. >>>>>>>>>> >>>>>>>>>> I checked iptables rules and i noticed that iptables rules for vm >>>>>>>>>> >>>>>>>>> is >>>>>> >>>>>>> in >>>>>>> >>>>>>>> all >>>>>>>>> >>>>>>>>>> drop state for incoming packages while i gave access to all >>>>>>>>>> >>>>>>>>> ingress >>>>> >>>>>> and >>>>>>> >>>>>>>> egress tcp/udp traffic ports for that instances. Below are >>>>>>>>>> >>>>>>>>> iptables >>>>> >>>>>> output >>>>>>>>> >>>>>>>>>> for selected vm: >>>>>>>>>> >>>>>>>>>> Chain i-2-6-VM (1 references) >>>>>>>>>> target prot opt source destination >>>>>>>>>> DROP all -- anywhere anywhere >>>>>>>>>> >>>>>>>>>> Chain i-2-6-VM-eg (1 references) >>>>>>>>>> target prot opt source destination >>>>>>>>>> RETURN all -- anywhere anywhere >>>>>>>>>> >>>>>>>>>> Chain i-2-6-def (2 references) >>>>>>>>>> target prot opt source destination >>>>>>>>>> ACCEPT all -- anywhere anywhere >>>>>>>>>> >>>>>>>>> state >>>>> >>>>>> RELATED,ESTABLISHED >>>>>>>>>> ACCEPT udp -- anywhere anywhere >>>>>>>>>> >>>>>>>>> PHYSDEV >>>>>> >>>>>>> match >>>>>>>>> >>>>>>>>>> --physdev-in vnet9 --physdev-is-bridged udp spt:bootpc dpt:bootps >>>>>>>>>> ACCEPT udp -- anywhere anywhere >>>>>>>>>> >>>>>>>>> PHYSDEV >>>>>> >>>>>>> match >>>>>>>>> >>>>>>>>>> --physdev-out vnet9 --physdev-is-bridged udp spt:bootps >>>>>>>>>> >>>>>>>>> dpt:bootpc >>>>> >>>>>> DROP all -- anywhere anywhere >>>>>>>>>> >>>>>>>>> PHYSDEV >>>>>> >>>>>>> match >>>>>>>>> >>>>>>>>>> --physdev-in vnet9 --physdev-is-bridged ! match-set i-2-6-VM src >>>>>>>>>> RETURN udp -- anywhere >>>>>>>>>> <https://maps.google.com/?q=N%C2%A0%C2%A0%C2%A0%C2%A0+udp%C2%A0+--%C2%A0+anywhere%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0&entry=gmail&source=g> >>>>>>>>>> anywhere >>>>>>>>>> >>>>>>>>> PHYSDEV >>>>>> >>>>>>> match >>>>>>>>> >>>>>>>>>> --physdev-in vnet9 --physdev-is-bridged match-set i-2-6-VM src >>>>>>>>>> >>>>>>>>> udp >>>>> >>>>>> dpt:domain >>>>>>>>>> RETURN tcp -- anywhere anywhere >>>>>>>>>> >>>>>>>>> PHYSDEV >>>>>> >>>>>>> match >>>>>>>>> >>>>>>>>>> --physdev-in vnet9 --physdev-is-bridged match-set i-2-6-VM src >>>>>>>>>> >>>>>>>>> tcp >>>>> >>>>>> dpt:domain >>>>>>>>>> i-2-6-VM-eg all -- anywhere anywhere >>>>>>>>>> >>>>>>>>> PHYSDEV >>>>>>> >>>>>>>> match --physdev-in vnet9 --physdev-is-bridged match-set i-2-6-VM >>>>>>>>>> >>>>>>>>> src >>>>>> >>>>>>> i-2-6-VM all -- anywhere anywhere >>>>>>>>>> >>>>>>>>> PHYSDEV >>>>>> >>>>>>> match >>>>>>>>> >>>>>>>>>> --physdev-out vnet9 --physdev-is-bridged >>>>>>>>>> >>>>>>>>>> All management and agent logs could be accessed from: >>>>>>>>>> http://51.15.199.7/4.11r1_Test_20190116.tgz >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> Özhan >>>>>>>>>> >>>>>>>>>> rohit.ya...@shapeblue.com >>>>>>>>>> www.shapeblue.com<http://www.shapeblue.com> >>>>>>>>>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK >>>>>>>>>> @shapeblue >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> rohit.ya...@shapeblue.com >>>>>>>>> www.shapeblue.com<http://www.shapeblue.com> >>>>>>>>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK >>>>>>>>> @shapeblue >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Daan >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Daan >>>>> >>>>> >>>> >>> rohit.ya...@shapeblue.com >>> www.shapeblue.com >>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK >>> @shapeblue >>> >>> >>>
