GitHub user TadiosAbebe created a discussion: Shared network issue on advanced
zone with security group
Hello community,
This past few days i was testing shared network in advanced zone with security
group and i think i am just missing a lot of understanding about it. and here
is my issue
- created a shared network with custom vlan(12) which is out of range of
specified for guest network(2048-3072)
- then as soon as i launch an instance on that shared network, other isolated
networks connectivity stops working for the physical host where the shared
network instance reside in.
- so for example if i have a VR1 and Instance1 in physical host 5, and as
soon as the shared network instance gets allocated on physical host 5, the
connectivity between Instance1 -> VR1, VR1 -> internet stops working and the
same for all others VR and instances on that host
- and I think i boiled down the above issue as being an iptable rules and here
is my findings
- iptables -L -n -v listing before creating a shared network and an
instance on the shared network
```
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source
destination
Chain FORWARD (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source
destination
0 0 DOCKER-USER 0 -- * * 0.0.0.0/0
0.0.0.0/0
0 0 DOCKER-ISOLATION-STAGE-1 0 -- * * 0.0.0.0/0
0.0.0.0/0
0 0 ACCEPT 0 -- * docker0 0.0.0.0/0
0.0.0.0/0 ctstate RELATED,ESTABLISHED
0 0 DOCKER 0 -- * docker0 0.0.0.0/0
0.0.0.0/0
0 0 ACCEPT 0 -- docker0 !docker0 0.0.0.0/0
0.0.0.0/0
0 0 ACCEPT 0 -- docker0 docker0 0.0.0.0/0
0.0.0.0/0
Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source
destination
Chain DOCKER (1 references)
pkts bytes target prot opt in out source
destination
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
pkts bytes target prot opt in out source
destination
0 0 DOCKER-ISOLATION-STAGE-2 0 -- docker0 !docker0 0.0.0.0/0
0.0.0.0/0
0 0 RETURN 0 -- * * 0.0.0.0/0
0.0.0.0/0
Chain DOCKER-ISOLATION-STAGE-2 (1 references)
pkts bytes target prot opt in out source
destination
0 0 DROP 0 -- * docker0 0.0.0.0/0
0.0.0.0/0
0 0 RETURN 0 -- * * 0.0.0.0/0
0.0.0.0/0
Chain DOCKER-USER (1 references)
pkts bytes target prot opt in out source
destination
0 0 RETURN 0 -- * * 0.0.0.0/0
0.0.0.0/0
```
- iptables -L -n -v listing after creating a shared network and an instance
on the shared network
```
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source
destination
Chain FORWARD (policy DROP 150 packets, 6817 bytes)
pkts bytes target prot opt in out source
destination
127 7160 BF-brbond0-12 0 -- * brbond0-12 0.0.0.0/0
0.0.0.0/0 PHYSDEV match --physdev-is-bridged
0 0 BF-brbond0-12 0 -- brbond0-12 * 0.0.0.0/0
0.0.0.0/0 PHYSDEV match --physdev-is-bridged
0 0 DROP 0 -- * brbond0-12 0.0.0.0/0
0.0.0.0/0
0 0 DROP 0 -- brbond0-12 * 0.0.0.0/0
0.0.0.0/0
150 6817 DOCKER-USER 0 -- * * 0.0.0.0/0
0.0.0.0/0
150 6817 DOCKER-ISOLATION-STAGE-1 0 -- * * 0.0.0.0/0
0.0.0.0/0
0 0 ACCEPT 0 -- * docker0 0.0.0.0/0
0.0.0.0/0 ctstate RELATED,ESTABLISHED
0 0 DOCKER 0 -- * docker0 0.0.0.0/0
0.0.0.0/0
0 0 ACCEPT 0 -- docker0 !docker0 0.0.0.0/0
0.0.0.0/0
0 0 ACCEPT 0 -- docker0 docker0 0.0.0.0/0
0.0.0.0/0
Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source
destination
Chain BF-brbond0-12 (2 references)
pkts bytes target prot opt in out source
destination
0 0 ACCEPT 0 -- * * 0.0.0.0/0
0.0.0.0/0 state RELATED,ESTABLISHED
127 7160 BF-brbond0-12-IN 0 -- * * 0.0.0.0/0
0.0.0.0/0 PHYSDEV match --physdev-is-in --physdev-is-bridged
121 5168 BF-brbond0-12-OUT 0 -- * * 0.0.0.0/0
0.0.0.0/0 PHYSDEV match --physdev-is-out --physdev-is-bridged
0 0 ACCEPT 0 -- * * 0.0.0.0/0
0.0.0.0/0 PHYSDEV match --physdev-out bond0.12 --physdev-is-bridged
Chain BF-brbond0-12-IN (1 references)
pkts bytes target prot opt in out source
destination
6 1992 i-167-1541-def 0 -- * * 0.0.0.0/0
0.0.0.0/0 PHYSDEV match --physdev-in vnet8 --physdev-is-bridged
Chain BF-brbond0-12-OUT (1 references)
pkts bytes target prot opt in out source
destination
121 5168 i-167-1541-def 0 -- * * 0.0.0.0/0
0.0.0.0/0 PHYSDEV match --physdev-out vnet8 --physdev-is-bridged
Chain DOCKER (1 references)
pkts bytes target prot opt in out source
destination
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
pkts bytes target prot opt in out source
destination
0 0 DOCKER-ISOLATION-STAGE-2 0 -- docker0 !docker0 0.0.0.0/0
0.0.0.0/0
150 6817 RETURN 0 -- * * 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-ISOLATION-STAGE-2 (1 references)
pkts bytes target prot opt in out source
destination
0 0 DROP 0 -- * docker0 0.0.0.0/0
0.0.0.0/0
0 0 RETURN 0 -- * * 0.0.0.0/0
0.0.0.0/0
Chain DOCKER-USER (1 references)
pkts bytes target prot opt in out source
destination
150 6817 RETURN 0 -- * * 0.0.0.0/0 0.0.0.0/0
Chain i-167-1541-VM (1 references)
pkts bytes target prot opt in out source
destination
0 0 ACCEPT 0 -- * * 0.0.0.0/0
0.0.0.0/0 state NEW
0 0 DROP 0 -- * * 0.0.0.0/0
0.0.0.0/0
Chain i-167-1541-VM-eg (1 references)
pkts bytes target prot opt in out source
destination
0 0 RETURN 0 -- * * 0.0.0.0/0
0.0.0.0/0 state NEW
0 0 DROP 0 -- * * 0.0.0.0/0
0.0.0.0/0
Chain i-167-1541-def (2 references)
pkts bytes target prot opt in out source
destination
0 0 ACCEPT 0 -- * * 0.0.0.0/0
0.0.0.0/0 state RELATED,ESTABLISHED
6 1992 ACCEPT 17 -- * * 0.0.0.0/0
0.0.0.0/0 PHYSDEV match --physdev-in vnet8 --physdev-is-bridged udp
spt:68 dpt:67
0 0 ACCEPT 17 -- * * 0.0.0.0/0
0.0.0.0/0 PHYSDEV match --physdev-out vnet8 --physdev-is-bridged udp
spt:67 dpt:68
0 0 DROP 17 -- * * 0.0.0.0/0
0.0.0.0/0 PHYSDEV match --physdev-in vnet8 --physdev-is-bridged udp
spt:67
0 0 DROP 0 -- * * 0.0.0.0/0
0.0.0.0/0 PHYSDEV match --physdev-in vnet8 --physdev-is-bridged !
match-set i-167-1541-VM src
121 5168 DROP 0 -- * * 0.0.0.0/0 0.0.0.0/0
PHYSDEV match --physdev-out vnet8 --physdev-is-bridged ! match-set
i-167-1541-VM dst
0 0 RETURN 17 -- * * 0.0.0.0/0
0.0.0.0/0 PHYSDEV match --physdev-in vnet8 --physdev-is-bridged
match-set i-167-1541-VM src udp dpt:53
0 0 RETURN 6 -- * * 0.0.0.0/0
0.0.0.0/0 PHYSDEV match --physdev-in vnet8 --physdev-is-bridged
match-set i-167-1541-VM src tcp dpt:53
0 0 i-167-1541-VM-eg 0 -- * * 0.0.0.0/0
0.0.0.0/0 PHYSDEV match --physdev-in vnet8 --physdev-is-bridged
match-set i-167-1541-VM src
0 0 i-167-1541-VM 0 -- * * 0.0.0.0/0
0.0.0.0/0 PHYSDEV match --physdev-out vnet8 --physdev-is-bridged
```
> As seen in the above output, the dropped packet in `Chain FORWARD (policy
> DROP 150 packets, 6817 bytes)` keeps just increasing by the seconds
> afterward. and I don't really understand why isolate network packets are
> getting dropped once i create a shared network.
- I think the DOCKER related chains in the iptables output are coming from
cephadm since i am using compute and storage on the same nodes
I am running apache cloudstack 4.20.1 with ubuntu server 24.04. I would be glad
if anyone can shade some light to this.
GitHub link: https://github.com/apache/cloudstack/discussions/11955
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
