jails in different private subnets on the same host
Is it possible to have two jails on the same host each one in a different private subnet, e.g. 192.168.1.0 and 10.33.1.0, and have routing between them working without issues? I know it's possible to run jails with IPs in those two subnets but it seems there is no routing and I am not sure if it's because I can't configure my router properly or there is a more fundamental problem. One issue I see is that the jail can't have a different default gateway than the host, and that for now is 192.168.1.1, but I don't see a reason why 10.33.1.0 wouldn't be able to use 192.168.1.1 as it's default gateway provided there is routing between those two subnets. Grzegorz ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: jails in different private subnets on the same host
> On 18 May 2016, at 14:00 , Grzegorz Junka wrote: > > Is it possible to have two jails on the same host each one in a different > private subnet, e.g. 192.168.1.0 and 10.33.1.0, and have routing between them > working without issues? > > I know it's possible to run jails with IPs in those two subnets but it seems > there is no routing and I am not sure if it's because I can't configure my > router properly or there is a more fundamental problem. One issue I see is > that the jail can't have a different default gateway than the host, and that > for now is 192.168.1.1, but I don't see a reason why 10.33.1.0 wouldn't be > able to use 192.168.1.1 as it's default gateway provided there is routing > between those two subnets. Given they are both on the same base system host, both addresses are connected locally and thus the kernel knows where to deliver these packets. If that doesn’t work, there is a bug somewhere. If you want different default gateways then you may want to look into using different FIBs for different jails. See route(8) and jail(8) for parameters to set and tune. /bz ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: jails in different private subnets on the same host
On 18/05/2016 14:11, Bjoern A. Zeeb wrote: >> On 18 May 2016, at 14:00 , Grzegorz Junka >> wrote: >> >> Is it possible to have two jails on the same host each one in a >> different private subnet, e.g. 192.168.1.0 and 10.33.1.0, and have >> routing between them working without issues? >> >> I know it's possible to run jails with IPs in those two subnets >> but it seems there is no routing and I am not sure if it's because >> I can't configure my router properly or there is a more >> fundamental problem. One issue I see is that the jail can't have a >> different default gateway than the host, and that for now is >> 192.168.1.1, but I don't see a reason why 10.33.1.0 wouldn't be >> able to use 192.168.1.1 as it's default gateway provided there is >> routing between those two subnets. > > Given they are both on the same base system host, both addresses > are connected locally and thus the kernel knows where to deliver > these packets. If that doesn’t work, there is a bug somewhere. > > If you want different default gateways then you may want to look > into using different FIBs for different jails. See route(8) and > jail(8) for parameters to set and tune. > > /bz > I can ping both jails from the main host, however when in the 10.33.1.0 jail I can't access any jail in the 192.168.1.0 network. This is what netstat -r shows: - root@dns1:/ # ifconfig em0: flags=8843 metric 0 mtu 1500 options=4219b ether 00:25:90:ae:e8:bc media: Ethernet autoselect (1000baseT ) status: active em1: flags=8843 metric 0 mtu 1500 options=4219b ether 00:25:90:ae:e8:bc media: Ethernet autoselect (1000baseT ) status: active lo0: flags=8049 metric 0 mtu 16384 options=63 lagg0: flags=8843 metric 0 mtu 1500 options=4219b ether 00:25:90:ae:e8:bc inet 192.168.1.60 netmask 0x broadcast 192.168.1.60 media: Ethernet autoselect status: active laggproto lacp lagghash l2,l3,l4 laggport: em0 flags=1c laggport: em1 flags=1c root@dns1:/ # netstat -r Routing tables Internet: DestinationGatewayFlags Netif Expire dns1 link#4 UHS lo0 - root@pjp1:/ # ifconfig em0: flags=8843 metric 0 mtu 1500 options=4219b ether 00:25:90:ae:e8:bc media: Ethernet autoselect (1000baseT ) status: active em1: flags=8843 metric 0 mtu 1500 options=4219b ether 00:25:90:ae:e8:bc media: Ethernet autoselect (1000baseT ) status: active lo0: flags=8049 metric 0 mtu 16384 options=63 lagg0: flags=8843 metric 0 mtu 1500 options=4219b ether 00:25:90:ae:e8:bc inet 10.33.1.40 netmask 0x broadcast 10.33.1.40 media: Ethernet autoselect status: active laggproto lacp lagghash l2,l3,l4 laggport: em0 flags=1c laggport: em1 flags=1c root@pjp1:/ # netstat -r netstat: kvm not available: /dev/mem: No such file or directory Routing tables rt_tables: symbol not in namelist - On the main host: root@somehost:~ # netstat -r Routing tables Internet: Destination GatewayFlags Netif Expire default 192.168.1.1UGS lagg0 pjp1.somehost.somedomain. link#4 UHS lo0 10.33.1.40/32 link#4 U lagg0 localhost link#3 UH lo0 192.168.1.0 link#4 U lagg0 somehost link#4 UHS lo0 web1.somehost.somedomain. link#4 UHS lo0 192.168.1.50/32 link#4 U lagg0 dns1.somehost.somedomain. link#4 UHS lo0 192.168.1.60/32 link#4 U lagg0 (... other jails) Internet6: DestinationGatewayFlags Netif Expire :: localhost UGRSlo0 localhost link#3 UH lo0 :::0.0.0.0 localhost UGRSlo0 fe80:: localhost UGRSlo0 fe80::%lo0 link#3 U lo0 fe80::1%lo0link#3 UHS lo0 ff01::%lo0 localhost U lo0 ff02:: localhost UGRSlo0 ff02::%lo0 localhost U lo0 - I would rather not set up different FIBs for different jails, unless required. First of all I would like to establish what's wrong. I just tried telnet 192.168.1.50 80 from the main host and from the 10.33.1.40 jail. From the main host it works without issues. From the jail it eventually connected after 15 or so seconds of waiting. Grzegorz ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
netstat -rn in jail doesn't work
What may be the reason that netstat -rn works in one jail and doesn't in another? root@app2:/ # netstat -rn Routing tables Internet: DestinationGatewayFlags Netif Expire 192.168.1.76 link#4 UHS lo0 root@pjp1:/ # netstat -rn netstat: kvm not available: /dev/mem: No such file or directory Routing tables rt_tables: symbol not in namelist Grzegorz ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: netstat -rn in jail doesn't work
On 2016-05-18 12:37, Grzegorz Junka wrote: > What may be the reason that netstat -rn works in one jail and doesn't in > another? > > root@app2:/ # netstat -rn > Routing tables > > Internet: > DestinationGatewayFlags Netif Expire > 192.168.1.76 link#4 UHS lo0 > > > root@pjp1:/ # netstat -rn > netstat: kvm not available: /dev/mem: No such file or directory > Routing tables > rt_tables: symbol not in namelist > > Grzegorz > > ___ > freebsd-jail@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-jail > To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org" Do you have /dev/mem exposed in one of them? -- Allan Jude ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: netstat -rn in jail doesn't work
On 18/05/2016 16:38, Allan Jude wrote: On 2016-05-18 12:37, Grzegorz Junka wrote: What may be the reason that netstat -rn works in one jail and doesn't in another? root@app2:/ # netstat -rn Routing tables Internet: DestinationGatewayFlags Netif Expire 192.168.1.76 link#4 UHS lo0 root@pjp1:/ # netstat -rn netstat: kvm not available: /dev/mem: No such file or directory Routing tables rt_tables: symbol not in namelist Grzegorz ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org" Do you have /dev/mem exposed in one of them? No. It's just that -rn doesn't show that error in that particular case, but it shows it at another occasion: root@app2:/ # netstat -a netstat: kvm not available: /dev/mem: No such file or directory Active Internet connections (including servers) Proto Recv-Q Send-Q Local Address Foreign Address (state) tcp4 0 0 app2.8484 *.* LISTEN tcp4 0 0 app2.smtp *.* LISTEN tcp4 0 0 app2.ssh *.* LISTEN udp4 0 0 app2.syslog*.* Active UNIX domain sockets Address Type Recv-Q Send-QInode Conn Refs Nextref Addr f8003068f870 dgram 0 00 f800306348700 f80030634780 f8003068f960 dgram 0 00 f80030634960 00 f80030634780 dgram 0 00 f80030634870 00 f80030634870 dgram 0 0 f801057be1d80 f8003068f8700 /var/run/logpriv f80030634960 dgram 0 0 f801058eb1d80 f8003068f9600 /var/run/log ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: netstat -rn in jail doesn't work
Grzegorz Junka wrote on 05/18/2016 18:37: What may be the reason that netstat -rn works in one jail and doesn't in another? root@app2:/ # netstat -rn Routing tables Internet: DestinationGatewayFlags Netif Expire 192.168.1.76 link#4 UHS lo0 root@pjp1:/ # netstat -rn netstat: kvm not available: /dev/mem: No such file or directory Routing tables rt_tables: symbol not in namelist I don't know the reason but I can confirm this behavior. I know about this for a long time. Netstat complains about /dev/mem for some other params too even if it outputs correct values for example for opened tcp connections: /# netstat -s -p tcp netstat: kvm not available: /dev/mem: No such file or directory tcp: 1517892073 packets sent 1453939900 data packets (2274781047202 bytes) 759536 data packets (929141944 bytes) retransmitted 59175 data packets unnecessarily retransmitted 0 resends initiated by MTU discovery 51907865 ack-only packets (26667901 delayed) 0 URG only packets 267 window probe packets 795506 window update packets 10493883 control packets 1487401217 packets received 1417951529 acks (for 2273802396874 bytes) 7502860 duplicate acks 38600 acks for unsent data 1368386110 packets (2153255668968 bytes) received in-sequence 222423 completely duplicate packets (39239815 bytes) 11980 old duplicate packets 221 packets with some dup. data (94160 bytes duped) 35171 out-of-order packets (15770219 bytes) 21 packets (11 bytes) of data after window 11 window probes 1863690 window update packets 1642030 packets received after close 281 discarded for bad checksums 0 discarded for bad header offset fields 0 discarded because packet too short 87 discarded due to memory problems 2448384 connection requests 7800552 connection accepts 0 bad connection attempts 109 listen queue overflows 339306 ignored RSTs in the windows 10221160 connections established (including accepts) 10554092 connections closed (including 1990441 drops) 5674590 connections updated cached RTT on close 5677848 connections updated cached RTT variance on close 1583021 connections updated cached ssthresh on close 10125 embryonic connections dropped 1405786035 segments updated rtt (of 1374995187 attempts) 404689 retransmit timeouts 1681 connections dropped by rexmit timeout 608 persist timeouts 0 connections dropped by persist timeout 0 Connections (fin_wait_2) dropped because of timeout 12388 keepalive timeouts 11896 keepalive probes sent 492 connections dropped by keepalive 38184853 correct ACK header predictions 46419366 correct data packet header predictions 7826351 syncache entries added 45759 retransmitted 55797 dupsyn 84 dropped 7800552 completed 40 bucket overflow 0 cache overflow 19220 reset 7941 stale 109 aborted 0 badack 230 unreach 0 zone failures 7826435 cookies sent 1784 cookies received 212203 hostcache entries added 28 bucket overflow 104273 SACK recovery episodes 242234 segment rexmits in SACK recovery episodes 303575028 byte rexmits in SACK recovery episodes 1538523 SACK options (SACK blocks) received 12421 SACK options (SACK blocks) sent 114 SACK scoreboard overflow 0 packets with ECN CE bit set 0 packets with ECN ECT(0) bit set 0 packets with ECN ECT(1) bit set 0 successful ECN handshakes 0 times ECN reduced the congestion window 0 packets with valid tcp-md5 signature received 0 packets with invalid tcp-md5 signature received 0 packets with tcp-md5 signature mismatch 0 packets with unexpected tcp-md5 signature received 0 packets without expected tcp-md5 signature received I tried netstat -rn in all 8 jails on our test machine. 4 of them works, the other 4 don't work. netstat -rn doesn't work in those jail which are older than host environment netstat -s -p tcp prints error message even in the newest jails: netstat: kvm not available: /dev/mem: No such file or directory Miroslav Lachman ___ freebsd-jail@freebsd.org mailing list https://lists.
Re: netstat -rn in jail doesn't work
OK, thanks, so it looks like it doesn't prevent the jail from working correctly, it's just the reporting that's broken. Grzegorz On 18/05/2016 18:28, Miroslav Lachman wrote: Grzegorz Junka wrote on 05/18/2016 18:37: What may be the reason that netstat -rn works in one jail and doesn't in another? root@app2:/ # netstat -rn Routing tables Internet: DestinationGatewayFlags Netif Expire 192.168.1.76 link#4 UHS lo0 root@pjp1:/ # netstat -rn netstat: kvm not available: /dev/mem: No such file or directory Routing tables rt_tables: symbol not in namelist I don't know the reason but I can confirm this behavior. I know about this for a long time. Netstat complains about /dev/mem for some other params too even if it outputs correct values for example for opened tcp connections: /# netstat -s -p tcp netstat: kvm not available: /dev/mem: No such file or directory tcp: 1517892073 packets sent 1453939900 data packets (2274781047202 bytes) 759536 data packets (929141944 bytes) retransmitted 59175 data packets unnecessarily retransmitted 0 resends initiated by MTU discovery 51907865 ack-only packets (26667901 delayed) 0 URG only packets 267 window probe packets 795506 window update packets 10493883 control packets 1487401217 packets received 1417951529 acks (for 2273802396874 bytes) 7502860 duplicate acks 38600 acks for unsent data 1368386110 packets (2153255668968 bytes) received in-sequence 222423 completely duplicate packets (39239815 bytes) 11980 old duplicate packets 221 packets with some dup. data (94160 bytes duped) 35171 out-of-order packets (15770219 bytes) 21 packets (11 bytes) of data after window 11 window probes 1863690 window update packets 1642030 packets received after close 281 discarded for bad checksums 0 discarded for bad header offset fields 0 discarded because packet too short 87 discarded due to memory problems 2448384 connection requests 7800552 connection accepts 0 bad connection attempts 109 listen queue overflows 339306 ignored RSTs in the windows 10221160 connections established (including accepts) 10554092 connections closed (including 1990441 drops) 5674590 connections updated cached RTT on close 5677848 connections updated cached RTT variance on close 1583021 connections updated cached ssthresh on close 10125 embryonic connections dropped 1405786035 segments updated rtt (of 1374995187 attempts) 404689 retransmit timeouts 1681 connections dropped by rexmit timeout 608 persist timeouts 0 connections dropped by persist timeout 0 Connections (fin_wait_2) dropped because of timeout 12388 keepalive timeouts 11896 keepalive probes sent 492 connections dropped by keepalive 38184853 correct ACK header predictions 46419366 correct data packet header predictions 7826351 syncache entries added 45759 retransmitted 55797 dupsyn 84 dropped 7800552 completed 40 bucket overflow 0 cache overflow 19220 reset 7941 stale 109 aborted 0 badack 230 unreach 0 zone failures 7826435 cookies sent 1784 cookies received 212203 hostcache entries added 28 bucket overflow 104273 SACK recovery episodes 242234 segment rexmits in SACK recovery episodes 303575028 byte rexmits in SACK recovery episodes 1538523 SACK options (SACK blocks) received 12421 SACK options (SACK blocks) sent 114 SACK scoreboard overflow 0 packets with ECN CE bit set 0 packets with ECN ECT(0) bit set 0 packets with ECN ECT(1) bit set 0 successful ECN handshakes 0 times ECN reduced the congestion window 0 packets with valid tcp-md5 signature received 0 packets with invalid tcp-md5 signature received 0 packets with tcp-md5 signature mismatch 0 packets with unexpected tcp-md5 signature received 0 packets without expected tcp-md5 signature received I tried netstat -rn in all 8 jails on our test machine. 4 of them works, the other 4 don't work. netstat -rn doesn't work in those jail which are older than host environment netstat -s -p tcp prints error message even in the newest jails: net
