Re: Poor performance with stable/13 and Mellanox ConnectX-6 (mlx5)
On Fri, 17 Jun 2022, at 02:38, Mike Jakubik wrote: > Hi, > > I believe you hit the nail on the head! I am now getting consistent > high speeds, even higher than on Linux! Is this a problem with the > scheduler? Should someone in that area of expertise be made aware of > this? More importantly i guess, would this affect real world > performance, these servers will be running RabbitMQ (it uses quite a > bit of bandwidth) and PostgreSQL w/ replication. pinning cores for unimpeded access is very common for high performance systems. Do this both for the nics and also your apps. Be mindful of the NUMA topooogy. You should look into both the erlang scheduler flags for core pinning, and also ensuring that your erlang processes have unimpeded access to their own cores too. A reasonable approach is to make a simple cowboy or Phoenix app and hammer it with wrk or similar load tool to get a feel for things, and then profile and tune your own app based on those specific results. For rabbit there is an excellent load testing tool from the pivotal team if you don’t have suitable load generators yourselves. Tsung is an excellent tool if you put in the work to craft something specific for your use case. Please post back to the list with your specific findings and nic/ tcp tunables, these are very helpful for the next person! Dave
missing SYN/ACK for inbound TCP solved by altering broadcast address - why?
I've found a workaround for this issue, but don't understand why this
occurs. Reading RFC1122 has left me none the wiser. What am I missing?
Is this a Linuxism or simple a standardisation loophole?
## Problem
- on 13.1-R, dhclient-set config works for all UDP, & outbound TCP
- but inbound TCP connections send no SYN/ACK at all back
- on Linux Ubuntu 22.04 & others, the DHCP supplied IP config
works as expected
failing FreeBSD config from dhclient:
inet 147.75.93.61 netmask 0xfffe broadcast 147.75.93.60
working Linux config (note broadcast)
inet 147.75.93.61 netmask 0xfffe broadcast 255.255.255.254
^^^
- full details below (dhcp lease, ifconfigs etc)
I worked around this by forcing broadcast-address in dhclient.conf:
## /etc/dhclient.conf
interface "ice0" {
supersede broadcast-address 255.255.255.255;
}
# repeat for other ifaces as required
Which is ~ok~ for the moment, but I'd like to understand why this
occurs, and fix it properly. Either at DHCPD end, or FreeBSD
config.
# Further details
- Ubuntu 22.04 from vendor
- FreeBSD 13.1-RELEASE amd64 vanilla install
- 4x ice(4) NICs (Intel E810) and 2x (unused) ix (igxbe)
- 2x of the ice(4) are bonded link aggregation
- dhclient only used to attach to 1 nic, ignoring FreeBSD side of bonding
## Linux ip addr
# ip addr
8: bond0: mtu 1500 qdisc noqueue state
UP group default qlen 1000
link/ether b4:96:91:d9:99:20 brd ff:ff:ff:ff:ff:ff
inet 147.75.92.187/31 brd 255.255.255.255 scope global bond0
...
## FreeBSD ifconfig
# ifconfig ice0
ice0: flags=8863 metric 0 mtu 1500
options=4e10438
ether b4:96:91:d9:9b:48
inet 147.75.93.61 netmask 0xfffe broadcast 147.75.93.60
media: Ethernet autoselect (25G-AUI )
status: active
nd6 options=29
...
root@metalBSD:~ # netstat -4rn
Routing tables
Internet:
DestinationGatewayFlags Netif Expire
default147.75.93.60 UGSice0
127.0.0.1 link#7 UH lo0
147.75.93.60/31link#3 U ice0
147.75.93.61 link#3 UHS lo0
root@metalBSD:~ # cat /var/db/dhclient.leases.ice0
- note no broadcast-address provided
- Linux & FreeBSD evidently derive it differently
lease {
interface "ice0";
fixed-address 147.75.93.61;
option subnet-mask 255.255.255.254;
option routers 147.75.93.60;
option domain-name-servers 147.75.207.207,147.75.207.208;
option host-name "intransigent09";
option dhcp-lease-time 172800;
option dhcp-message-type 5;
option dhcp-server-identifier 139.178.78.140;
renew 1 2022/6/27 18:40:06;
rebind 2 2022/6/28 12:40:06;
expire 2 2022/6/28 18:40:06;
}
A+
Dave
PR241792 dhclient trivial fix + MFC request
TLDR cloud providers expect FreeBSD to behave like linux, and assign a default interface route to a /32 lease. Without this, images start up without a default interface. Very inconvenient. Index: sbin/dhclient/dhclient-script === --- sbin/dhclient/dhclient-script (revision 354408) +++ sbin/dhclient/dhclient-script (working copy) @@ -173,6 +173,9 @@ if [ "$new_ip_address" = "$router" ]; then route add default -iface $router >/dev/null 2>&1 else + if [ "$new_subnet_mask" = "255.255.255.255" ]; then + route add "$router" -iface "$interface" >/dev/null 2>&1 + fi route add default $router >/dev/null 2>&1 fi fi https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=241792 for more details. Would be great to have this in CURRENT and MFC'd to 13.x and 12.x too if possible. A+ Dave
Re: SO_SPLICE implementation
On Thu, 29 Aug 2024, at 21:43, Mark Johnston wrote: > Hello, > > Drew Gallatin and I have been working on an implementation of SO_SPLICE, > an interface which allows TCP connections to be spliced together. This > is intended for use in proxy applications to reduce the overhead of > copying data between connections. Awesome! > At the moment the interface isn't widely used, though I'll make sure > that net/relayd can take advantage of it (which it already does on > OpenBSD). If there are existing applications that might take advantage > of it, especially in the FreeBSD base system, I'd be interested to hear > suggestions. net/haproxy would be another good one. Currently it's enabled by option https://docs.haproxy.org/3.0/configuration.html#4.2-option splice-auto and AFAICT requires https://github.com/haproxy/haproxy/blob/master/Makefile#L30 preset in https://github.com/haproxy/haproxy/blob/master/Makefile#L416 > The code review is here if anyone's interested in looking at the > implementation: https://reviews.freebsd.org/D46411 A+ Dave
NATted outbound traffic sometimes uses backup CARP IP on LACP/LAGG interface
Hi,
Outgoing traffic (from a jail) via PF NAT over a LAGG/LACP sometimes has
the *backup* CARP IP address assigned to it.
Obivously as this IP is only active on the "other" server, the return
TCP connection traffic never actually gets back to our CARP master, and
the other server sees spurious TCP connections. This is very
reproducible and appears to be deterministic, like a round robin IP
allocation. In practice, inside a jail, `curl $URL` will fail
repeatedly.
Hopefully this is some misconfiguration on my part - what am I doing
wrong?
BTW I wrote this up a while back on the forums where the config files
are easier to read: https://forums.freebsd.org/threads/61552
###
# /etc/rc.conf network
ifconfig_igb0="up"
ifconfig_igb1="up"
cloned_interfaces="${cloned_interfaces} lagg0"
defaultrouter="1.2.3.81"
ipv6_defaultrouter="1:2:3:4::1"
ifconfig_lagg0="inet 1.2.3.83/28 laggproto lacp laggport igb0 laggport
igb1"
ifconfig_lagg0_ipv6="inet6 1:2:3:4::83/64"
# carp on
kld_list="${kld_list} carp"
ifconfig_lagg0_aliases="\
inet vhid 1 advskew 100 pass pw1 1.2.3.84/32 \
inet6 vhid 2 advskew 100 pass pw2 1:2:3:4::84/64 \
inet vhid 3 advskew 0 pass pw3 1.2.3.85/32 \
inet6 vhid 4 advskew 0 pass pw4 1:2:3:4::85/64 \
"
# jail networks use their own separate cloned if
cloned_interfaces="${cloned_interfaces} lo1"
ifconfig_lo1_aliases="inet 10.241.0.0-15/16"
###
# /etc/pf.conf
# macros
protocols = "{ tcp, udp, icmp }"
# interfaces
extl_if="lagg0"
intl_if="lo0"
jail_if="lo1"
# networks
intl_net = $intl_if:network
jail_net = $jail_if:network
internet = $extl_if:network
# limits
set limit { states 20, frags 8, src-nodes 8 }
set timeout { adaptive.start 18, adaptive.end 20 }
# clean packets are happy packets
scrub in all
# jails are allowed outbound connections but not inbound
nat on $extl_if proto $protocols from $jail_net to any -> ($extl_if)
# o ye of little faith
pass in all
pass out all
###
running configs ##
pfctl indeed shows its a round-robin
###
# pfctl -vnf /etc/pf.conf
protocols = "{ tcp, udp, icmp }"
extl_if = "lagg0"
intl_if = "lo0"
jail_if = "lo1"
intl_net = "lo0:network"
jail_net = "lo1:network"
internet = "lagg0:network"
set limit states 20
set limit frags 8
set limit src-nodes 8
set timeout adaptive.start 18
set timeout adaptive.end 20
scrub in all fragment reassemble
nat on lagg0 inet proto tcp from 10.241.0.0/16 to any -> (lagg0)
round-robin
nat on lagg0 inet proto tcp from 10.241.0.1 to any -> (lagg0)
round-robin
... repeated for each IP
###
# ifconfig
lagg0: flags=8943 metric
0 mtu 1500
options=6403bb
ether 78:45:c4:fa:d2:99
inet 1.2.3.82 netmask 0xfff0 broadcast 1.2.3.95
* inet 1.2.3.84 netmask 0x broadcast 1.2.3.84 vhid 1
* inet 1.2.3.85 netmask 0x broadcast 1.2.3.85 vhid 3
inet6 fe80::7a45:c4ff:fefa:d299%lagg0 prefixlen 64 scopeid 0x4
inet6 1:2:3:4::82 prefixlen 64
* inet6 1:2:3:4::84 prefixlen 64 vhid 2
* inet6 1:2:3:4::85 prefixlen 64 vhid 4
nd6 options=21
media: Ethernet autoselect
status: active
* carp: MASTER vhid 1 advbase 1 advskew 0
* carp: BACKUP vhid 3 advbase 1 advskew 100
* carp: MASTER vhid 2 advbase 1 advskew 0
* carp: BACKUP vhid 4 advbase 1 advskew 100
groups: lagg
laggproto lacp lagghash l2,l3,l4
* laggport: igb0 flags=1c
* laggport: igb1 flags=1c
# I removed the lines appended with !!!.. so that the system
actually works atm
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: NATted outbound traffic sometimes uses backup CARP IP on LACP/LAGG interface
On Thu, 14 Sep 2017, at 23:32, Kristof Provost wrote: > On 14 Sep 2017, at 16:21, Dave Cottlehuber wrote: > > Outgoing traffic (from a jail) via PF NAT over a LAGG/LACP sometimes > > has the *backup* CARP IP address assigned to it. > > > I think this is your problem. You’re telling pf to nat to the IP > address of lagg0, but lagg0 has multiple addresses assigned. > > ‘(lagg0:0)’ should work, or just use the IP address. Thanks Kristof! ($if:0) works perfectly, but I'll need to reboot to test this with the original carp setup though. Can you explain what $if:0 resolves to, for example how does it relate to to the primary ipv4/6 addresses bound to that interface? I couldn't find a reference in the usual ifconfig manpages about this (ifname:#) format, the BNF grammar for pf.conf doesn't cover it either, and `pfctl -vnf ...` simply shows (lagg0:0). A+ Dave ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
'no route to host" for cloned lo1 iface 12.0-CURRENT r334376+56a973815425(master) amd64
I have 10.241.0.0-15/16 bound to lo1 for jails, and I just realised it doesn't
work anymore. Boxes at $DAYJOB use the same config and work just fine on
11.1R/amd64, otherwise I'd be fired this used to work ~ 1 month ago, both
on my desktop, and my laptop, and now doesn't.
Have I mis-configured something blindingly obvious here before I embark on a
lengthy bisect?
# grep -v \# /etc/rc.conf , elided
hostname=akai
kld_list="${kld_list} if_iwm if_axge if_ure if_ue"
ifconfig_ue0=DHCP
ifconfig_ue0_ipv6="inet6 accept_rtadv"
wlans_iwm0="wlan0"
ifconfig_wlan0="WPA DHCP"
ifconfig_wlan0_ipv6="inet6 accept_rtadv"
create_args_wlan0="country AT regdomain ETSI"
cloned_interfaces="${cloned_interfaces} lo1"
ifconfig_lo1_aliases="inet 10.241.0.0-15/16"
ip6addrctl_policy="ipv4_prefer"
# netstat -4rn
Internet:
DestinationGatewayFlags Netif Expire
default172.16.2.1 UGS ue0
10.241.0.0 link#4 UH lo1
10.241.0.1 link#4 UH lo1
10.241.0.2 link#4 UH lo1
10.241.0.3 link#4 UH lo1
10.241.0.4 link#4 UH lo1
10.241.0.5 link#4 UH lo1
10.241.0.6 link#4 UH lo1
10.241.0.7 link#4 UH lo1
10.241.0.8 link#4 UH lo1
10.241.0.9 link#4 UH lo1
10.241.0.10link#4 UH lo1
10.241.0.11link#4 UH lo1
10.241.0.12link#4 UH lo1
10.241.0.13link#4 UH lo1
10.241.0.14link#4 UH lo1
10.241.0.15link#4 UH lo1
127.0.0.1 link#1 UH lo0
172.16.2.0/24 link#2 U ue0
172.16.2.92link#3 UHS lo0
172.16.2.93link#2 UHS lo0
# ifconfig
lo0: flags=8049 metric 0 mtu 16384
options=680003
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
inet 127.0.0.1 netmask 0xff00
nd6 options=21
groups: lo
ue0: flags=8843 metric 0 mtu 1500
options=8000b
ether 74:da:38:d8:fe:a7
inet6 fe80::76da:38ff:fed8:fea7%ue0 prefixlen 64 scopeid 0x2
inet 172.16.2.93 netmask 0xff00 broadcast 172.16.2.255
nd6 options=23
media: Ethernet autoselect (1000baseT )
status: active
wlan0: flags=8943 metric 0 mtu
1500
ether 00:28:f8:d0:91:52
inet6 fe80::228:f8ff:fed0:9152%wlan0 prefixlen 64 scopeid 0x3
inet 172.16.2.92 netmask 0xff00 broadcast 172.16.2.255
nd6 options=23
media: IEEE 802.11 Wireless Ethernet OFDM/54Mbps mode 11g
status: associated
ssid skunkwerks channel 6 (2437 MHz 11g) bssid 80:2a:a8:5a:bd:3f
regdomain ETSI country AT authmode WPA2/802.11i privacy ON
deftxkey UNDEF AES-CCM 2:128-bit txpower 30 bmiss 10 scanvalid 60
protmode CTS wme roaming MANUAL
groups: wlan
lo1: flags=8049 metric 0 mtu 16384
options=680003
inet 10.241.0.0 netmask 0x
inet 10.241.0.1 netmask 0x
inet 10.241.0.2 netmask 0x
inet 10.241.0.3 netmask 0x
inet 10.241.0.4 netmask 0x
inet 10.241.0.5 netmask 0x
inet 10.241.0.6 netmask 0x
inet 10.241.0.7 netmask 0x
inet 10.241.0.8 netmask 0x
inet 10.241.0.9 netmask 0x
inet 10.241.0.10 netmask 0x
inet 10.241.0.11 netmask 0x
inet 10.241.0.12 netmask 0x
inet 10.241.0.13 netmask 0x
inet 10.241.0.14 netmask 0x
inet 10.241.0.15 netmask 0x
nd6 options=29
groups: lo
bridge0: flags=8843 metric 0 mtu 1500
description: vm-public
ether 02:63:d4:40:fe:00
nd6 options=1
groups: bridge
id 00:00:00:00:00:00 priority 0 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 0 ifcost 0 port 0
member: wlan0 flags=143
ifmaxaddr 0 port 3 priority 128 path cost 370370
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: 'no route to host" for cloned lo1 iface 12.0-CURRENT r334376+56a973815425(master) amd64
On Wed, 30 May 2018, at 15:40, Dave Cottlehuber wrote:
> I have 10.241.0.0-15/16 bound to lo1 for jails, and I just realised it>
> doesn't work anymore. Boxes at $DAYJOB use the same config and
> work just> fine on 11.1R/amd64, otherwise I'd be fired this used to work
> ~ 1> month ago, both on my desktop, and my laptop, and now doesn't.
ping 10.241.0.0 or any of the 15 IPs says no route to host. trace route
does the same thing. Removing the spurious bridge has no effect either.
my boot environment from r333243 can ping though, netstat and rc.conf
are identical between runs.
Sorry I wrote this on a phone and copy paste failed.
> Have I mis-configured something blindingly obvious here before
> I embark> on a lengthy bisect?
>
> # grep -v \# /etc/rc.conf , elided
> hostname=akai
> kld_list="${kld_list} if_iwm if_axge if_ure if_ue"
> ifconfig_ue0=DHCP
> ifconfig_ue0_ipv6="inet6 accept_rtadv"
> wlans_iwm0="wlan0"
> ifconfig_wlan0="WPA DHCP"
> ifconfig_wlan0_ipv6="inet6 accept_rtadv"
> create_args_wlan0="country AT regdomain ETSI"
> cloned_interfaces="${cloned_interfaces} lo1"
> ifconfig_lo1_aliases="inet 10.241.0.0-15/16"
> ip6addrctl_policy="ipv4_prefer"
>
> # netstat -4rn
> Internet:
> DestinationGatewayFlags Netif Expire
> default172.16.2.1 UGS ue0
> 10.241.0.0 link#4 UH lo1
> 10.241.0.1 link#4 UH lo1
> 10.241.0.2 link#4 UH lo1
> 10.241.0.3 link#4 UH lo1
> 10.241.0.4 link#4 UH lo1
> 10.241.0.5 link#4 UH lo1
> 10.241.0.6 link#4 UH lo1
> 10.241.0.7 link#4 UH lo1
> 10.241.0.8 link#4 UH lo1
> 10.241.0.9 link#4 UH lo1
> 10.241.0.10link#4 UH lo1
> 10.241.0.11link#4 UH lo1
> 10.241.0.12link#4 UH lo1
> 10.241.0.13link#4 UH lo1
> 10.241.0.14link#4 UH lo1
> 10.241.0.15link#4 UH lo1
> 127.0.0.1 link#1 UH lo0
> 172.16.2.0/24 link#2 U ue0
> 172.16.2.92link#3 UHS lo0
> 172.16.2.93link#2 UHS lo0
>
> # ifconfig
> lo0: flags=8049 metric 0 mtu 16384
> options=680003
> inet6 ::1 prefixlen 128
> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
> inet 127.0.0.1 netmask 0xff00
> nd6 options=21
> groups: lo
> ue0: flags=8843 metric 0 mtu
> 1500
> options=8000b
> ether 74:da:38:d8:fe:a7
> inet6 fe80::76da:38ff:fed8:fea7%ue0 prefixlen 64 scopeid 0x2
> inet 172.16.2.93 netmask 0xff00 broadcast 172.16.2.255
> nd6 options=23
> media: Ethernet autoselect (1000baseT )
> status: active
> wlan0: flags=8943
> metric> 0 mtu 1500
> ether 00:28:f8:d0:91:52
> inet6 fe80::228:f8ff:fed0:9152%wlan0 prefixlen 64 scopeid 0x3
> inet 172.16.2.92 netmask 0xff00 broadcast 172.16.2.255
> nd6 options=23
> media: IEEE 802.11 Wireless Ethernet OFDM/54Mbps mode 11g
> status: associated
> ssid skunkwerks channel 6 (2437 MHz 11g) bssid 80:2a:a8:5a:bd:3f
> regdomain ETSI country AT authmode WPA2/802.11i privacy ON
> deftxkey UNDEF AES-CCM 2:128-bit txpower 30 bmiss 10 scanvalid 60
> protmode CTS wme roaming MANUAL
> groups: wlan
> lo1: flags=8049 metric 0 mtu 16384
> options=680003
> inet 10.241.0.0 netmask 0x
> inet 10.241.0.1 netmask 0x
> inet 10.241.0.2 netmask 0x
> inet 10.241.0.3 netmask 0x
> inet 10.241.0.4 netmask 0x
> inet 10.241.0.5 netmask 0x
> inet 10.241.0.6 netmask 0x
> inet 10.241.0.7 netmask 0x
> inet 10.241.0.8 netmask 0x
> inet 10.241.0.9 netmask 0x
> inet 10.241.0.10 netmask 0x
> inet 10.241.0.11 netmask 0x
> inet 10.241.0.12 netmask 0x
> inet 10.241.0.13 netmask 0x
> inet 10.241.0.14 netmask 0x
> inet 10.241.0.15 netmask 0x
> nd6 options=29
> groups: lo
> bridge0: flags=8843
> metric 0 mtu> 1500
> description: vm-public
> ether 02:63:d4:40:fe:00
> nd6 options=1
> groups: bridge
> id 00:00:00:00:00:00 priority 0 hellotime 2 fwddelay 15
> maxage 20 holdcnt 6 proto rstp maxaddr 20
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: 'no route to host" for cloned lo1 iface 12.0-CURRENT r334376+56a973815425(master) amd64
On Wed, 30 May 2018, at 17:46, Rodney W. Grimes wrote:
> > > > > > ifconfig_lo1_aliases="inet 10.241.0.0-15/16"
> > > > > > lo1: flags=8049 metric 0 mtu 16384
> > > > > > inet 10.241.0.0 netmask 0x
> > > > > > inet 10.241.0.1 netmask 0x
> > > > > > inet 10.241.0.2 netmask 0x
Thanks Rod, Andreas, Herbert for your help! Back at a proper computer now.
I think there are 2 things; invalid IP (see end for some interesting notes),
and also expansion of ifconfig__aliases.
# ifconfig_$(if)_aliases
This is my config:
> cloned_interfaces="lo1"
> ifconfig_lo1_aliases="inet 10.24 1.0.0-15/16"
But, I *don't* have a line like this:
> ifconfig_lo1="inet 10.241.0.0/16"
and if I add it and bump the range to 10.241.0.1/16, then all is well again
and ping $DODGY_IP works again, but I get 2 entries with /16 mask:
inet 10.241.0.0 netmask 0x
inet 10.241.0.1 netmask 0x
inet 10.241.0.2 netmask 0x
So the solution seems to be this, to keep the 0xfff to just 1 IP:
> cloned_interfaces="lo1"
> ifconfig_lo1="inet 10.241.0.0/16"
> ifconfig_lo1_aliases="inet 10.24 1.0.0-15/16"
Presumably I've copy-pasted this a long time ago and never questioned it. I
checked several random websites, and there are quite a few skipping
`ifconfig_lo1`, using just the aliases, and mainly with jail configs, so I
guess this change will catch other people too.
I'm not sure what's changed, as nothing recent in /etc/rc.d or
/etc/network.subr commits seems related. What's the best option here?
Just a doc patch saying you can't use aliases without a prior ifconfig_ ?
# invalid IP
TLDR 10.241.0.0/16 is technically not a valid host IP but it has obviously
worked in the past.
I've been binding 10.241.0.1-15 to jail IPs, and abusing 10.241.0.0 as the
"magic ip" that is bound to net/haproxy or spiped in the host system to broker
exernal connections into the jail IP ranges from external internet. I will
rectify my configuration but I will miss the symmetry :-)
https://tools.ietf.org/html/rfc1122#section-3.3.6 is the closest description I
could find for this. Interestingly, they blame 4.2BSD for this and say it's
addressed since 4.3:
## 3.3.6 Broadcasts
Section 3.2.1.3 defined the four standard IP broadcast address
forms:
Limited Broadcast: {-1, -1}
Directed Broadcast: {,-1}
Subnet Directed Broadcast:
{,,-1}
All-Subnets Directed Broadcast: {,-1,-1}
A host MUST recognize any of these forms in the destination
address of an incoming datagram.
There is a class of hosts* that use non-standard broadcast
address forms, substituting 0 for -1. All hosts SHOULD
recognize and accept any of these non-standard broadcast
addresses as the destination address of an incoming datagram.
_
*4.2BSD Unix and its derivatives, but not 4.3BSD.
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Proxy a TCP connection
On Thu, 14 Jun 2018, at 16:01, Andrea Venturoli wrote: > On 05/21/18 18:10, Andrea Venturoli wrote: > > > Thanks to anyone who answered. > > > > I'm currently trying net/bounce, as suggested by Eugene. > > If that won't work properly, I'll sure give plugdaemon a shot. > > Just an update in case anyone is interested... > > Bounce is still dying occasionally net/haproxy is superb for this and logs extensively with a handy admin interface as well. Ask offlist if you’d like some help getting set up. Dave ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
unloading pf causes desktop system to freeze since ~ r335381
[cross-posting for advice on general debugging + network-specific thoughts] TLDR since a week or so, probably around r335381 I can reliably get my machine to hang*** by unloading pf, while there's network traffic (e.g. video streaming or rsync) and waiting a minute or two I still see it with r335576 a few times today. config & logs below, h/w is intel xeon v2667v4 on supermicro X10SRA-F dual igb nics. However each time there's no crashdump, & the usual ctrl-alt-esc does't work either. I'm a bit lost as to what I can do here to capture something useful. A few minutes later, the machine spontaneously reboots, I assume due to a h/w watchdog kicking up, and there is no crashdump info present in /var/crash/ as I'd normally expect. ***hang means simultaneously: - keyboard is unresponsive (capslock/numlock keys don't cause keyboard LEDs to toggle state) - control-alt-esc doesn't work to get to the debugger - music playing via mpd to an external USB DAC stops - network sessions & tap interfaces fail - X session contents freeze but remain visible Is there some way I can get a kernel dump even though the system has fully hung? - dmesg, rc.conf, ifconfig, sysctls etc https://git.io/f4HQZ - supermicro X10SRA-F bios v2.0a settings https://git.io/f4HQb A+ Dave ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: unloading pf causes desktop system to freeze since ~ r335651 [coredump]
On Mon, 25 Jun 2018, at 23:08, Dave Cottlehuber wrote: > [cross-posting for advice on general debugging + network-specific thoughts] The HPET NMI watchdog patch was very timely - works a treat: https://reviews.freebsd.org/D15630 > However each time there's no crashdump, & the usual ctrl-alt-esc does't > work either. I bumped my /usr/src to latest HEAD, applied HPET NMI watchdog hack & after freezing via `service pf stop`, I was rewarded with a coredump on next reboot; full log: https://git.io/f4Q4P [1202] panic: Assertion !in_epoch() && !mtx_owned(&(&(*(__typeof(vnet_entry_tcbinfo)*) (__curthread())->td_vnet))->vnet_data_base) + (uintptr_t)&vnet_entry_tcbinfo)))->ipi_lock) failed at /usr/src/sys/netinet/tcp_input.c:802 [1202] cpuid = 4 [1202] time = 1529997533 [1202] KDB: stack backtrace: [1202] db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfe0077ddc4a0 [1202] vpanic() at vpanic+0x1a3/frame 0xfe0077ddc500 [1202] doadump() at doadump/frame 0xfe0077ddc580 [1202] tcp_input() at tcp_input+0x940/frame 0xfe0077ddc6c0 [1202] ip_input() at ip_input+0x3f7/frame 0xfe0077ddc720 [1202] netisr_dispatch_src() at netisr_dispatch_src+0xa2/frame 0xfe0077ddc780 [1202] ether_demux() at ether_demux+0x16e/frame 0xfe0077ddc7b0 [1202] ether_nh_input() at ether_nh_input+0x402/frame 0xfe0077ddc820 [1202] netisr_dispatch_src() at netisr_dispatch_src+0xa2/frame 0xfe0077ddc880 [1202] ether_input() at ether_input+0x8f/frame 0xfe0077ddc8c0 [1202] iflib_rxeof() at iflib_rxeof+0xcce/frame 0xfe0077ddc9b0 [1202] _task_fn_rx() at _task_fn_rx+0x7f/frame 0xfe0077ddc9f0 [1202] gtaskqueue_run_locked() at gtaskqueue_run_locked+0x139/frame 0xfe0077ddca40 [1202] gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x88/frame 0xfe0077ddca70 [1202] fork_exit() at fork_exit+0x84/frame 0xfe0077ddcab0 [1202] fork_trampoline() at fork_trampoline+0xe/frame 0xfe0077ddcab0 [1202] --- trap 0, rip = 0, rsp = 0, rbp = 0 --- [1202] KDB: enter: panic db:0:kdb.enter.panic> run lockinfo db:1:lockinfo> show locks db:1:lockinfo> show alllocks Process 12789 (h2o) thread 0xf8020dd4f580 (101673) Process 17635 (pflogd) thread 0xf8017c91c580 (101027) db:1:lockinfo> show lockedvnods Locked vnodes db:0:kdb.enter.panic> show pcpu cpuid= 4 dynamic pcpu = 0xfe00848dd8c0 curthread= 0xf801067f2000: pid 0 tid 100029 "if_io_tqg_4" curpcb = 0xfe0077ddcb80 fpcurthread = none idlethread = 0xf80106796000: tid 17 "idle: cpu4" curpmap = 0x81ffbe08 tssp = 0x82066ac0 commontssp = 0x82066ac0 rsp0 = 0xfe0077ddcb80 gs32p= 0x8206d6f8 ldt = 0x8206d738 tss = 0x8206d728 curvnet = 0xf801000ca0c0 spin locks held: db:0:kdb.enter.panic> bt Tracing pid 0 tid 100029 td 0xf801067f2000 kdb_enter() at kdb_enter+0x3b/frame 0xfe0077ddc4a0 vpanic() at vpanic+0x1c0/frame 0xfe0077ddc500 doadump() at doadump/frame 0xfe0077ddc580 tcp_input() at tcp_input+0x940/frame 0xfe0077ddc6c0 ip_input() at ip_input+0x3f7/frame 0xfe0077ddc720 netisr_dispatch_src() at netisr_dispatch_src+0xa2/frame 0xfe0077ddc780 ether_demux() at ether_demux+0x16e/frame 0xfe0077ddc7b0 ether_nh_input() at ether_nh_input+0x402/frame 0xfe0077ddc820 netisr_dispatch_src() at netisr_dispatch_src+0xa2/frame 0xfe0077ddc880 ether_input() at ether_input+0x8f/frame 0xfe0077ddc8c0 iflib_rxeof() at iflib_rxeof+0xcce/frame 0xfe0077ddc9b0 _task_fn_rx() at _task_fn_rx+0x7f/frame 0xfe0077ddc9f0 gtaskqueue_run_locked() at gtaskqueue_run_locked+0x139/frame 0xfe0077ddca40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x88/frame 0xfe0077ddca70 fork_exit() at fork_exit+0x84/frame 0xfe0077ddcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfe0077ddcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Please keep replies just to freebsd-net@ from here on. A+ Dave ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: 'no route to host" for cloned lo1 iface 12.0-CURRENT r334376+56a973815425(master) amd64
> > > > # invalid IP
> > > >
> > > > TLDR 10.241.0.0/16 is technically not a valid host IP but it has
> > > obviously worked in the past.
Just following this up, kmacy@'s recent commits resolved this - thanks!
Finally, I came up with this, as a way of keeping the "symmetry" of the
0.0 and still if I read the RFC correctly, have a valid host IP:
# jail networks
cloned_interfaces="${cloned_interfaces} lo1"
ifconfig_lo1="inet 10.241.0.0/15" --- should be a host mask x.1.0.0
ifconfig_lo1_aliases="inet 10.241.0.1-15/32"
# ifconfig lo1
lo1: flags=8049 metric 0 mtu 16384
options=680003
inet6 ::1 prefixlen 128 tentative
inet 10.241.0.0 netmask 0xfffe
inet 10.241.0.1 netmask 0x
inet 10.241.0.2 netmask 0x
inet 10.241.0.3 netmask 0x
inet 10.241.0.4 netmask 0x
inet 10.241.0.5 netmask 0x
inet 10.241.0.6 netmask 0x
...
thanks everybody for the patches and the insight.
FWIW the alternative (original breaking config) also works again:
# jail networks
cloned_interfaces="${cloned_interfaces} lo1"
ifconfig_lo1_aliases="inet 10.241.0.1-15/15"
A+
Dave
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: unloading pf causes desktop system to freeze since ~ r335651 [solved]
On Tue, 26 Jun 2018, at 09:52, Dave Cottlehuber wrote: > > The HPET NMI watchdog patch was very timely - works a treat: > https://reviews.freebsd.org/D15630 > > > However each time there's no crashdump, & the usual ctrl-alt-esc does't > > work either. > > I bumped my /usr/src to latest HEAD, applied HPET NMI watchdog hack & > after freezing via `service pf stop`, I was rewarded with a coredump > on next reboot; full log: https://git.io/f4Q4P > > [1202] panic: Assertion !in_epoch() && ! > mtx_owned(&(&(*(__typeof(vnet_entry_tcbinfo)*) (__curthread())- > >td_vnet))->vnet_data_base) + (uintptr_t)&vnet_entry_tcbinfo)))- > >ipi_lock) failed at /usr/src/sys/netinet/tcp_input.c:802 I narrowed it down to setting this sysctl: net.inet.ip.process_options=0 # ignore IP options in the incoming packets (default 1) and it looks like it was fixed here: r335749 thanks glebius@ https://svnweb.freebsd.org/base/head/sys/netinet/tcp_subr.c?r1=335749&r2=335748&pathrev=335749 I've not had a repeat since building off this. A+ Dave ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Working around unsupported Ethernet card with PXE or UEFI?
On Fri, 21 Jun 2019, at 04:33, Thomas Mueller wrote: > Is it possible to build and install FreeBSD so as to be bootable and access > the internet with an Ethernet card that doesn't work in FreeBSD? > > Could Ethernet support be obtained through UEFI? > > Or could FreeBSD be installed on another computer or on a NAS NFS share? NAS > would probably not have TFTP but would have NFS and iSCSI capability. > > Motherboard has PXE capability. Server OS for FreeBSD installation on other > computer could be NetBSD, where the Ethernet (Realtek 8111E or 8168, re(4) > driver) works well. > > > Tom Hi Tom https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/config-network-setup.html NDIS may do what you want but it may suck. In terms of time vs effort a 15$ USB Ethernet dongle is going to be your best result. If this is a server consider getting a 2nd hand intel NIC. I can get a 4 port pci intel Gb for under 100$ years which is more than sufficient. Lower spec cards will be even cheaper. There are references on the forums and mailing lists with ones that work for both USB and iNICs. Dave ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Working around unsupported Ethernet card with PXE or UEFI?
On Tue, 25 Jun 2019, at 01:05, Thomas Mueller wrote: > > Hi Tom > > https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/config-network-setup.html > > > NDIS may do what you want but it may suck. > > > In terms of time vs effort a 15$ USB Ethernet dongle is going to be your > > best result. If this is a server consider getting a 2nd hand intel NIC. I > > can get a 4 port pci intel Gb for under 100$ years which is more than > > sufficient. > > Lower spec cards will be even cheaper. > > > There are references on the forums and mailing lists with ones that work > > for both USB and iNICs. > > > Dave > > NDIS is not dependable in my experience, and I have had instances of > the kernel failing to compile with NDIS. Not surprised to hear that. > How dependable are USB Ethernet dongles to work with FreeBSD? I got this one which I use for my laptop when normal FreeBSD wifi isn't sufficient "works for me": EU4306: https://www.edimax.com/edimax/merchandise/merchandise_detail/data/edimax/global/network_adapters_usb_adapters/eu-4306/ axge(4): https://www.freebsd.org/cgi/man.cgi?query=axge It's sufficient for doing make install over NFS and sending/receiving full zpool dumps, I can't speak for performance nor cpu usage but then you get what you pay for :-). I'm reasonably sure that you can't expect server-level performance out of this, and an intel PCI nic(like i350 or i210) is surely a better bet. A+ Dave ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
ngrep/tcpdump and cloned interfaces
hi,
I posted this a week ago to freebsd-questions but got nothing. I hope
its
suitable for asking here.
Today I wanted to observe traffic that is proxied via haproxy between IP
addresses both bound to a lo1 cloned interface.
To my surprise ngrep & tcpdump showed no activity on lo1, but it did
show the expected traffic on lo0. Now I'm not even sure I understand
what a cloned interface is anymore.
Why does this traffic appear on the other interface at all?
Most importantly, does a jail with a lo1-bound IP address have any
ability outside firewall rules to receive or view traffic using a
lo0-bound IP in a different subnet?
# ngrep -texd lo0 port 1978
T 2017/03/29 19:45:17.838356 10.241.0.3:48176 -> 10.241.0.3:1978 [AP]
50 4f 53 54 20 2f 72 7063 2f 73 65 74 20 48 54POST /rpc/set HT
54 50 2f 31 2e 31 0d 0a55 73 65 72 2d 41 67 65TP/1.1..User-Age
6e 74 3a 20 46 75 72 6c3a 3a 48 54 54 50 2f 33nt: Furl::HTTP/3
2e 30 39 0d 0a 43 6f 6e74 65 6e 74 2d 54 79 70.09..Content-Typ
65 3a 20 74 65 78 74 2f74 61 62 2d 73 65 70 61e: text/tab-sepa
# sockstat -46l
# sockstat -46l |grep 1978
www haproxy36440 8 tcp4 10.241.0.0:1978 *:*
kyototycoon ktserver73187 6 tcp4 10.241.0.3:1978 *:*
# ifconfig snippets
lo0: flags=8049 metric 0 mtu 16384
options=63
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
inet 127.0.0.1 netmask 0xff00
nd6 options=21
groups: lo
lo1: flags=8049 metric 0 mtu 16384
options=63
inet 10.241.0.0 netmask 0x
inet 10.241.0.3 netmask 0x
inet 10.241.0.2 netmask 0x
inet 10.241.0.1 netmask 0x
inet 10.241.0.5 netmask 0x
inet 10.241.0.4 netmask 0x
nd6 options=29
groups: lo
# /etc/pf.conf snippet
protocols = "{ tcp, udp, icmp }"
extl_if="lagg0"
jail_if="lo1"
jail_net = $jail_if:network
nat on $extl_if proto $protocols from $jail_net to any -> ($extl_if)
A+
Dave
___
freebsd-questi...@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to
"freebsd-questions-unsubscr...@freebsd.org"
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
if_tap limits in 12.1R amd64?
I use a port, net/zerotier, that uses if_tap(4) to provide a mesh VPN. If I add many IPv6 aliases to its /80 routable IPv6 interface, sometime around adding ~ 1100 or more aliases, the port dies. If I'm really lucky, system panics. Now that I'm watching it, no such luck Am I running into some limitations or thresholds for if_tap? lo has no trouble soaking up this number of aliases. thanks Dave ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Problem with net.inet.tcp.path_mtu_discovery=1
On Wed, 4 Jun 2025, at 16:36, Christos Chatzaras wrote: > Hello, > > I manage some servers hosting websites. What does tcpdump/wireshark show for traffic, particularly icmp? Wireshark is very helpful in explaining some issues. What is the actual MTU on the working net vs the failing one? Is there a local MTU where the failing websites start working again? see ping(8) and use -v -D -s …. together to find a working MTU and cross check with tcpdump to find where things seem to break. On a recent cloud environment I needed to add ‘ set reassemble yes no-df’ to my pf.conf to address MTU issues between VNET jails and the internet. Happy hunting Dave
Re: SSH connection problem to two FreeBSD VMs externaly hosted
On Thu, 17 Jul 2025, at 11:21, Gordon Bergling wrote:
> Hi,
>
> I have two FreeBSD externaly hosted, one at Hetzner and one on Azure.
>
> Both systems running latest 14.3-RELEASE, but I can't no long connect to them,
> wether from a local 14.2-RELEASE, or the latest macOS. Nothing has changed in
> terms of configuration. All systems use public-key authentication. The error
> I am getting is the following:
>
> sshd[10965]: error: Fssh_kex_input_kexinit: unknown kex type 10 [preauth]
>
> Has anyone an idea whould could cause this?
>
> Seeking out in forums about trying different KexAlgorithms options didn't
> solved the problem.
>
> Any help is much appreciated!
>
> --Gordon
>
> Attachments:
> * signature.asc
Odd. I have no issue from a 14.2 client -> 14.3 server connecting,
with defaults, and ed25519 private key.
My best guess is that your sshd binary (or config) isn't correctly
upgraded for some reason. What does file(1) report on server & client?
On 14.2-RELEASE:
root@picard:/ # file /usr/sbin/sshd
/usr/sbin/sshd: ELF 64-bit LSB pie executable, x86-64, version 1 (FreeBSD),
dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 14.2,
FreeBSD-style, stripped
root@picard:/ # file /usr/bin/ssh
/usr/bin/ssh: ELF 64-bit LSB pie executable, x86-64, version 1 (FreeBSD),
dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 14.2,
FreeBSD-style, stripped
root@picard:/ #
If this is not correct, it's worth checking with `freebsd-update IDS` on server
& client, for what else is incorrect.
Are there any non-default settings in /etc/ssh/ssh_config for client,
and /etc/ssh/sshd_config for server?
Assuming that's sorted, please post output of `ssh -vv ...`, so we can see the
negotiation, forcing key exchange algorithm on the client:
ssh -vv -o
KexAlgorithms=curve25519-sha256,curve25519-sha...@libssh.org,diffie-hellman-group-exchange-sha256
you@there
BTW I assume the kex list comes from crypto/openssh/kex.h, so #10 would be
KEX_KEM_SNTRUP761X25519_SHA512
enum kex_exchange {
KEX_DH_GRP1_SHA1 = 1,
KEX_DH_GRP14_SHA1,
KEX_DH_GRP14_SHA256,
KEX_DH_GRP16_SHA512,
KEX_DH_GRP18_SHA512,
KEX_DH_GEX_SHA1,
KEX_DH_GEX_SHA256,
KEX_ECDH_SHA2,
KEX_C25519_SHA256,
KEX_KEM_SNTRUP761X25519_SHA512, <
KEX_KEM_MLKEM768X25519_SHA256,
KEX_MAX
};
A+
Dave
