On 16.4.2024. 20:22, Szél Gábor wrote: > Dear @misc! > > We have several more complex networks where openbsd is the router. > > Structure of the network: > > * OpenBSD redundant routers > - two OpenBSD > - CARP > - pfsync > - LACP trunks for LAN (2x 10Gbit) (1 side switch #1, 2 side switch > #2 + VPC ) use OpenBSD aggr device > * Cisco Nexus 3K switch-es > - VPC (2x40Gbit) > - redundant LACP links (1 side switch #1, 2 side switch #2 + VPC ) > * many VLANs > * PF default block all trafic, and allowed traffic only > * the servers connected usually 2x10Gbit LACP > > *hardware:* > > * we updated this system in one place to OpenBSD 7.4 > hardware: Dell PE 640 (2x Xeon Gold 6134 CPU, 64Gb RAM, Intel X710 > network cards) > * we migrated the settings from the previous system (OpenBSD 7.0) > the previous hardware was different! (2x Xeon E5-2650, 64Gb RAM, > Intel X520 network cards) > > *Problem:* > > After upgrade with hardware change, we have very poor network performance!! > Example: A simple veeam backup restore that goes through the openbsd > router hangs the network completely (very big lag) > In this case, the SSH connection on the router is have lag! > But OpenBSD dont have high CPU usage. > > If i make simple iperf speed test from OpenBSD to other server (all > device have 10Gbit LACP link): > > [ ID] Interval Transfer Bitrate > [ 5] 0.00-1.00 sec 171 MBytes 1.44 Gbits/sec > [ 5] 1.00-2.00 sec 313 MBytes 2.63 Gbits/sec > [ 5] 2.00-3.00 sec 398 MBytes 3.34 Gbits/sec > [ 5] 3.00-4.00 sec 384 MBytes 3.22 Gbits/sec > [ 5] 4.00-5.00 sec 419 MBytes 3.51 Gbits/sec > [ 5] 5.00-6.00 sec 376 MBytes 3.16 Gbits/sec > [ 5] 6.00-7.00 sec 325 MBytes 2.73 Gbits/sec > [ 5] 7.00-8.00 sec 337 MBytes 2.82 Gbits/sec > [ 5] 8.00-9.00 sec 339 MBytes 2.85 Gbits/sec > [ 5] 9.00-10.00 sec 332 MBytes 2.78 Gbits/sec > [ 5] 10.00-10.19 sec 62.5 MBytes 2.75 Gbits/sec > > Between other devices, servers, etc ... , the speed is perfectly fine > (stable 9-10 Gbits/sec) > Only routed performace is very-very slow. > > if I make a speed test between two OpenBSDs (master router, backup router) > Better value but not perfect: > > [ ID] Interval Transfer Bitrate > [ 5] 0.00-1.00 sec 740 MBytes 6.20 Gbits/sec > [ 5] 1.00-2.00 sec 781 MBytes 6.55 Gbits/sec > [ 5] 2.00-3.00 sec 784 MBytes 6.58 Gbits/sec > [ 5] 3.00-4.00 sec 783 MBytes 6.57 Gbits/sec > [ 5] 4.00-5.00 sec 786 MBytes 6.59 Gbits/sec > [ 5] 5.00-6.00 sec 796 MBytes 6.68 Gbits/sec > [ 5] 6.00-7.00 sec 779 MBytes 6.54 Gbits/sec > [ 5] 7.00-8.00 sec 774 MBytes 6.49 Gbits/sec > [ 5] 8.00-9.00 sec 780 MBytes 6.55 Gbits/sec > [ 5] 9.00-10.00 sec 786 MBytes 6.59 Gbits/sec > [ 5] 10.00-10.00 sec 640 KBytes 10.2 Gbits/sec > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate > [ 5] 0.00-10.00 sec 7.61 GBytes 6.54 Gbits/sec > receiver > > PF have ~2000 rules, but > If i disabled PF on tested OpenBSD router, nothing changes. > > we've run out of ideas, what would be worth watching?
Hi, can you upgrade firewalls to 7.5 and change ixl cards with ix or mcx if you have ? Do you maybe have ipsec tunnels on firewalls ? If you have can disable them for test?
