I dont know if Net is the correct place to ask about Infiniband or not. Machine is FreeBSD 14.0-RELEASE-p5.
The problem we are seeing is our IB MTU is getting set to 2044. It is not getting the MTU setting from the subnet manager like our Linux nodes. It should be getting set to 4092. ib0: flags=1008043<UP,BROADCAST,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 2044 options=8009b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LINKSTATE> lladdr 0.0.0.7a.fe.80.0.0.0.0.0.0.10.70.fd.3.0.e5.c6.c8 inet 10.149.200.68 netmask 0xffff0000 broadcast 10.149.255.255 nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> We vaguely remember an obscure Solaris setting to turn "something" off that solves the issue but can't find anything for FreeBSD and the Solaris cluster nodes are a decade gone at least. Any suggestions? Thanks! -Tom -- -------------------------------------------------------------------- - Tom Arnold - "...is it a virus, a drug, or a religion?" - Sysabend Caretaker - Juanita Shrugs. "What's the difference?" ------------------------ -- Neal Stephenson, Snow Crash