Hello,
I'm using, FreeBSD stable/12-n232956-5fb6cfadcd9d-dirty: Mon Apr 12
09:03:59 +03 2021 .
I'm trying to run iperf between two hosts that each one has Chelsio
T62100-LP-CR card. There two host is connected with AOC cable. ifconfig
output of one host is below:
cc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=62ec07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6,HWRXTSTMP,VXLAN_HWCSUM,VXLAN_HWTSO>
ether 00:07:43:3e:5e:80
inet6 fe80::207:43ff:fe3e:5e80%cc0 prefixlen 64 scopeid 0xa
inet 192.168.0.70 netmask 0xffffff00 broadcast 192.168.0.255
groups: lanlar
media: Ethernet 100GBase-SR4 <full-duplex,rxpause,txpause>
status: active
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
On the host A running :
iperf -s
On the host B running :
iperf -c IP_OF_HOST_A -P 8
iperf version = iperf-2.1.0.r2
When I rerun iperf -c on host B, host A throws these line to dmesg:
t6nex0: ! PL_PERR_CAUSE 0x19404 = 0x04000000, E 0x1fff63ff, F 0xffffffff
t6nex0: ! [0x04000000] SGE
t6nex0: * PL_INT_CAUSE 0x1940c = 0x04000040, E 0x1fff677d, F 0x00000000
t6nex0: * [0x04000000] SGE
t6nex0: * [0x00000040] PL
t6nex0: ! SGE_INT_CAUSE5 0x110c = 0x00001000, E 0x00ffffff, F 0x00ffffff
t6nex0: ! PL_PL_INT_CAUSE 0x19430 = 0x00000014, E 0x00000010, F 0x00000010
t6nex0: ! [0x00000010] Fatal parity error
t6nex0: ? [0x00000004]
t6nex0: encountered fatal error, adapter stopped.
----------------------
I tried to run "ifconfig cc0 down; ifconfig cc0 up". and then there lines
throwed to dmesg:
cc0: link state changed to DOWN
t6nex0: command 0x16 in mbox 4 timed out (0x4014c010).
t6nex0: mbox 4 cmdsent 16a0094400000001 05dc050000000000 0000000000000000
0000000000000000 0000000000000000 0000000000000000 0000000000000000
0000000000000000
t6nex0: mbox 4 current 16a0094400000001 05dc050000000000 0000000000000000
0000000000000000 0000000000000000 0000000000000000 0000000000000000
0000000000000000
t6nex0: device log follows.
Seq# Tstamp Level Facility Message
1949 46807821058 CRIT HW i2c transaction failed to
complete
1950 46807821058 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
1951 46807941058 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
1952 46807941059 CRIT HW i2c transaction failed to
complete
1953 46807941059 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
1954 46808061059 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6246, i2cm_op 0x1 failed with err -71
1955 46808061059 CRIT HW i2c transaction failed to
complete
1956 46808061060 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
1957 46808181059 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
1958 46808181060 CRIT HW i2c transaction failed to
complete
1959 46808181060 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
1960 46808301059 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
1961 46808301060 CRIT HW i2c transaction failed to
complete
1962 46808301061 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
1963 46808421059 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6246, i2cm_op 0x1 failed with err -71
1964 46808421059 CRIT HW i2c transaction failed to
complete
1965 46808421060 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
1966 46808514573 INFO PORT port[1] Link not up, try
FEC 0x800000
1967 46808514574 INFO PORT port[1] speed update: 0x40
1968 46808524385 INFO PORT hw_mac_init_port[1], ptype
0x11, speed 0x40, lanes 0xf0, fec 0x800000
1969 46808551626 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
16811, i2cm_op 0x1 failed with err -71
1970 46808551627 CRIT HW i2c transaction failed to
complete
1971 46808551627 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
1972 46808671061 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6246, i2cm_op 0x1 failed with err -71
1973 46808671062 CRIT HW i2c transaction failed to
complete
1974 46808671062 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
1975 46808791061 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
1976 46808791062 CRIT HW i2c transaction failed to
complete
1977 46808791062 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
1978 46808911061 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
1979 46808911062 CRIT HW i2c transaction failed to
complete
1980 46808911062 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
1981 46809031062 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6246, i2cm_op 0x1 failed with err -71
1982 46809031062 CRIT HW i2c transaction failed to
complete
1983 46809031063 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
1984 46809151062 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
1985 46809151063 CRIT HW i2c transaction failed to
complete
1986 46809151063 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
1987 46809271063 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
1988 46809271063 CRIT HW i2c transaction failed to
complete
1989 46809271064 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
1990 46809391062 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6246, i2cm_op 0x1 failed with err -71
1991 46809391062 CRIT HW i2c transaction failed to
complete
1992 46809391063 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
1993 46809511063 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
1994 46809511064 CRIT HW i2c transaction failed to
complete
1995 46809511064 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
1996 46809631064 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
1997 46809631064 CRIT HW i2c transaction failed to
complete
1998 46809631065 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
1999 46809751064 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
2000 46809751065 CRIT HW i2c transaction failed to
complete
2001 46809751065 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
2002 46809871064 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
2003 46809871065 CRIT HW i2c transaction failed to
complete
2004 46809871065 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
2005 46809991064 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6246, i2cm_op 0x1 failed with err -71
2006 46809991064 CRIT HW i2c transaction failed to
complete
2007 46809991065 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
2008 46810111065 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
2009 46810111066 CRIT HW i2c transaction failed to
complete
2010 46810111066 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
2011 46810231066 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
2012 46810231066 CRIT HW i2c transaction failed to
complete
2013 46810231067 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
2014 46810351065 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6246, i2cm_op 0x1 failed with err -71
2015 46810351066 CRIT HW i2c transaction failed to
complete
2016 46810351066 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
2017 46810471066 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
2018 46810471067 CRIT HW i2c transaction failed to
complete
2019 46810471067 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
2020 46810591067 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6246, i2cm_op 0x1 failed with err -71
2021 46810591067 CRIT HW i2c transaction failed to
complete
2022 46810591068 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
2023 46810711067 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
2024 46810711068 CRIT HW i2c transaction failed to
complete
2025 46810711068 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
2026 46810831067 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
2027 46810831068 CRIT HW i2c transaction failed to
complete
2028 46810831069 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
2029 46810951067 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6246, i2cm_op 0x1 failed with err -71
2030 46810951067 CRIT HW i2c transaction failed to
complete
2031 46810951068 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
2032 46811071068 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
2033 46811071069 CRIT HW i2c transaction failed to
complete
2034 46811071069 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
2035 46811191069 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
2036 46811191069 CRIT HW i2c transaction failed to
complete
2037 46811191070 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
2038 46811311069 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
2039 46811311070 CRIT HW i2c transaction failed to
complete
2040 46811311070 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
2041 46811431069 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
2042 46811431070 CRIT HW i2c transaction failed to
complete
2043 46811431070 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
2044 46811551070 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6246, i2cm_op 0x1 failed with err -71
2045 46811551070 CRIT HW i2c transaction failed to
complete
2046 46811551071 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
2047 46811671070 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
2048 46811671071 CRIT HW i2c transaction failed to
complete
2049 46811671071 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
2050 46811791071 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
2051 46811791071 CRIT HW i2c transaction failed to
complete
2052 46811791072 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
2053 46811911071 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
2054 46811911071 CRIT HW i2c transaction failed to
complete
2055 46811911072 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
2056 46812031071 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
2057 46812031072 CRIT HW i2c transaction failed to
complete
2058 46812031072 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
2059 46812151072 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
2060 46812151072 CRIT HW i2c transaction failed to
complete
2061 46812151073 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
2062 46812271072 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
2063 46812271073 CRIT HW i2c transaction failed to
complete
2064 46812271073 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
2065 46812391072 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
2066 46812391073 CRIT HW i2c transaction failed to
complete
2067 46812391073 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
2068 46812511073 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6246, i2cm_op 0x1 failed with err -71
2069 46812511073 CRIT HW i2c transaction failed to
complete
2070 46812511074 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
2071 46812631073 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
2072 46812631074 CRIT HW i2c transaction failed to
complete
2073 46812631074 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
2074 46812751074 INFO HW hw_i2c_transaction:
Address WRITE operation ndata 1 addr_op 0xa0 reg 0x56 data[0] 0x0 diff
6245, i2cm_op 0x1 failed with err -71
2075 46812751074 CRIT HW i2c transaction failed to
complete
2076 46812751075 ERR PORT
port_link_state_handler[0]: Something went terribly wrong. ret = -5
t6nex0: encountered fatal error, adapter stopped.
cc0: set_rxmode (ffff) failed: 60
cc0: disable_vi failed: 60
----------------------
And cc0 stops working, after reboot it starts work. The problem is
reproducable.
# pciconf -lv
t6iov0@pci0:59:0:0: class=0x020000 card=0x00001425 chip=0x60071425
rev=0x00 hdr=0x00
vendor = 'Chelsio Communications Inc'
device = 'T62100-LP-CR Unified Wire Ethernet Controller'
class = network
subclass = ethernet
t6iov1@pci0:59:0:1: class=0x020000 card=0x00001425 chip=0x60071425
rev=0x00 hdr=0x00
vendor = 'Chelsio Communications Inc'
device = 'T62100-LP-CR Unified Wire Ethernet Controller'
class = network
subclass = ethernet
t6iov2@pci0:59:0:2: class=0x020000 card=0x00001425 chip=0x60071425
rev=0x00 hdr=0x00
vendor = 'Chelsio Communications Inc'
device = 'T62100-LP-CR Unified Wire Ethernet Controller'
class = network
subclass = ethernet
t6iov3@pci0:59:0:3: class=0x020000 card=0x00001425 chip=0x60071425
rev=0x00 hdr=0x00
vendor = 'Chelsio Communications Inc'
device = 'T62100-LP-CR Unified Wire Ethernet Controller'
class = network
subclass = ethernet
t6nex0@pci0:59:0:4: class=0x020000 card=0x00001425 chip=0x64071425
rev=0x00 hdr=0x00
vendor = 'Chelsio Communications Inc'
device = 'T62100-LP-CR Unified Wire Ethernet Controller'
class = network
subclass = ethernet
none110@pci0:59:0:5: class=0x010000 card=0x00001425 chip=0x65071425
rev=0x00 hdr=0x00
vendor = 'Chelsio Communications Inc'
device = 'T62100-LP-CR Unified Wire Storage Controller'
class = mass storage
subclass = SCSI
none111@pci0:59:0:6: class=0x0c0400 card=0x00001425 chip=0x66071425
rev=0x00 hdr=0x00
vendor = 'Chelsio Communications Inc'
device = 'T62100-LP-CR Unified Wire Storage Controller'
class = serial bus
subclass = Fibre Channel
Regards
Ozkan
_______________________________________________
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
