Re: packet loss on ixgbe using vlans and routing
Hi Jack, Have you found anything yet? The box is not in production yet, so I can run test code if you need it. John On Fri, Aug 20, 2010 at 10:39:47AM -0700, Jack Vogel wrote: > OK, am setting up the hardware to look into this John. > > Jack > > > On Fri, Aug 20, 2010 at 7:04 AM, John Hay wrote: > > > Hi Jack, > > > > Have you had a chance to look at it yet? I would love to get these > > network cards working. :-) > > > > John > > > > On Fri, Jul 23, 2010 at 01:36:10AM -0700, Jack Vogel wrote: > > > Yes, I am here, I have been reading this, but I am also very busy with a > > > couple of things, please be patient, I will get on this asap. > > > > > > Cheers, > > > > > > Jack > > > > > > > > > On Fri, Jul 23, 2010 at 12:40 AM, John Hay wrote: > > > > > > > Hi, > > > > > > > > (Jack any chance that you can look at this please?) > > > > > > > > It looks like there are 2 problems with the ixgbe driver on FreeBSD-8. > > > > I have a Dell T710 with 4 X 10G ethernet interfaces (2 X Dual port > > Intel > > > > 82599 cards). It is running FreeBSD RELENG_8. > > > > > > > > 1 - When routing (using vlans) there is heavy packet loss that go away > > > > when you do "ifconfig ix2 -rxcsum". The packet loss seems to be on the > > > > receive side because I do not see them on the receiving interface with > > > > tcpdump. This seems to impact both ipv4 and ipv6. > > > > > > > > My test setup is the Dell T710 with its ix2 connected to a 10G port of > > > > a Nortel 4526GTX. On that port I have 2 vlans configured with half of > > > > the 1G ports in the one vlan and the other half in the other vlan. > > > > > > > > If I test with iperf from one of the machines on a 1G port to the T710, > > > > I get 920Mbit/s. If I do it simultaneously from a few machines > > connected > > > > to the 1G ports, all of them basically saturate their 1G links. > > > > > > > > If I now try to route from the one vlan to the other, ie. doing an > > iperf > > > > from a 1G connected machine, through the T710, to another 1G connected > > > > machine, I see packet loss, sometimes iperf is only able to do > > 100kbits/s. > > > > (Configuring a tcp relay, like socat, on the T710, and working through > > it, > > > > I again get 900Mbit/s and more.) > > > > > > > > So it seems that as long as the T710 with the 10G card is the start or > > > > end point of the connection, I get no packet loss, but as soon as it > > > > has to route, something go wrong. > > > > > > > > 2 - I see packet loss (0 - 40%) on IPv6 packets in vlans, when the > > > > machine is not the originator of the packets. This happen even with > > > > the "ifconfig ix2 -rxcsum". > > > > > > > > Let me try to describe a little more. If a neigbouring machine ping6 > > it, > > > > there will be packet loss. If it act as a router for ipv6, there will > > be > > > > packet loss. This happen even when the network is pretty idle and with > > > > different switches (Nortel and Cisco equipment). The packet loss is > > > > very fluctuating. Pinging 1000 packets might loose 1% one time and the > > > > next time 30%. Looking with tcpdump, I can see the packets arriving and > > > > going out, but the packet never arrive at the next machine. (My feeling > > is > > > > that they get lost inside the card.) The error counters on the switch > > > > does not increment. > > > > > > > > I do not see packet loss if the machine originate the packets, for > > example > > > > ping6 from the machine. Also ipv4 packets do not have any packets loss. > > If > > > > I do not use vlans, I don't see packet loss with ipv6 either. > > > > > > > > The machine also have bce 1G interfaces and I do not see the packet > > loss > > > > on them. > > > > > > > > Here is some info about the machine / setup. The numbers are pretty low > > > > because I rebooted after compiling a kernel with IPFIREWALL, > > ROUTETABLES, > > > > MROUTING and FLOWTABLE removed. I'll add my kernel config file with > > empty > > > > and commented out lines removed. > > > > > > > > pciconf -lvc > > > > i...@pci0:129:0:0: class=0x02 card=0x00038086 chip=0x10fb8086 > > > > rev=0x01 hdr=0x00 > > > >vendor = 'Intel Corporation' > > > >class = network > > > >subclass = ethernet > > > >cap 01[40] = powerspec 3 supports D0 D3 current D0 > > > >cap 05[50] = MSI supports 1 message, 64 bit, vector masks > > > >cap 11[70] = MSI-X supports 64 messages in map 0x20 enabled > > > >cap 10[a0] = PCI-Express 2 endpoint max data 256(512) link x8(x8) > > > > i...@pci0:129:0:1: class=0x02 card=0x00038086 chip=0x10fb8086 > > > > rev=0x01 hdr=0x00 > > > >vendor = 'Intel Corporation' > > > >class = network > > > >subclass = ethernet > > > >cap 01[40] = powerspec 3 supports D0 D3 current D0 > > > >cap 05[50] = MSI supports 1 message, 64 bit, vector masks > > > >cap 11[70] = MSI-X supports 64 messages in map 0x20 enabled > > > >cap 10[a0] = PCI-Express 2 endpoint max data 256
Re: seeking current supported crypto co-processors
On 09/03/10 02:35, Ricky Charlet wrote: Howdy, I'm seeking current cryptographic coprocessors supported in FreeBSD 8.x. By perusing through the crypto-dev (and subsequently referenced) man page(s) I found this list: Hifn 7751/7951/7811/7955/7956 crypto accelerator SafeNet 1141/1741 Bluesteel 5501/5601 Broadcom bcm5801/5802/5805/5820/5821/5822/5823/5825 Those are all pretty old (and in some cases, no longer existent). I'm surveying these lists to see if anyone knows of more modern chips working with FreeBSD 8.x. Or if you feel some chip on the list above is up to the task of near about 1 Gb throughput across a PCIe and has friendly vendor support for FreeBSD, I'd sure like to hear about that too. I'm not saying they are useless but are you really sure you need them? Even on the last generation of CPUs without AES instructions you can easily get 125 MB/s of AES-128 encryption and 300 MB/s of RC4 per CPU core, so even one core can saturate a 1 Gbit/s link. You can setup a cheap box to be a SSL proxy in front of the real web servers to offload SSL. ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
mac address cleaning ignored ...
Hi All, i have faced some "weird" mac address caching issue ... need help to understand who is who :( i have box A: FreeBSD 8.1-STABLE #2 amd64 and box B: FreeBSD 8.1-STABLE #3 amd64 both are connected to TP Link switch TL-SG5426 after nic change on box B i can ping box B from box A but i can not ping A from B ... i see the box A continue to reply to the box B old nic mac address ... how can it be possible if i have made: - arp cleaning on box A after what arp shows correct mac for host B -- pf reload/restart - arp cleaning on box B after what arp shows correct mac for host B -- pf reload/restart -- cold reboot - host/dns/mac cleaning on switch after what mac table contains correct mac for host B -- switch soft reload on box A i have quagga running ... can it cach mac addresses? the nic change was needed due to the heavy onboard nic re(4) card=0x83a31043 chip=0x816810ec Gigabit Ethernet NIC(NDIS 6.0) (RTL8168/8111/8111c) flapping really i have no idea what to do except cold reboot of box A ... but who can to continue to remembers box B old nic mac address after all of that cleanings? thanks in advance -- Zeus V. Panchenko IT Dpt., IBS ltdGMT+2 (EET) ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: misc/150249: [ixgbe] Media type detection broken
Synopsis: [ixgbe] Media type detection broken Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: vwe Responsible-Changed-When: Fri Sep 3 18:57:36 UTC 2010 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=150249 ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: misc/150251: [patch] [ixgbe] Late cable insertion broken
Old Synopsis: [ixgbe] Late cable insertion broken New Synopsis: [patch] [ixgbe] Late cable insertion broken Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: vwe Responsible-Changed-When: Fri Sep 3 18:58:05 UTC 2010 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=150251 ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
RE: seeking current supported crypto co-processors
Thanks Andre, I'm hoping not to get too distracted by which algorithms I want supported. To answer directly, I want the FIPS-140-2 algorithms in block modes and optionally the Suite-B NSA stuff too. http://csrc.nist.gov/publications/fips/fips140-2/fips1402annexa.pdf http://www.nsa.gov/ia/programs/suiteb_cryptography/index.shtml But the main thrust of my question is not what algs are supported by what parts... but instead, are their PCIe attachable crypto co-processors with current vendor support for FreeBSD8.x? I appreciated your pointers to VIA and various MIPS and specifically octeon processors. And I am newly enlightened by your pointers to very new Intel parts coming out with cipher/hash support... that may help me in the near future. But at the moment, I am currently bound to Intel parts without the AES feature set. If anyone else reading this thread want's to chime in with info about current supported crypto co-processors that plug in via PCIe, please drop a note. --- Ricky Charlet Adara Networks USA 408-433-4942 -Original Message- From: Andre Oppermann [mailto:an...@freebsd.org] Sent: Thursday, September 02, 2010 11:07 PM To: Ricky Charlet Cc: freebsd-secur...@freebsd.org; freebsd-net@freebsd.org Subject: Re: seeking current supported crypto co-processors On 03.09.2010 02:35, Ricky Charlet wrote: > Howdy, > > I'm seeking current cryptographic coprocessors supported in FreeBSD 8.x. By > perusing through the > crypto-dev (and subsequently referenced) man page(s) I found this list: Hifn > 7751/7951/7811/7955/7956 crypto accelerator SafeNet 1141/1741 Bluesteel > 5501/5601 Broadcom > bcm5801/5802/5805/5820/5821/5822/5823/5825 > > Those are all pretty old (and in some cases, no longer existent). I'm > surveying these lists to > see if anyone knows of more modern chips working with FreeBSD 8.x. Or if you > feel some chip on > the list above is up to the task of near about 1 Gb throughput across a PCIe > and has friendly > vendor support for FreeBSD, I'd sure like to hear about that too. What cypto algorithms do you need? Stream encryption and/or PKI KEX? For AES stream encrpytion there are some CPU's that directly support the crypto primitives on the silicon. For newer x86/amd64 CPU's see: http://en.wikipedia.org/wiki/AES_instruction_set A number of VIA x86 CPU's have supported a set of crypto algorithms inlcuding stream cyphers, cryptographic hashing and RSA for quite some time on their silicon. http://www.via.com.tw/en/initiatives/padlock/hardware.jsp Other than that there are some embedded crypto engines with their own (mostly MIPS based) single and multi-core CPU's. AKAIK they have a FreeBSD API and the FreeBSD MIPS port should work on at least some of them: http://www.caviumnetworks.com/ Cavium also has some plug-in crypto accelerator cards under the brand name Nitrox. IIRC they have some drivers for FreeBSD available. -- Andre ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
RE: seeking current supported crypto co-processors
Thanks Ivan, You have some valid points about performance. I was hoping not to get distracted from the main thrust of my question by performance considerations though. Are their PCIe attachable crypto co-processors with current vendor support for FreeBSD8.x? If anyone else reading this thread want's to chime in with info about current supported crypto co-processors that plug in via PCIe, please drop a note. However, I think you do deserve a reply on the performance topic... I am close enough to agreeing with you to not argue much about whether modern CPU parts can saturate a 1 Gb link with crypto data. The CPU part I am currently married to (a touch old but not that bad), seems to be able to through around 200Mb of IP-ESP data around. However, in spite of these observations, I would prefer if my system could handle that throughput load and yet have CPU power left over for other tasks. I'm very attracted to Andre's mention of "newer x86/amd64 CPU's see: http://en.wikipedia.org/wiki/AES_instruction_set";. Does anyone know if FreeBSD supports or will support this through either /dev/crypto or through openssl (or any other mechanism I guess)? --- Ricky Charlet Adara Networks USA 408-433-4942 -Original Message- From: owner-freebsd-...@freebsd.org [mailto:owner-freebsd-...@freebsd.org] On Behalf Of Ivan Voras Sent: Friday, September 03, 2010 2:49 AM To: freebsd-net@freebsd.org Cc: freebsd-secur...@freebsd.org Subject: Re: seeking current supported crypto co-processors On 09/03/10 02:35, Ricky Charlet wrote: > Howdy, > > > I'm seeking current cryptographic coprocessors supported in FreeBSD > 8.x. By perusing through the crypto-dev (and subsequently referenced) man > page(s) I found this list: > Hifn 7751/7951/7811/7955/7956 crypto accelerator > SafeNet 1141/1741 > Bluesteel 5501/5601 > Broadcom bcm5801/5802/5805/5820/5821/5822/5823/5825 > > Those are all pretty old (and in some cases, no longer existent). > I'm surveying these lists to see if anyone knows of more modern chips working > with FreeBSD 8.x. Or if you feel some chip on the list above is up to the > task of near about 1 Gb throughput across a PCIe and has friendly vendor > support for FreeBSD, I'd sure like to hear about that too. > I'm not saying they are useless but are you really sure you need them? Even on the last generation of CPUs without AES instructions you can easily get 125 MB/s of AES-128 encryption and 300 MB/s of RC4 per CPU core, so even one core can saturate a 1 Gbit/s link. You can setup a cheap box to be a SSL proxy in front of the real web servers to offload SSL. ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org" ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: seeking current supported crypto co-processors
On Fri, Sep 03, 2010 at 02:26:37PM -0700, Ricky Charlet wrote: > Thanks Ivan, > > You have some valid points about performance. I was hoping not to get > distracted from the main thrust of my question by performance considerations > though. > > Are their PCIe attachable crypto co-processors with current vendor > support for FreeBSD8.x? If anyone else reading this thread want's to chime > in with info about current supported crypto co-processors that plug in via > PCIe, please drop a note. > > > However, I think you do deserve a reply on the performance topic... > > I am close enough to agreeing with you to not argue much about > whether modern CPU parts can saturate a 1 Gb link with crypto data. The CPU > part I am currently married to (a touch old but not that bad), seems to be > able to through around 200Mb of IP-ESP data around. However, in spite of > these observations, I would prefer if my system could handle that throughput > load and yet have CPU power left over for other tasks. > > I'm very attracted to Andre's mention of "newer x86/amd64 > CPU's see: http://en.wikipedia.org/wiki/AES_instruction_set";. Does > anyone know if FreeBSD supports or will support this through either > /dev/crypto or through openssl (or any other mechanism I guess)? I believe recent OpenSSL 1.x supports AESNI in usermode. For the AES acceleration in the kernel and /dev/crypto support see the aesni driver in the recent HEAD, working both on i386 and amd64 architectures. I had a plan to merge the driver into RELENG_8, but it is stalled due to some issues (not related to the driver quality). pgpv3ra9Wm2aY.pgp Description: PGP signature
Re: NFE adapter 'hangs'
On Fri, Sep 03, 2010 at 07:59:26AM +0100, Melissa Jenkins wrote: > > Thank you for your very quick response :) > [...] > >Also I'd like to know whether both RX and TX are dead or only one > >RX/TX path is hung. Can you see incoming traffic with tcpdump when > >you think the controller is in stuck? > > Yes, though not very much. The traffic to 4800 is every second so you can see > in the following trace when it stops > > 07:10:42.287163 IP 192.168.1.203 > 224.0.0.240: pfsync 108 > 07:10:42.911995 > 07:10:43.112073 STP 802.1d, Config, Flags [Topology change], bridge-id > 8000.c4:7d:4f:a9:ac:30.8008, length 43 > 07:10:43.148659 IP 192.168.1.203.57026 > 192.168.1.255.4800: UDP, length 60 > 07:10:43.148684 IP 172.31.1.203 > 172.31.1.129: GREv0, length 92: IP > 192.168.1.203.57026 > 192.168.1.129.4800: UDP, length 60 > 07:10:43.148689 IP 172.31.1.203 > 172.31.1.129: GREv0, length 92: IP > 192.168.1.203.57026 > 192.168.1.1.4800: UDP, length 60 > 07:10:43.148918 IP 192.168.1.213.40677 > 192.168.1.255.4800: UDP, length 48 [...] > a bit later on, still broken, a slight odd message: > 07:11:43.079720 IP 172.31.1.129 > 172.31.1.213: GREv0, length 52: IP > 192.168.1.129.60446 > 192.168.1.213.179: tcp 12 [bad hdr length 16 - too > short, < 20] > 07:11:44.210794 IP 172.31.1.129 > 172.31.1.203: GREv0, length 84: IP > 192.168.1.129.64744 > 192.168.1.203.4800: UDP, length 52 > 07:11:44.210831 IP 172.31.1.129 > 172.31.1.213: GREv0, length 84: IP > 192.168.1.129.64744 > 192.168.1.213.4800: UDP, length 52 > > Now this really is odd, I don't recognise either of those MAC addresses, > though the SQL shown is used on this machine ( > 07:12:13.054393 45:43:54:20:41:63 > 00:00:03:53:45:4c, ethertype Unknown > (0x6374), length 60: > 0x: 556e 6971 7565 4964 2046 524f 4d20 7261 UniqueId.FROM.ra > 0x0010: 6461 6363 7420 2057 4845 5245 2043 616c dacct..WHERE.Cal > 0x0020: 6c69 6e67 5374 6174 696f 6e49 6420 lingStationId. Hmm, it seems you're using really complex setup. It's very hard to narrow down guilty ones under these environments. Could you setup simple network configuration that reproduces the issue? One of possible cause would be wrong(garbled) data might be passed up to upper stack. But I have no idea why you see GRE packets with truncated TCP header(172.31.1.129 > 172.31.1.213). How about disabling TX/RX checksum offloading as well as TSO? [...] > > I then restarted the interface (nfe down/up, route restart) > > From dmesg at the time (slight obfuscated) > Sep 3 07:10:19 manch2 bgpd[89612]: neighbor XX: received notification: > HoldTimer expired, unknown subcode 0 > Sep 3 07:10:49 manch2 bgpd[89612]: neighbor XX connect: Host is down > # at this point I took the interface down & up and reloaded the routing tables > Sep 3 07:12:07 manch2 kernel: carp0: link state changed to DOWN > Sep 3 07:12:07 manch2 kernel: carp0: link state changed to DOWN > Sep 3 07:12:07 manch2 kernel: nfe0: link state changed to DOWN > Sep 3 07:12:07 manch2 kernel: carp0: link state changed to DOWN > Sep 3 07:12:11 manch2 kernel: nfe0: link state changed to UP > Sep 3 07:12:11 manch2 kernel: carp0: link state changed to DOWN > Sep 3 07:12:14 manch2 kernel: carp0: link state changed to UP Hmm, it does not look right, carp0 showed link DOWN message four times in a row. By the way, are you using IPMI on MCP55? nfe(4) is not ready to handle MAC operation with IPMI. ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
mac address cleaning ignored ...
cold reboot of box A helped ... why only cold? :( -- Zeus V. Panchenko IT Dpt., IBS ltdGMT+2 (EET) ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
