Re: ix(intel) vs mlxen(mellanox) 10Gb performance

[Daniel Braniss]

> On Aug 18, 2015, at 12:49 AM, Rick Macklem  wrote:
> 
> Daniel Braniss wrote:
>> 
>>> On Aug 17, 2015, at 3:21 PM, Rick Macklem  wrote:
>>> 
>>> Daniel Braniss wrote:
 
> On Aug 17, 2015, at 1:41 PM, Christopher Forgeron 
> wrote:
> 
> FYI, I can regularly hit 9.3 Gib/s with my Intel X520-DA2's and FreeBSD
> 10.1. Before 10.1 it was less.
> 
 
 this is NOT iperf/3 where i do get close to wire speed,
 it’s NFS writes, i.e., almost real work :-)
 
> I used to tweak the card settings, but now it's just stock. You may want
> to
> check your settings, the Mellanox may just have better defaults for your
> switch.
> 
>>> Have you tried disabling TSO for the Intel? With TSO enabled, it will be
>>> copying
>>> every transmitted mbuf chain to a new chain of mbuf clusters via.
>>> m_defrag() when
>>> TSO is enabled. (Assuming you aren't an 82598 chip. Most seem to be the
>>> 82599 chip
>>> these days?)
>>> 
>> 
>> hi Rick
>> 
>> how can i check the chip?
>> 
> Haven't a clue. Does "dmesg" tell you? (To be honest, since disabling TSO 
> helped,
> I'll bet you don't have a 82598.)
> 
>>> This has been fixed in the driver very recently, but those fixes won't be
>>> in 10.1.
>>> 
>>> rick
>>> ps: If you could test with 10.2, it would be interesting to see how the ix
>>> does with
>>>   the current driver fixes in it?
>> 
>> I new TSO was involved!
>> ok, firstly, it’s 10.2 stable.
>> with TSO enabled, ix is bad, around 64MGB/s.
>> disabling TSO it’s better, around 130
>> 
> Hmm, could you check to see of these lines are in sys/dev/ixgbe/if_ix.c at 
> around
> line#2500?
>  /* TSO parameters */
> 2572   ifp->if_hw_tsomax = 65518;
> 2573   ifp->if_hw_tsomaxsegcount = IXGBE_82599_SCATTER;
> 2574   ifp->if_hw_tsomaxsegsize = 2048;
> 
> They are in stable/10. I didn't look at releng/10.2. (And if they're in a 
> #ifdef
> for FreeBSD11, take the #ifdef away.)
> If they are there and not ifdef'd, I can't explain why disabling TSO would 
> help.
> Once TSO is fixed so that it handles the 64K transmit segments without 
> copying all
> the mbufs, I suspect you might get better perf. with it enabled?
> 

this is 10.2 :
they are on lines  2509-2511 and I don’t see any #ifdefs around it.

the plot thickens :-)

danny

> Good luck with it, rick
> 
>> still, mlxen0 is about 250! with and without TSO
>> 
>> 
>>> 
> On Mon, Aug 17, 2015 at 6:41 AM, Slawa Olhovchenkov  > wrote:
> On Mon, Aug 17, 2015 at 10:27:41AM +0300, Daniel Braniss wrote:
> 
>> hi,
>> I have a host (Dell R730) with both cards, connected to an HP8200
>> switch at 10Gb.
>> when writing to the same storage (netapp) this is what I get:
>> ix0:~130MGB/s
>> mlxen0  ~330MGB/s
>> this is via nfs/tcpv3
>> 
>> I can get similar (bad) performance with the mellanox if I increase
>> the file size
>> to 512MGB.
> 
> Look like mellanox have internal beffer for caching and do ACK
> acclerating.
> 
>> so at face value, it seems the mlxen does a better use of resources
>> than the intel.
>> Any ideas how to improve ix/intel's performance?
> 
> Are you sure about netapp performance?
> ___
> freebsd-...@freebsd.org  mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-net
> 
> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org
> "
> 
 
 ___
 freebsd-stable@freebsd.org mailing list
 https://lists.freebsd.org/mailman/listinfo/freebsd-stable
 To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
>> 
>> ___
>> freebsd-stable@freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
>> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ix(intel) vs mlxen(mellanox) 10Gb performance

[Rick Macklem]

Daniel Braniss wrote:
> 
> > On Aug 18, 2015, at 12:49 AM, Rick Macklem  wrote:
> > 
> > Daniel Braniss wrote:
> >> 
> >>> On Aug 17, 2015, at 3:21 PM, Rick Macklem  wrote:
> >>> 
> >>> Daniel Braniss wrote:
>  
> > On Aug 17, 2015, at 1:41 PM, Christopher Forgeron
> > 
> > wrote:
> > 
> > FYI, I can regularly hit 9.3 Gib/s with my Intel X520-DA2's and FreeBSD
> > 10.1. Before 10.1 it was less.
> > 
>  
>  this is NOT iperf/3 where i do get close to wire speed,
>  it’s NFS writes, i.e., almost real work :-)
>  
> > I used to tweak the card settings, but now it's just stock. You may
> > want
> > to
> > check your settings, the Mellanox may just have better defaults for
> > your
> > switch.
> > 
> >>> Have you tried disabling TSO for the Intel? With TSO enabled, it will be
> >>> copying
> >>> every transmitted mbuf chain to a new chain of mbuf clusters via.
> >>> m_defrag() when
> >>> TSO is enabled. (Assuming you aren't an 82598 chip. Most seem to be the
> >>> 82599 chip
> >>> these days?)
> >>> 
> >> 
> >> hi Rick
> >> 
> >> how can i check the chip?
> >> 
> > Haven't a clue. Does "dmesg" tell you? (To be honest, since disabling TSO
> > helped,
> > I'll bet you don't have a 82598.)
> > 
> >>> This has been fixed in the driver very recently, but those fixes won't be
> >>> in 10.1.
> >>> 
> >>> rick
> >>> ps: If you could test with 10.2, it would be interesting to see how the
> >>> ix
> >>> does with
> >>>   the current driver fixes in it?
> >> 
> >> I new TSO was involved!
> >> ok, firstly, it’s 10.2 stable.
> >> with TSO enabled, ix is bad, around 64MGB/s.
> >> disabling TSO it’s better, around 130
> >> 
> > Hmm, could you check to see of these lines are in sys/dev/ixgbe/if_ix.c at
> > around
> > line#2500?
> >  /* TSO parameters */
> > 2572 ifp->if_hw_tsomax = 65518;
> > 2573 ifp->if_hw_tsomaxsegcount = 
> > IXGBE_82599_SCATTER;
> > 2574 ifp->if_hw_tsomaxsegsize = 2048;
> > 
> > They are in stable/10. I didn't look at releng/10.2. (And if they're in a
> > #ifdef
> > for FreeBSD11, take the #ifdef away.)
> > If they are there and not ifdef'd, I can't explain why disabling TSO would
> > help.
> > Once TSO is fixed so that it handles the 64K transmit segments without
> > copying all
> > the mbufs, I suspect you might get better perf. with it enabled?
> > 
> 
> this is 10.2 :
> they are on lines  2509-2511 and I don’t see any #ifdefs around it.
> 
> the plot thickens :-)
> 
If this is just a test machine, maybe you could test with these lines (at about 
#880)
in sys/netinet/tcp_output.c commented out? (It looks to me like this will 
disable TSO
for almost all the NFS writes.)
- around line #880 in sys/netinet/tcp_output.c:
/*
 * In case there are too many small fragments
 * don't use TSO:
 */
if (len <= max_len) {
len = max_len;
sendalot = 1;
tso = 0;
}

This was added along with the other stuff that did the if_hw_tsomaxsegcount, 
etc and I
never noticed it until now (not my patch).

rick

> danny
> 
> > Good luck with it, rick
> > 
> >> still, mlxen0 is about 250! with and without TSO
> >> 
> >> 
> >>> 
> > On Mon, Aug 17, 2015 at 6:41 AM, Slawa Olhovchenkov  > > wrote:
> > On Mon, Aug 17, 2015 at 10:27:41AM +0300, Daniel Braniss wrote:
> > 
> >> hi,
> >> I have a host (Dell R730) with both cards, connected to an HP8200
> >> switch at 10Gb.
> >> when writing to the same storage (netapp) this is what I get:
> >> ix0:~130MGB/s
> >> mlxen0  ~330MGB/s
> >> this is via nfs/tcpv3
> >> 
> >> I can get similar (bad) performance with the mellanox if I
> >> increase
> >> the file size
> >> to 512MGB.
> > 
> > Look like mellanox have internal beffer for caching and do ACK
> > acclerating.
> > 
> >> so at face value, it seems the mlxen does a better use of
> >> resources
> >> than the intel.
> >> Any ideas how to improve ix/intel's performance?
> > 
> > Are you sure about netapp performance?
> > ___
> > freebsd-...@freebsd.org  mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-net
> > 
> > To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org
> > "
> > 
>  
>  ___
>  freebsd-stable@freebsd.org mailing list
>  https://lists.freebsd.org/mail

ping from web application

[Marko Cupać]

Hi,

I use web applicaton (net-mgmt/phpipam) which should have the ability
to check hosts' availability via ping. I can even specify path to ping
executable.

This functionality does not work on FreeBSD by default, and suggested
workaround is to set setuid bit on /sbin/ping.

I don't like to modify permissions of base files. Is there an
alternative solution? e.g. a port?

Thank you in advance,
-- 
Marko Cupać
https://www.mimar.rs/
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ix(intel) vs mlxen(mellanox) 10Gb performance

[Hans Petter Selasky]

On 08/18/15 14:53, Rick Macklem wrote:

If this is just a test machine, maybe you could test with these lines (at about 
#880)
in sys/netinet/tcp_output.c commented out? (It looks to me like this will 
disable TSO
for almost all the NFS writes.)
- around line #880 in sys/netinet/tcp_output.c:
/*
 * In case there are too many small fragments
 * don't use TSO:
 */
if (len <= max_len) {
len = max_len;
sendalot = 1;
tso = 0;
}

This was added along with the other stuff that did the if_hw_tsomaxsegcount, 
etc and I
never noticed it until now (not my patch).


FYI:

These lines are needed by other hardware, like the mlxen driver. If you 
remove them mlxen will start doing m_defrag(). I believe if you set the 
correct parameters in the "struct ifnet" for the TSO size/count limits 
this problem will go away. If you print the "len" and "max_len" and also 
the cases where TSO limits are reached, you'll see what parameter is 
triggering it and needs to be increased.


--HPS
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ix(intel) vs mlxen(mellanox) 10Gb performance

[Hans Petter Selasky]

On 08/18/15 14:53, Rick Macklem wrote:

2572 ifp->if_hw_tsomax = 65518;

>2573 ifp->if_hw_tsomaxsegcount = IXGBE_82599_SCATTER;
>2574 ifp->if_hw_tsomaxsegsize = 2048;


Hi,

If IXGBE_82599_SCATTER is the maximum scatter/gather entries the 
hardware can do, remember to subtract one fragment for the TCP/IP-header 
mbuf!


I think there is an off-by-one here:

ifp->if_hw_tsomax = 65518;
ifp->if_hw_tsomaxsegcount = IXGBE_82599_SCATTER - 1;
ifp->if_hw_tsomaxsegsize = 2048;

Refer to:


 *
 * NOTE: The TSO limits only apply to the data payload part of
 * a TCP/IP packet. That means there is no need to subtract
 * space for ethernet-, vlan-, IP- or TCP- headers from the
 * TSO limits unless the hardware driver in question requires
 * so.


In sys/net/if_var.h

Thank you!

--HPS

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ix(intel) vs mlxen(mellanox) 10Gb performance

[Daniel Braniss]

sorry, it’s been a tough day, we had a major meltdown, caused by a faulty gbic 
:-(
anyways, could you tell me what to do?
comment out, fix the off by one?

the machine is not yet production.

thanks,
danny

> On 18 Aug 2015, at 16:32, Hans Petter Selasky  wrote:
> 
> On 08/18/15 14:53, Rick Macklem wrote:
>> 2572  ifp->if_hw_tsomax = 65518;
>>> >2573ifp->if_hw_tsomaxsegcount = 
>>> >IXGBE_82599_SCATTER;
>>> >2574ifp->if_hw_tsomaxsegsize = 2048;
> 
> Hi,
> 
> If IXGBE_82599_SCATTER is the maximum scatter/gather entries the hardware can 
> do, remember to subtract one fragment for the TCP/IP-header mbuf!
> 
> I think there is an off-by-one here:
> 
> ifp->if_hw_tsomax = 65518;
> ifp->if_hw_tsomaxsegcount = IXGBE_82599_SCATTER - 1;
> ifp->if_hw_tsomaxsegsize = 2048;
> 
> Refer to:
> 
>> *
>> * NOTE: The TSO limits only apply to the data payload part of
>> * a TCP/IP packet. That means there is no need to subtract
>> * space for ethernet-, vlan-, IP- or TCP- headers from the
>> * TSO limits unless the hardware driver in question requires
>> * so.
> 
> In sys/net/if_var.h
> 
> Thank you!
> 
> --HPS
> 

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ping from web application

[Lowell Gilbert]

Marko Cupać  writes:

> I use web applicaton (net-mgmt/phpipam) which should have the ability
> to check hosts' availability via ping. I can even specify path to ping
> executable.
>
> This functionality does not work on FreeBSD by default, and suggested
> workaround is to set setuid bit on /sbin/ping.
>
> I don't like to modify permissions of base files. Is there an
> alternative solution? e.g. a port?

In what way does ping(8) not work? A look at its error output should
tell you what the problem is.

Additionally, the standard permissions on /sbin/ping *are* suid root.
It certainly won't work if you've changed that, so just change it back.

And yes, there are other ping programs present, including some with
pretty graphical web page UIs. But there's no reason that PHP should
have trouble calling /sbin/ping.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Issue with /boot/menu.rc.local after 10.2 update

[Cassiano Peixoto]

Hi guys,

I've been using /boot/menu.rc.local for years to my custom menu option. But
after 10.2 update, it's not working anymore. I always get the message
"toggle_menuitem not found" on beastie menu.

Here is my /boot/menu.rc.local:

set optionsmenu_init[7]="init_console"
set optionsmenu_caption[7]="[C]onsole. Off"
set optionstoggled_text[7]="[C]onsole. On"
set optionsmenu_command[7]="toggle_console"
set optionsmenu_keycode[7]=118
set optionsansi_caption[7]="^[[1mC^[[37monsole. ^[[34;1mOff^[[37m"
set optionstoggled_ansi[7]="^[[1mC^[[37monsole. ^[[32;7mOn^[[0;37m"

\
\ Console Boot
\

: console_enabled? ( -- flag )
s" boot_single" getenv -1 <> dup if
drop ( c-addr flag -- flag )
then
;

: console_enable ( -- )
s" set console=comconsole,vidconsole" evaluate
;

: console_disable ( -- )
s" set console=vidconsole,comconsole" evaluate
;

: toggle_console ( N -- N TRUE )
toggle_menuitem
menu-redraw

\ Now we're going to make the change effective

dup toggle_stateN @ 0= if
console_disable
else
console_enable
then

TRUE \ loop menu again
;

So what has changed? What should i do to make it works again?

Thanks.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ix(intel) vs mlxen(mellanox) 10Gb performance

[Slawa Olhovchenkov]

On Tue, Aug 18, 2015 at 05:09:41PM +0300, Daniel Braniss wrote:

> sorry, it's been a tough day, we had a major meltdown, caused by a faulty 
> gbic :-(
> anyways, could you tell me what to do?
> comment out, fix the off by one?
> 
> the machine is not yet production.

Can you collect this information?
https://lists.freebsd.org/pipermail/freebsd-stable/2015-August/083113.html

And 'show interface' (or equivalent: error/collsion/events counters)
from both ports from HP8200.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Swap Questions

[Antony Uspensky]

On Mon, 17 Aug 2015, Tim Daneliuk wrote:


On 08/17/2015 12:53 PM, Antony Uspensky wrote:

On Sat, 15 Aug 2015, Tim Daneliuk wrote:


So -L does fix the problem - sort of.  The machine picks up the file as
additional swap on boot just fine.  HWOEVER, when I try to reboot or shut
down the host, I get a panic telling me some noise about not being able
to shutdown swap for some reason.


Try to swapoff (by hands) before shutdown.
Shutdown sequence, I think, unmounts carrying disk before swapping off a 
carried file. If I am right, -L should be processed on shutdown also.
Just a guess.


Yes, that did it.

But, isn't this kind of an operational bug?  Shouldn't the shutdown logic
do the swapoff before the unmount if it sees files being used for swap?


Yes. Must.


i.e. Should I enter this as a bug report?


Yes, please.


The only reason this matters - and it's a pretty big reason - is for production
servers when someone logs in remotely, becomes root, and issued "reboot".  The
machine hangs at the panic and never comes back ... something you do not see
unless you are in a console of some sort ...


___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Swap Questions

[Tim Daneliuk]

On 08/18/2015 12:29 PM, Antony Uspensky wrote:
> On Mon, 17 Aug 2015, Tim Daneliuk wrote:
> 
>> On 08/17/2015 12:53 PM, Antony Uspensky wrote:
>>> On Sat, 15 Aug 2015, Tim Daneliuk wrote:
>>>
 So -L does fix the problem - sort of.  The machine picks up the file as
 additional swap on boot just fine.  HWOEVER, when I try to reboot or shut
 down the host, I get a panic telling me some noise about not being able
 to shutdown swap for some reason.
>>>
>>> Try to swapoff (by hands) before shutdown.
>>> Shutdown sequence, I think, unmounts carrying disk before swapping off a 
>>> carried file. If I am right, -L should be processed on shutdown also.
>>> Just a guess.
>>
>> Yes, that did it.
>>
>> But, isn't this kind of an operational bug?  Shouldn't the shutdown logic
>> do the swapoff before the unmount if it sees files being used for swap?
> 
> Yes. Must.
> 
>> i.e. Should I enter this as a bug report?
> 
> Yes, please.
> 
>> The only reason this matters - and it's a pretty big reason - is for 
>> production
>> servers when someone logs in remotely, becomes root, and issued "reboot".  
>> The
>> machine hangs at the panic and never comes back ... something you do not see
>> unless you are in a console of some sort ...
> 

Done.  Bug report #202420
-- 

Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ping from web application

[Tom Samplonius]

> On Aug 18, 2015, at 7:15 AM, Lowell Gilbert 
>  wrote:
> 
> Marko Cupać  writes:
> 
>> I use web applicaton (net-mgmt/phpipam) which should have the ability
>> to check hosts' availability via ping. I can even specify path to ping
>> executable.
>> 
>> This functionality does not work on FreeBSD by default, and suggested
>> workaround is to set setuid bit on /sbin/ping.
>> 
>> I don't like to modify permissions of base files. Is there an
>> alternative solution? e.g. a port?
> 
> In what way does ping(8) not work? A look at its error output should
> tell you what the problem is.
> 
> Additionally, the standard permissions on /sbin/ping *are* suid root.
> It certainly won't work if you've changed that, so just change it back.
> 
> And yes, there are other ping programs present, including some with
> pretty graphical web page UIs. But there's no reason that PHP should
> have trouble calling /sbin/ping.


  It is a pretty standard issue:  only apps running as root can send ICMP 
directly, as ping does.  PHP runs in Apache, and to prevent security issues 
with privilege escalation setuid programs are forced to run as an unprivileged 
user.

  I would check to see how “fping” in Nagios solved this issue.


Tom

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Issue with /boot/menu.rc.local after 10.2 update

[Cassiano Peixoto]

I just found out the problem. I don't know who added the following phrase
to /boot/menu-commands.4th:

only forth definitions

Line 31 and 354. It just broke /boot/menu.rc.local as i said before. After
removing that everything worked again. Hope it helps someone else. I'm
going to open a PR anyway.

Thanks.

On Tue, Aug 18, 2015 at 11:13 AM, Cassiano Peixoto <
peixotocassi...@gmail.com> wrote:

> Hi guys,
>
> I've been using /boot/menu.rc.local for years to my custom menu option.
> But after 10.2 update, it's not working anymore. I always get the message
> "toggle_menuitem not found" on beastie menu.
>
> Here is my /boot/menu.rc.local:
>
> set optionsmenu_init[7]="init_console"
> set optionsmenu_caption[7]="[C]onsole. Off"
> set optionstoggled_text[7]="[C]onsole. On"
> set optionsmenu_command[7]="toggle_console"
> set optionsmenu_keycode[7]=118
> set optionsansi_caption[7]="^[[1mC^[[37monsole. ^[[34;1mOff^[[37m"
> set optionstoggled_ansi[7]="^[[1mC^[[37monsole. ^[[32;7mOn^[[0;37m"
>
> \
> \ Console Boot
> \
>
> : console_enabled? ( -- flag )
> s" boot_single" getenv -1 <> dup if
> drop ( c-addr flag -- flag )
> then
> ;
>
> : console_enable ( -- )
> s" set console=comconsole,vidconsole" evaluate
> ;
>
> : console_disable ( -- )
> s" set console=vidconsole,comconsole" evaluate
> ;
>
> : toggle_console ( N -- N TRUE )
> toggle_menuitem
> menu-redraw
>
> \ Now we're going to make the change effective
>
> dup toggle_stateN @ 0= if
> console_disable
> else
> console_enable
> then
>
> TRUE \ loop menu again
> ;
>
> So what has changed? What should i do to make it works again?
>
> Thanks.
>
>
>
>
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: [FreeBSD-Announce] FreeBSD Errata Notice FreeBSD-EN-15:12.netstat

[Philip Gollucci]

We do not run 32bit.

On Tue, Aug 18, 2015 at 4:15 PM, FreeBSD Errata Notices <
errata-noti...@freebsd.org> wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA512
>
>
> =
> FreeBSD-EN-15:12.netstatErrata
> Notice
>   The FreeBSD
> Project
>
> Topic:  Incorrect netstat(1) data handling on 32-bit systems
>
> Category:   core
> Module: netstat
> Announced:  2015-08-18
> Credits:Mark Johnston
> Affects:FreeBSD 10.2-RELEASE
> Corrected:  2015-07-31 00:21:41 UTC (stable/10, 10.2-STABLE)
> 2015-08-18 19:30:17 UTC (releng/10.2, 10.2-RC3-p1)
> 2015-08-18 19:30:17 UTC (releng/10.2, 10.2-RELEASE-p1)
>
> For general information regarding FreeBSD Errata Notices and Security
> Advisories, including descriptions of the fields above, security
> branches, and the following sections, please visit
> https://security.freebsd.org/>.
>
> I.   Background
>
> The netstat(1) utility displays the contents of various network related
> data
> structures.
>
> II.  Problem Description
>
> The netstat(1) utility incorrectly handles reported values on 32-bit
> systems.
>
> III. Impact
>
> Due to how netstat(1) processes IPSEC counters, the utility may produce
> incorrect output on 32-bit systems.
>
> IV.  Workaround
>
> No workaround is available, however systems without IPSEC compiled into the
> kernel are not affected.
>
> V.   Solution
>
> Perform one of the following:
>
> 1) Upgrade your system to a supported FreeBSD stable or release / security
> branch (releng) dated after the correction date.
>
> 2) To update your present system via a binary patch:
>
> Systems running a RELEASE version of FreeBSD on the i386 or amd64
> platforms can be updated via the freebsd-update(8) utility:
>
> # freebsd-update fetch
> # freebsd-update install
>
> 3) To update your present system via a source code patch:
>
> The following patches have been verified to apply to the applicable
> FreeBSD release branches.
>
> a) Download the relevant patch from the location below, and verify the
> detached PGP signature using your PGP utility.
>
> # fetch https://security.FreeBSD.org/patches/EN-15:12/netstat.patch
> # fetch https://security.FreeBSD.org/patches/EN-15:12/netstat.patch.asc
> # gpg --verify netstat.patch.asc
>
> b) Apply the patch.  Execute the following commands as root:
>
> # cd /usr/src
> # patch < /path/to/patch
>
> c) Recompile the operating system using buildworld and installworld as
> described in https://www.FreeBSD.org/handbook/makeworld.html>.
>
> VI.  Correction details
>
> The following list contains the correction revision numbers for each
> affected branch.
>
> Branch/path  Revision
> - -
> stable/10/r286099
> releng/10.2/  r286901
> - -
>
> To see which files were modified by a particular revision, run the
> following command, replacing NN with the revision number, on a
> machine with Subversion installed:
>
> # svn diff -cNN --summarize svn://svn.freebsd.org/base
>
> Or visit the following URL, replacing NN with the revision number:
>
> https://svnweb.freebsd.org/base?view=revision&revision=NN>
>
> VII. References
>
> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=201700>
>
> The latest revision of this Errata Notice is available at
> https://security.FreeBSD.org/advisories/FreeBSD-EN-15:12.netstat.asc
>
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v2.1.7 (FreeBSD)
>
> iQIcBAEBCgAGBQJV05A1AAoJEO1n7NZdz2rnAzIQANyLdOOQhe9dHyAV4N5YKM3B
> Z/86dY/KIIViVb1uzkBASBNnkHlG+mCMOQpzX2x8yCPF4i7bIEfPa4r2Bzw9pvCF
> RWRKvZXERESh/RndQFhxcmJMAyYPq7MdK0IZzG53vinlMoSz2WTKj2vSR7t2jfo+
> ObTfgdkqN/PZs/W+AQY8a4DMdxCLg1KCZiSpQRO7ea+4AxsI8qNgoytvG6HRno/z
> uGe6Ad82ZfysKgqe9JO4gvRTR77ebQAVSSr3qylQcOGHohy9tFHcI2FEAAqLJrQY
> b5DDLOawLRsQm0hwkLCTOZX2QvIFgz0gGRpvPcN9ZKValMc5DKQv36z3hOByK+3i
> dDHFG/Diy2JNP0tsKtW8IyyLvW2DAUoTs1nVaWMvLKkMUr+loOYvoaLdGT0xQP2d
> M6UT40mRMznfH/Gq/0DJFArsYcyB9YRl7rD0dy1HhqApogHQrTjsT+1vtBtpaTmv
> LHA77tHyzI0TxOvmx3hglj/z4BLZDPU6ydXr3zeOYBpLz5p02GKxHUc+JrmWBfOV
> Jep0+Fr2fYST5bGVtExNQV6cTlBZPnGR4JxJEUQA6a+FdyJcDuzOTNcs0YzwjuSC
> dIk5pdxI3nkc+zf9GZLXUdLcxXfo6jBUy0fSWkzirGzBfo0wxseE6cbxxTH7vumx
> CLGGmHiqxVuF/nP4ScHi
> =3aK1
> -END PGP SIGNATURE-
> ___
> freebsd-annou...@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-announce
> To unsubscribe, send any mail to "freebsd-announce-unsubscr...@freebsd.org
> "
>



-- 
*PHILIP M. GOLLUCCI*
Sr. Director of IT | *Curb*
*w.* 703-579-6947
*m.* 703-

Re: [FreeBSD-Announce] FreeBSD Errata Notice FreeBSD-EN-15:12.netstat

[Philip Gollucci]

sorry, wrong cc.

On Tue, Aug 18, 2015 at 4:25 PM, Philip Gollucci 
wrote:

> We do not run 32bit.
>
> On Tue, Aug 18, 2015 at 4:15 PM, FreeBSD Errata Notices <
> errata-noti...@freebsd.org> wrote:
>
>> -BEGIN PGP SIGNED MESSAGE-
>> Hash: SHA512
>>
>>
>> =
>> FreeBSD-EN-15:12.netstatErrata
>> Notice
>>   The FreeBSD
>> Project
>>
>> Topic:  Incorrect netstat(1) data handling on 32-bit systems
>>
>> Category:   core
>> Module: netstat
>> Announced:  2015-08-18
>> Credits:Mark Johnston
>> Affects:FreeBSD 10.2-RELEASE
>> Corrected:  2015-07-31 00:21:41 UTC (stable/10, 10.2-STABLE)
>> 2015-08-18 19:30:17 UTC (releng/10.2, 10.2-RC3-p1)
>> 2015-08-18 19:30:17 UTC (releng/10.2, 10.2-RELEASE-p1)
>>
>> For general information regarding FreeBSD Errata Notices and Security
>> Advisories, including descriptions of the fields above, security
>> branches, and the following sections, please visit
>> https://security.freebsd.org/>.
>>
>> I.   Background
>>
>> The netstat(1) utility displays the contents of various network related
>> data
>> structures.
>>
>> II.  Problem Description
>>
>> The netstat(1) utility incorrectly handles reported values on 32-bit
>> systems.
>>
>> III. Impact
>>
>> Due to how netstat(1) processes IPSEC counters, the utility may produce
>> incorrect output on 32-bit systems.
>>
>> IV.  Workaround
>>
>> No workaround is available, however systems without IPSEC compiled into
>> the
>> kernel are not affected.
>>
>> V.   Solution
>>
>> Perform one of the following:
>>
>> 1) Upgrade your system to a supported FreeBSD stable or release / security
>> branch (releng) dated after the correction date.
>>
>> 2) To update your present system via a binary patch:
>>
>> Systems running a RELEASE version of FreeBSD on the i386 or amd64
>> platforms can be updated via the freebsd-update(8) utility:
>>
>> # freebsd-update fetch
>> # freebsd-update install
>>
>> 3) To update your present system via a source code patch:
>>
>> The following patches have been verified to apply to the applicable
>> FreeBSD release branches.
>>
>> a) Download the relevant patch from the location below, and verify the
>> detached PGP signature using your PGP utility.
>>
>> # fetch https://security.FreeBSD.org/patches/EN-15:12/netstat.patch
>> # fetch https://security.FreeBSD.org/patches/EN-15:12/netstat.patch.asc
>> # gpg --verify netstat.patch.asc
>>
>> b) Apply the patch.  Execute the following commands as root:
>>
>> # cd /usr/src
>> # patch < /path/to/patch
>>
>> c) Recompile the operating system using buildworld and installworld as
>> described in https://www.FreeBSD.org/handbook/makeworld.html>.
>>
>> VI.  Correction details
>>
>> The following list contains the correction revision numbers for each
>> affected branch.
>>
>> Branch/path  Revision
>> -
>> -
>> stable/10/r286099
>> releng/10.2/  r286901
>> -
>> -
>>
>> To see which files were modified by a particular revision, run the
>> following command, replacing NN with the revision number, on a
>> machine with Subversion installed:
>>
>> # svn diff -cNN --summarize svn://svn.freebsd.org/base
>>
>> Or visit the following URL, replacing NN with the revision number:
>>
>> https://svnweb.freebsd.org/base?view=revision&revision=NN>
>>
>> VII. References
>>
>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=201700>
>>
>> The latest revision of this Errata Notice is available at
>> https://security.FreeBSD.org/advisories/FreeBSD-EN-15:12.netstat.asc
>>
>> -BEGIN PGP SIGNATURE-
>> Version: GnuPG v2.1.7 (FreeBSD)
>>
>> iQIcBAEBCgAGBQJV05A1AAoJEO1n7NZdz2rnAzIQANyLdOOQhe9dHyAV4N5YKM3B
>> Z/86dY/KIIViVb1uzkBASBNnkHlG+mCMOQpzX2x8yCPF4i7bIEfPa4r2Bzw9pvCF
>> RWRKvZXERESh/RndQFhxcmJMAyYPq7MdK0IZzG53vinlMoSz2WTKj2vSR7t2jfo+
>> ObTfgdkqN/PZs/W+AQY8a4DMdxCLg1KCZiSpQRO7ea+4AxsI8qNgoytvG6HRno/z
>> uGe6Ad82ZfysKgqe9JO4gvRTR77ebQAVSSr3qylQcOGHohy9tFHcI2FEAAqLJrQY
>> b5DDLOawLRsQm0hwkLCTOZX2QvIFgz0gGRpvPcN9ZKValMc5DKQv36z3hOByK+3i
>> dDHFG/Diy2JNP0tsKtW8IyyLvW2DAUoTs1nVaWMvLKkMUr+loOYvoaLdGT0xQP2d
>> M6UT40mRMznfH/Gq/0DJFArsYcyB9YRl7rD0dy1HhqApogHQrTjsT+1vtBtpaTmv
>> LHA77tHyzI0TxOvmx3hglj/z4BLZDPU6ydXr3zeOYBpLz5p02GKxHUc+JrmWBfOV
>> Jep0+Fr2fYST5bGVtExNQV6cTlBZPnGR4JxJEUQA6a+FdyJcDuzOTNcs0YzwjuSC
>> dIk5pdxI3nkc+zf9GZLXUdLcxXfo6jBUy0fSWkzirGzBfo0wxseE6cbxxTH7vumx
>> CLGGmHiqxVuF/nP4ScHi
>> =3aK1
>> -END PGP SIGNATURE-
>> ___
>> freebsd-annou...@freebsd.org mailing list
>> htt

Re: ix(intel) vs mlxen(mellanox) 10Gb performance

[Rick Macklem]

Hans Petter Selasky wrote:
> On 08/18/15 14:53, Rick Macklem wrote:
> > 2572 ifp->if_hw_tsomax = 65518;
> >> >2573   ifp->if_hw_tsomaxsegcount = 
> >> >IXGBE_82599_SCATTER;
> >> >2574   ifp->if_hw_tsomaxsegsize = 2048;
> 
> Hi,
> 
> If IXGBE_82599_SCATTER is the maximum scatter/gather entries the
> hardware can do, remember to subtract one fragment for the TCP/IP-header
> mbuf!
> 
Ouch! Yes, I now see that the code that counts the # of mbufs is before the
code that adds the tcp/ip header mbuf.

In my opinion, this should be fixed by setting if_hw_tsomaxsegcount to whatever
the driver provides - 1. It is not the driver's responsibility to know if a 
tcp/ip
header mbuf will be added and is a lot less confusing that expecting the driver
author to know to subtract one. (I had mistakenly thought that tcp_output() had
added the tc/ip header mbuf before the loop that counts mbufs in the list. Btw,
this tcp/ip header mbuf also has leading space for the MAC layer header.)

> I think there is an off-by-one here:
> 
> ifp->if_hw_tsomax = 65518;
> ifp->if_hw_tsomaxsegcount = IXGBE_82599_SCATTER - 1;
> ifp->if_hw_tsomaxsegsize = 2048;
> 
> Refer to:
> 
> >  *
> >  * NOTE: The TSO limits only apply to the data payload part of
> >  * a TCP/IP packet. That means there is no need to subtract
> >  * space for ethernet-, vlan-, IP- or TCP- headers from the
> >  * TSO limits unless the hardware driver in question requires
> >  * so.
> 
This comment suggests that the driver author doesn't need to do this.

However, unless this is fixed in tcp_output(), the above patch should be
applied to the driver.
> In sys/net/if_var.h
> 
> Thank you!
> 
> --HPS
> 
The problem I see is that, after doing the calculation of how many mbufs can
be in the TSO segment, the code in tcp_output() will have calculated a value
for "len" that will always be less that "tp->t_maxopd - optlen" when the
if_hw_tsosegcount limit has been hit (see where it does a "break;" out of
the while loop).
--> This does not imply "too many small fragments" for NFS, just that the
driver's transmit segment limit has been reached, where most of them
are mbuf clusters, but not the first ones.
As such the code:
/*
 * In case there are too many small fragments
 * don't use TSO:
 */
if (len <= max_len) {
len = max_len;
sendalot = 1;
tso = 0;
}
Will always happen for this case and "tso" gets set to 0. Not what we want to
happen, imho.
The above code block was what I suggested should be commented out or deleted
for the test.

It appears you should also add the "- 1" in the driver sys/dev/ixgbe/if_ix.c.

rick

> ___
> freebsd-...@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
> 
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ix(intel) vs mlxen(mellanox) 10Gb performance

[Rick Macklem]

Hans Petter Selasky wrote:
> On 08/18/15 14:53, Rick Macklem wrote:
> > If this is just a test machine, maybe you could test with these lines (at
> > about #880)
> > in sys/netinet/tcp_output.c commented out? (It looks to me like this will
> > disable TSO
> > for almost all the NFS writes.)
> > - around line #880 in sys/netinet/tcp_output.c:
> > /*
> >  * In case there are too many small fragments
> >  * don't use TSO:
> >  */
> > if (len <= max_len) {
> > len = max_len;
> > sendalot = 1;
> > tso = 0;
> > }
> >
> > This was added along with the other stuff that did the
> > if_hw_tsomaxsegcount, etc and I
> > never noticed it until now (not my patch).
> 
> FYI:
> 
> These lines are needed by other hardware, like the mlxen driver. If you
> remove them mlxen will start doing m_defrag(). I believe if you set the
> correct parameters in the "struct ifnet" for the TSO size/count limits
> this problem will go away. If you print the "len" and "max_len" and also
> the cases where TSO limits are reached, you'll see what parameter is
> triggering it and needs to be increased.
> 
Well, if the driver isn't setting if_hw_tsomaxsegcount correctly, then it
is the driver that needs to be fixed.
Having the above code block disable TSO for all of the NFS writes, including
the ones that set if_hw_tsomaxsegcount correctly doesn't make sense to me.
If the driver authors don't set these, the drivers do lots of m_defrag()
calls. I have posted more than once to freebsd-net@ asking the driver authors
to set these and some now have. (I can't do it, because I don't have the
hardware to test it with.)

I do think that most/all of them don't subtract 1 for the tcp/ip header and
I don't think they should be expected to, since the driver isn't supposed to
worry about the protocol at that level.
--> I think tcp_output() should subtract one from the if_hw_tsomaxsegcount
provided by the driver to handle this, since it chooses to count mbufs
(the while() loop at around line #825 in sys/netinet/tcp_output.c.)
before it prepends the tcp/ip header mbuf.

rick

> --HPS
> 
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Panic [page fault] in _ieee80211_crypto_delkey(): stable/10/amd64 @r286878

[David Wolfskill]

I was minding my own business in a staff meeting this afternoon, and my
laptop rebooted; seems it got a panic.  I've copied the core.txt.0 file
to , along with a
verbose dmesg.boot from this morning and output of "pciconf -l -v".

This was running:
FreeBSD localhost 10.2-STABLE FreeBSD 10.2-STABLE #122  
r286878M/286880:1002500: Tue Aug 18 04:06:33 PDT 2015 
r...@g1-252.catwhisker.org:/common/S1/obj/usr/src/sys/CANARY  amd64

Excerpts from core.txt.0:

panic: page fault
...
Unread portion of the kernel message buffer:
panic: page fault
cpuid = 2
KDB: stack backtrace:
#0 0x80946e00 at kdb_backtrace+0x60
#1 0x8090a9e6 at vpanic+0x126
#2 0x8090a8b3 at panic+0x43
#3 0x80c8467b at trap_fatal+0x36b
#4 0x80c8497d at trap_pfault+0x2ed
#5 0x80c8401a at trap+0x47a
#6 0x80c6a1b2 at calltrap+0x8
#7 0x809eff5e at ieee80211_crypto_delkey+0x1e
#8 0x80a04d45 at ieee80211_ioctl_delkey+0x65
#9 0x80a03bd2 at ieee80211_ioctl_set80211+0x572
#10 0x80a2c323 at in_control+0x203
#11 0x809cd57b at ifioctl+0x15eb
#12 0x8095ecf5 at kern_ioctl+0x255
#13 0x8095e9f0 at sys_ioctl+0x140
#14 0x80c84f97 at amd64_syscall+0x357
#15 0x80c6a49b at Xfast_syscall+0xfb
Uptime: 9h45m0s
...
Loaded symbols for /usr/local/modules/rtc.ko
#0  doadump (textdump=) at pcpu.h:219
219 pcpu.h: No such file or directory.
in pcpu.h
(kgdb) #0  doadump (textdump=) at pcpu.h:219
#1  0x8090a642 in kern_reboot (howto=260)
at /usr/src/sys/kern/kern_shutdown.c:451
#2  0x8090aa25 in vpanic (fmt=, 
ap=) at /usr/src/sys/kern/kern_shutdown.c:758
#3  0x8090a8b3 in panic (fmt=0x0)
at /usr/src/sys/kern/kern_shutdown.c:687
#4  0x80c8467b in trap_fatal (frame=, 
eva=) at /usr/src/sys/amd64/amd64/trap.c:851
#5  0x80c8497d in trap_pfault (frame=0xfe060d88b510, 
usermode=) at /usr/src/sys/amd64/amd64/trap.c:674
#6  0x80c8401a in trap (frame=0xfe060d88b510)
at /usr/src/sys/amd64/amd64/trap.c:440
#7  0x80c6a1b2 in calltrap ()
at /usr/src/sys/amd64/amd64/exception.S:236
#8  0x809f003a in _ieee80211_crypto_delkey ()
at /usr/src/sys/net80211/ieee80211_crypto.c:105
#9  0x809eff5e in ieee80211_crypto_delkey (vap=0xfe03d907, 
key=0xfe03d9070800) at /usr/src/sys/net80211/ieee80211_crypto.c:461
#10 0x80a04d45 in ieee80211_ioctl_delkey (vap=0xfe03d907, 
ireq=)
at /usr/src/sys/net80211/ieee80211_ioctl.c:1252
#11 0x80a03bd2 in ieee80211_ioctl_set80211 ()
at /usr/src/sys/net80211/ieee80211_ioctl.c:2814
#12 0x80a2c323 in in_control (so=, 
cmd=9214790412651315593, data=0xfe060d88bb80 "", ifp=0x3, 
td=) at /usr/src/sys/netinet/in.c:308
#13 0x809cd57b in ifioctl (so=0xfe03d9070800, cmd=2149607914, 
data=0xfe060d88b8e0 "wlan0", td=0xf80170abb940)
at /usr/src/sys/net/if.c:2770
#14 0x8095ecf5 in kern_ioctl (td=0xf80170abb940, 
fd=, com=18446741891212314624) at file.h:320
#15 0x8095e9f0 in sys_ioctl (td=0xf80170abb940, 
uap=0xfe060d88ba40) at /usr/src/sys/kern/sys_generic.c:718
#16 0x80c84f97 in amd64_syscall (td=0xf80170abb940, traced=0)
at subr_syscall.c:134
#17 0x80c6a49b in Xfast_syscall ()
at /usr/src/sys/amd64/amd64/exception.S:396
#18 0x0008012a2f9a in ?? ()
Previous frame inner to this frame (corrupt stack?)
Current language:  auto; currently minimal
(kgdb) 


Physical 802.11 hardware is iwn(4).

I can copy the vmcore.0 file itself after I get home -- it's ~625MB,
and I'd rather not try to get that through over a WAN before I need
to catch the shuttle to get home. :-}

Peace,
david
-- 
David H. Wolfskill  da...@catwhisker.org
Those who would murder in the name of God or prophet are blasphemous cowards.

See http://www.catwhisker.org/~david/publickey.gpg for my public key.


pgpWb4UItbSf2.pgp
Description: PGP signature

Re: ix(intel) vs mlxen(mellanox) 10Gb performance

[Rick Macklem]

Daniel Braniss wrote:
> 
> > On Aug 18, 2015, at 12:49 AM, Rick Macklem  wrote:
> > 
> > Daniel Braniss wrote:
> >> 
> >>> On Aug 17, 2015, at 3:21 PM, Rick Macklem  wrote:
> >>> 
> >>> Daniel Braniss wrote:
>  
> > On Aug 17, 2015, at 1:41 PM, Christopher Forgeron
> > 
> > wrote:
> > 
> > FYI, I can regularly hit 9.3 Gib/s with my Intel X520-DA2's and FreeBSD
> > 10.1. Before 10.1 it was less.
> > 
>  
>  this is NOT iperf/3 where i do get close to wire speed,
>  it’s NFS writes, i.e., almost real work :-)
>  
> > I used to tweak the card settings, but now it's just stock. You may
> > want
> > to
> > check your settings, the Mellanox may just have better defaults for
> > your
> > switch.
> > 
> >>> Have you tried disabling TSO for the Intel? With TSO enabled, it will be
> >>> copying
> >>> every transmitted mbuf chain to a new chain of mbuf clusters via.
> >>> m_defrag() when
> >>> TSO is enabled. (Assuming you aren't an 82598 chip. Most seem to be the
> >>> 82599 chip
> >>> these days?)
> >>> 
Oops, I think I screwed up. It looks like t_maxopd is limited to somewhat less
than the mtu.

If that is the case, the code block wouldn't do what I thought it would do.

However, if_hw_tsomaxsegcount does need to be one less than the limit for the
driver, since the tcp/ip header isn't yet prepended when it is counted.

I think the code in tcp_output() should subtract 1, but you can change it in
the driver to test this.

Thanks for doing this, rick

> >> 
> >> hi Rick
> >> 
> >> how can i check the chip?
> >> 
> > Haven't a clue. Does "dmesg" tell you? (To be honest, since disabling TSO
> > helped,
> > I'll bet you don't have a 82598.)
> > 
> >>> This has been fixed in the driver very recently, but those fixes won't be
> >>> in 10.1.
> >>> 
> >>> rick
> >>> ps: If you could test with 10.2, it would be interesting to see how the
> >>> ix
> >>> does with
> >>>   the current driver fixes in it?
> >> 
> >> I new TSO was involved!
> >> ok, firstly, it’s 10.2 stable.
> >> with TSO enabled, ix is bad, around 64MGB/s.
> >> disabling TSO it’s better, around 130
> >> 
> > Hmm, could you check to see of these lines are in sys/dev/ixgbe/if_ix.c at
> > around
> > line#2500?
> >  /* TSO parameters */
> > 2572 ifp->if_hw_tsomax = 65518;
> > 2573 ifp->if_hw_tsomaxsegcount = 
> > IXGBE_82599_SCATTER;
> > 2574 ifp->if_hw_tsomaxsegsize = 2048;
> > 
> > They are in stable/10. I didn't look at releng/10.2. (And if they're in a
> > #ifdef
> > for FreeBSD11, take the #ifdef away.)
> > If they are there and not ifdef'd, I can't explain why disabling TSO would
> > help.
> > Once TSO is fixed so that it handles the 64K transmit segments without
> > copying all
> > the mbufs, I suspect you might get better perf. with it enabled?
> > 
> 
> this is 10.2 :
> they are on lines  2509-2511 and I don’t see any #ifdefs around it.
> 
> the plot thickens :-)
> 
> danny
> 
> > Good luck with it, rick
> > 
> >> still, mlxen0 is about 250! with and without TSO
> >> 
> >> 
> >>> 
> > On Mon, Aug 17, 2015 at 6:41 AM, Slawa Olhovchenkov  > > wrote:
> > On Mon, Aug 17, 2015 at 10:27:41AM +0300, Daniel Braniss wrote:
> > 
> >> hi,
> >> I have a host (Dell R730) with both cards, connected to an HP8200
> >> switch at 10Gb.
> >> when writing to the same storage (netapp) this is what I get:
> >> ix0:~130MGB/s
> >> mlxen0  ~330MGB/s
> >> this is via nfs/tcpv3
> >> 
> >> I can get similar (bad) performance with the mellanox if I
> >> increase
> >> the file size
> >> to 512MGB.
> > 
> > Look like mellanox have internal beffer for caching and do ACK
> > acclerating.
> > 
> >> so at face value, it seems the mlxen does a better use of
> >> resources
> >> than the intel.
> >> Any ideas how to improve ix/intel's performance?
> > 
> > Are you sure about netapp performance?
> > ___
> > freebsd-...@freebsd.org  mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-net
> > 
> > To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org
> > "
> > 
>  
>  ___
>  freebsd-stable@freebsd.org mailing list
>  https://lists.freebsd.org/mailman/listinfo/freebsd-stable
>  To unsubscribe, send any mail to
>  "freebsd-stable-unsubscr...@freebsd.org"
> >> 
> >> ___
> >> freebsd-stable@freebsd.org mailing list
> >> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> >> To unsubscribe, send any