ix(intel) vs mlxen(mellanox) 10Gb performance

[Daniel Braniss]

hi,
I have a host (Dell R730) with both cards, connected to an HP8200 
switch at 10Gb.
when writing to the same storage (netapp) this is what I get:
ix0:~130MGB/s
mlxen0  ~330MGB/s
this is via nfs/tcpv3

I can get similar (bad) performance with the mellanox if I increase the 
file size
to 512MGB.
so at face value, it seems the mlxen does a better use of resources 
than the intel.
Any ideas how to improve ix/intel’s performance?

cheers,
dnny

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ix(intel) vs mlxen(mellanox) 10Gb performance

[Slawa Olhovchenkov]

On Mon, Aug 17, 2015 at 10:27:41AM +0300, Daniel Braniss wrote:

> hi,
>   I have a host (Dell R730) with both cards, connected to an HP8200 
> switch at 10Gb.
>   when writing to the same storage (netapp) this is what I get:
>   ix0:~130MGB/s
>   mlxen0  ~330MGB/s
>   this is via nfs/tcpv3
> 
>   I can get similar (bad) performance with the mellanox if I increase the 
> file size
>   to 512MGB.

Look like mellanox have internal beffer for caching and do ACK acclerating.

>   so at face value, it seems the mlxen does a better use of resources 
> than the intel.
>   Any ideas how to improve ix/intel's performance?

Are you sure about netapp performance?
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ix(intel) vs mlxen(mellanox) 10Gb performance

[Daniel Braniss]

> On Aug 17, 2015, at 12:41 PM, Slawa Olhovchenkov  wrote:
> 
> On Mon, Aug 17, 2015 at 10:27:41AM +0300, Daniel Braniss wrote:
> 
>> hi,
>>  I have a host (Dell R730) with both cards, connected to an HP8200 
>> switch at 10Gb.
>>  when writing to the same storage (netapp) this is what I get:
>>  ix0:~130MGB/s
>>  mlxen0  ~330MGB/s
>>  this is via nfs/tcpv3
>> 
>>  I can get similar (bad) performance with the mellanox if I increase the 
>> file size
>>  to 512MGB.
> 
> Look like mellanox have internal beffer for caching and do ACK acclerating.
what ever they are doing, it’s impressive :-)

> 
>>  so at face value, it seems the mlxen does a better use of resources 
>> than the intel.
>>  Any ideas how to improve ix/intel's performance?
> 
> Are you sure about netapp performance?

yes, and why should it act differently if the request is coming from the same 
host? in any case
the numbers are quiet consistent since I have measured it from several hosts, 
and at different times.

danny

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ix(intel) vs mlxen(mellanox) 10Gb performance

[Christopher Forgeron]

FYI, I can regularly hit 9.3 Gib/s with my Intel X520-DA2's and FreeBSD
10.1. Before 10.1 it was less.

I used to tweak the card settings, but now it's just stock. You may want to
check your settings, the Mellanox may just have better defaults for your
switch.

On Mon, Aug 17, 2015 at 6:41 AM, Slawa Olhovchenkov  wrote:

> On Mon, Aug 17, 2015 at 10:27:41AM +0300, Daniel Braniss wrote:
>
> > hi,
> >   I have a host (Dell R730) with both cards, connected to an HP8200
> switch at 10Gb.
> >   when writing to the same storage (netapp) this is what I get:
> >   ix0:~130MGB/s
> >   mlxen0  ~330MGB/s
> >   this is via nfs/tcpv3
> >
> >   I can get similar (bad) performance with the mellanox if I
> increase the file size
> >   to 512MGB.
>
> Look like mellanox have internal beffer for caching and do ACK acclerating.
>
> >   so at face value, it seems the mlxen does a better use of
> resources than the intel.
> >   Any ideas how to improve ix/intel's performance?
>
> Are you sure about netapp performance?
> ___
> freebsd-...@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
>
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ix(intel) vs mlxen(mellanox) 10Gb performance

[Daniel Braniss]

> On Aug 17, 2015, at 1:41 PM, Christopher Forgeron  
> wrote:
> 
> FYI, I can regularly hit 9.3 Gib/s with my Intel X520-DA2's and FreeBSD 10.1. 
> Before 10.1 it was less.
> 

this is NOT iperf/3 where i do get close to wire speed,
it’s NFS writes, i.e., almost real work :-)

> I used to tweak the card settings, but now it's just stock. You may want to 
> check your settings, the Mellanox may just have better defaults for your 
> switch. 
> 
> On Mon, Aug 17, 2015 at 6:41 AM, Slawa Olhovchenkov  > wrote:
> On Mon, Aug 17, 2015 at 10:27:41AM +0300, Daniel Braniss wrote:
> 
> > hi,
> >   I have a host (Dell R730) with both cards, connected to an HP8200 
> > switch at 10Gb.
> >   when writing to the same storage (netapp) this is what I get:
> >   ix0:~130MGB/s
> >   mlxen0  ~330MGB/s
> >   this is via nfs/tcpv3
> >
> >   I can get similar (bad) performance with the mellanox if I increase 
> > the file size
> >   to 512MGB.
> 
> Look like mellanox have internal beffer for caching and do ACK acclerating.
> 
> >   so at face value, it seems the mlxen does a better use of resources 
> > than the intel.
> >   Any ideas how to improve ix/intel's performance?
> 
> Are you sure about netapp performance?
> ___
> freebsd-...@freebsd.org  mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-net 
> 
> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org 
> "
> 

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ix(intel) vs mlxen(mellanox) 10Gb performance

[Slawa Olhovchenkov]

On Mon, Aug 17, 2015 at 01:35:06PM +0300, Daniel Braniss wrote:

> 
> > On Aug 17, 2015, at 12:41 PM, Slawa Olhovchenkov  wrote:
> > 
> > On Mon, Aug 17, 2015 at 10:27:41AM +0300, Daniel Braniss wrote:
> > 
> >> hi,
> >>I have a host (Dell R730) with both cards, connected to an HP8200 
> >> switch at 10Gb.
> >>when writing to the same storage (netapp) this is what I get:
> >>ix0:~130MGB/s
> >>mlxen0  ~330MGB/s
> >>this is via nfs/tcpv3
> >> 
> >>I can get similar (bad) performance with the mellanox if I increase the 
> >> file size
> >>to 512MGB.
> > 
> > Look like mellanox have internal beffer for caching and do ACK acclerating.
> what ever they are doing, it's impressive :-)
> 
> > 
> >>so at face value, it seems the mlxen does a better use of resources 
> >> than the intel.
> >>Any ideas how to improve ix/intel's performance?
> > 
> > Are you sure about netapp performance?
> 
> yes, and why should it act differently if the request is coming from the same 
> host? in any case
> the numbers are quiet consistent since I have measured it from several hosts, 
> and at different times.

In any case, for 10Gb expect about 1200MGB/s.
I see lesser speed.
What netapp maximum performance? From other hosts, or local, any?
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ix(intel) vs mlxen(mellanox) 10Gb performance

[Alban Hertroys]

On 17 August 2015 at 13:39, Slawa Olhovchenkov  wrote:

> In any case, for 10Gb expect about 1200MGB/s.

Your usage of units is confusing. Above you claim you expect 1200
million gigabytes per second, or 1.2 * 10^18 Bytes/s. I don't think
any known network interface can do that, including highly experimental
ones.

I suspect you intended to claim that you expect 1.2GB/s (Gigabytes per
second) over that 10Gb/s (Gigabits per second) network.
That's still on the high side of what's possible. On TCP/IP there is
some TCP overhead, so 1.0 GB/s is probably more realistic.

WRT the actual problem you're trying to solve, I'm no help there.
-- 
If you can't see the forest for the trees,
Cut the trees and you'll see there is no forest.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ix(intel) vs mlxen(mellanox) 10Gb performance

[Slawa Olhovchenkov]

On Mon, Aug 17, 2015 at 01:49:27PM +0200, Alban Hertroys wrote:

> On 17 August 2015 at 13:39, Slawa Olhovchenkov  wrote:
> 
> > In any case, for 10Gb expect about 1200MGB/s.
> 
> Your usage of units is confusing. Above you claim you expect 1200

I am use as topic starter and expect MeGaBytes per second

> million gigabytes per second, or 1.2 * 10^18 Bytes/s. I don't think
> any known network interface can do that, including highly experimental
> ones.
> 
> I suspect you intended to claim that you expect 1.2GB/s (Gigabytes per
> second) over that 10Gb/s (Gigabits per second) network.
> That's still on the high side of what's possible. On TCP/IP there is
> some TCP overhead, so 1.0 GB/s is probably more realistic.

TCP give 5-7% overhead (include retrasmits).
10^9/8*0.97 = 1.2125

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: freebsd-update to 10.2-RELEASE broken ?

[Bob Bishop]

> On 16 Aug 2015, at 23:08, Bjoern A. Zeeb  
> wrote:
> 
> 
>> On 16 Aug 2015, at 21:16 , Christian Kratzer  wrote:
>> 
>> Hi,
>> 
>> On Sun, 16 Aug 2015, Kurt Jaeger wrote:
>>> Hi!
>>> 
 It could be the classic fall back to TCP on SRV records problem on
 your upstream DNS forwarder if you're using one:
 
 http://lists.freebsd.org/pipermail/freebsd-ports/2012-May/074801.html
>>> 
>>> If I query that same DNS resolver using the line from
>>> the script, it works every time. It's a 10.1p16 host with
>>> a very recent ports build, and directly connected (no NAT, no
>>> fw etc).
>>> 
>>> If that would be the problem, how could I diagnose it in depth ?
>> 
>> freebsd-update upgrade just failed on 3 other vm even when I explicitly
>> specified the server using freebsd-update -s.
>> 
>> I had success on another vm when I changed to using google dns.
>> 
>> I am not aware that anything would be blocking tcp dns in my setups.
>> 
>> Must be something else dns related.
>> 
>> Perhaps I will run a local resolver in a vm and logg all queries and dns
>> traffic.
> 
> Or run tcpdump for port 53;   also curious if it might be an IPv4 vs. IPv6 
> issue?

I saw the issue on machines with IPv4/IPv6 and IPv4 only.

> — 
> Bjoern A. Zeeb  Charles Haddon Spurgeon:
> "Friendship is one of the sweetest joys of life.  Many might have failed
> beneath the bitterness of their trial  had they not found a friend."
> 
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


--
Bob Bishop  +44 (0)118 940 1243
r...@gid.co.ukfax +44 (0)118 940 1295
 mobile +44 (0)783 626 4518





___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: freebsd-update to 10.2-RELEASE broken ?

[Christian Kratzer]

Hi,

On Mon, 17 Aug 2015, Bob Bishop wrote:

Or run tcpdump for port 53;   also curious if it might be an IPv4 vs. IPv6 
issue?


I saw the issue on machines with IPv4/IPv6 and IPv4 only.


i have tried with preference set to ipv4 and ipv6. Both fail similarly.  Have 
not tried ipv4 or ipv6 only though.

I had success with using google dns 8.8.8.8 as revolver on two machines/VM.

But this hack has also since failed on other machines/VM.

Greetings
Christian

--
Christian Kratzer   CK Software GmbH
Email:   c...@cksoft.de   Wildberger Weg 24/2
Phone:   +49 7032 893 997 - 0   D-71126 Gaeufelden
Fax: +49 7032 893 997 - 9   HRB 245288, Amtsgericht Stuttgart
Mobile:  +49 171 1947 843   Geschaeftsfuehrer: Christian Kratzer
Web: http://www.cksoft.de/
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ix(intel) vs mlxen(mellanox) 10Gb performance

[Rick Macklem]

Daniel Braniss wrote:
> 
> > On Aug 17, 2015, at 1:41 PM, Christopher Forgeron 
> > wrote:
> > 
> > FYI, I can regularly hit 9.3 Gib/s with my Intel X520-DA2's and FreeBSD
> > 10.1. Before 10.1 it was less.
> > 
> 
> this is NOT iperf/3 where i do get close to wire speed,
> it’s NFS writes, i.e., almost real work :-)
> 
> > I used to tweak the card settings, but now it's just stock. You may want to
> > check your settings, the Mellanox may just have better defaults for your
> > switch.
> > 
Have you tried disabling TSO for the Intel? With TSO enabled, it will be copying
every transmitted mbuf chain to a new chain of mbuf clusters via. m_defrag() 
when
TSO is enabled. (Assuming you aren't an 82598 chip. Most seem to be the 82599 
chip
these days?)

This has been fixed in the driver very recently, but those fixes won't be in 
10.1.

rick
ps: If you could test with 10.2, it would be interesting to see how the ix does 
with
the current driver fixes in it?

> > On Mon, Aug 17, 2015 at 6:41 AM, Slawa Olhovchenkov  > > wrote:
> > On Mon, Aug 17, 2015 at 10:27:41AM +0300, Daniel Braniss wrote:
> > 
> > > hi,
> > >   I have a host (Dell R730) with both cards, connected to an HP8200
> > >   switch at 10Gb.
> > >   when writing to the same storage (netapp) this is what I get:
> > >   ix0:~130MGB/s
> > >   mlxen0  ~330MGB/s
> > >   this is via nfs/tcpv3
> > >
> > >   I can get similar (bad) performance with the mellanox if I increase
> > >   the file size
> > >   to 512MGB.
> > 
> > Look like mellanox have internal beffer for caching and do ACK acclerating.
> > 
> > >   so at face value, it seems the mlxen does a better use of resources
> > >   than the intel.
> > >   Any ideas how to improve ix/intel's performance?
> > 
> > Are you sure about netapp performance?
> > ___
> > freebsd-...@freebsd.org  mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-net
> > 
> > To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org
> > "
> > 
> 
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ix(intel) vs mlxen(mellanox) 10Gb performance

[Slawa Olhovchenkov]

On Mon, Aug 17, 2015 at 10:27:41AM +0300, Daniel Braniss wrote:

> hi,
>   I have a host (Dell R730) with both cards, connected to an HP8200 
> switch at 10Gb.
>   when writing to the same storage (netapp) this is what I get:
>   ix0:~130MGB/s
>   mlxen0  ~330MGB/s
>   this is via nfs/tcpv3
> 
>   I can get similar (bad) performance with the mellanox if I increase the 
> file size
>   to 512MGB.
>   so at face value, it seems the mlxen does a better use of resources 
> than the intel.
>   Any ideas how to improve ix/intel's performance?

Any way, please show

OS version
/var/run/dmesg.boot
What's tuning perfomed (loader.conf, sysctl.conf)?
top -PHS in both cases
ifconfig -a in both cases
netstat -rn in both cases
I am don't know netapp -- what is hardware configuration (disks and
etc) and software tuning (MTU?).

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: freebsd-update to 10.2-RELEASE broken ?

[Mark Martinec]

On Sun, 16 Aug 2015, Kimmo Paasiala wrote:

It could be the classic fall back to TCP on SRV records problem on
your upstream DNS forwarder if you're using one:
http://lists.freebsd.org/pipermail/freebsd-ports/2012-May/074801.html

The cure would be to use your own caching DNS resolver (configured to
query the authoritative name servers directly) such as dns/unbound.



2015-08-16 Christian Kratzer wrote:

I run my own bind9 resolvers on freebsd 10 at both sites.   I never
particurlarly like the concept of an "upstream" resolver.

All my resolvers are behind firewalls although different kinds.
ASA at one site and freebsd pf at the other.

I will investigate though.  Thanks for the tip.


ASA firewall has a nasty setting to *discard* DNS UDP packets
with UDP message size over 512 bytes, i.e. it does not allow EDNS0
option. Check that you have this DNS deep packet inspection
misfeature turned off. Check also the firewall log.

This would affect UDP DNS responses to a SRV query
_http._tcp.update.FreeBSD.org, which comes close to the size limit
(possibly depending on geolocation). Using google's public DNS server
may avoid the problem by stripping nonessential records from the
DNS reply (like the ADDITIONAL SECTION).

  Mark
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: freebsd-update to 10.2-RELEASE broken ?

[Mark Martinec]

On Sun, 16 Aug 2015, Kimmo Paasiala wrote:

It could be the classic fall back to TCP on SRV records problem on
your upstream DNS forwarder if you're using one:
http://lists.freebsd.org/pipermail/freebsd-ports/2012-May/074801.html

The cure would be to use your own caching DNS resolver (configured to
query the authoritative name servers directly) such as dns/unbound.



2015-08-16 Christian Kratzer wrote:

I run my own bind9 resolvers on freebsd 10 at both sites.   I never
particurlarly like the concept of an "upstream" resolver.

All my resolvers are behind firewalls although different kinds.
ASA at one site and freebsd pf at the other.

I will investigate though.  Thanks for the tip.


ASA firewall has a nasty setting to *discard* DNS UDP packets
with UDP message size over 512 bytes, i.e. it does not allow EDNS0
option. Check that you have this DNS deep packet inspection
misfeature turned off. Check also the firewall log.

This would affect UDP DNS responses to a SRV query
_http._tcp.update.FreeBSD.org, which comes close to the size limit
(possibly depending on geolocation). Using google's public DNS server
may avoid the problem by stripping nonessential records from the
DNS reply (like the ADDITIONAL SECTION).

  Mark
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ix(intel) vs mlxen(mellanox) 10Gb performance

[Daniel Braniss]

> On Aug 17, 2015, at 3:21 PM, Rick Macklem  wrote:
> 
> Daniel Braniss wrote:
>> 
>>> On Aug 17, 2015, at 1:41 PM, Christopher Forgeron 
>>> wrote:
>>> 
>>> FYI, I can regularly hit 9.3 Gib/s with my Intel X520-DA2's and FreeBSD
>>> 10.1. Before 10.1 it was less.
>>> 
>> 
>> this is NOT iperf/3 where i do get close to wire speed,
>> it’s NFS writes, i.e., almost real work :-)
>> 
>>> I used to tweak the card settings, but now it's just stock. You may want to
>>> check your settings, the Mellanox may just have better defaults for your
>>> switch.
>>> 
> Have you tried disabling TSO for the Intel? With TSO enabled, it will be 
> copying
> every transmitted mbuf chain to a new chain of mbuf clusters via. m_defrag() 
> when
> TSO is enabled. (Assuming you aren't an 82598 chip. Most seem to be the 82599 
> chip
> these days?)
> 

hi Rick

how can i check the chip?

> This has been fixed in the driver very recently, but those fixes won't be in 
> 10.1.
> 
> rick
> ps: If you could test with 10.2, it would be interesting to see how the ix 
> does with
>the current driver fixes in it?

I new TSO was involved! 
ok, firstly, it’s 10.2 stable.
with TSO enabled, ix is bad, around 64MGB/s.
disabling TSO it’s better, around 130

still, mlxen0 is about 250! with and without TSO


> 
>>> On Mon, Aug 17, 2015 at 6:41 AM, Slawa Olhovchenkov >> > wrote:
>>> On Mon, Aug 17, 2015 at 10:27:41AM +0300, Daniel Braniss wrote:
>>> 
 hi,
  I have a host (Dell R730) with both cards, connected to an HP8200
  switch at 10Gb.
  when writing to the same storage (netapp) this is what I get:
  ix0:~130MGB/s
  mlxen0  ~330MGB/s
  this is via nfs/tcpv3
 
  I can get similar (bad) performance with the mellanox if I increase
  the file size
  to 512MGB.
>>> 
>>> Look like mellanox have internal beffer for caching and do ACK acclerating.
>>> 
  so at face value, it seems the mlxen does a better use of resources
  than the intel.
  Any ideas how to improve ix/intel's performance?
>>> 
>>> Are you sure about netapp performance?
>>> ___
>>> freebsd-...@freebsd.org  mailing list
>>> https://lists.freebsd.org/mailman/listinfo/freebsd-net
>>> 
>>> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org
>>> "
>>> 
>> 
>> ___
>> freebsd-stable@freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
>> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ix(intel) vs mlxen(mellanox) 10Gb performance

[Alban Hertroys]

On 17 August 2015 at 13:54, Slawa Olhovchenkov  wrote:
> On Mon, Aug 17, 2015 at 01:49:27PM +0200, Alban Hertroys wrote:
>
>> On 17 August 2015 at 13:39, Slawa Olhovchenkov  wrote:
>>
>> > In any case, for 10Gb expect about 1200MGB/s.
>>
>> Your usage of units is confusing. Above you claim you expect 1200
>
> I am use as topic starter and expect MeGaBytes per second

That's a highly unusual way of writing MB/s.

There are standards for unit prefixes: k means kilo, M means Mega, G
means Giga, etc. See:
https://en.wikipedia.org/wiki/International_System_of_Units#Prefixes

>> million gigabytes per second, or 1.2 * 10^18 Bytes/s. I don't think
>> any known network interface can do that, including highly experimental
>> ones.
>>
>> I suspect you intended to claim that you expect 1.2GB/s (Gigabytes per
>> second) over that 10Gb/s (Gigabits per second) network.
>> That's still on the high side of what's possible. On TCP/IP there is
>> some TCP overhead, so 1.0 GB/s is probably more realistic.
>
> TCP give 5-7% overhead (include retrasmits).
> 10^9/8*0.97 = 1.2125

In information science, Bytes are counted in multiples of 2, not 10. A
kb is 1024 bits or 2^10 b. So 10 Gb is 10 * 2^30 bits.

It's also not unusual to be more specific about that 2-base and use
kib, Mib and Gib instead.

Apparently you didn't know that...

Also, if you take 5% off, you are left with (0.95 * 10 * 2^30) / 8 =
1.1875 B/s, not 0.97 * ... Your calculations were a bit optimistic.

Now I have to admit I'm used to use a factor of 10 to convert from b/s
to B/s (that's 20%!), but that's probably no longer correct, what with
jumbo frames and all.

-- 
If you can't see the forest for the trees,
Cut the trees and you'll see there is no forest.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: freebsd-update to 10.2-RELEASE broken ?

[Slawa Olhovchenkov]

On Sun, Aug 16, 2015 at 07:16:03PM +0200, Christian Kratzer wrote:

> Hi,
> 
> I have been trying to update several of my FreeBSD 10.1 amd64 VM to 
> 10.2-RELEASE with freebsd-update and have been failing with an incorrect hash 
> error.
> 
> This is what happens with a plain vanilla 10.1-RELEASE vm when I try to 
> update to 10.2-RELEASE
> 
> Fetching 2356 files... 
> 068eb594e5f6b97554750a8321892e4c229a6f26455f40e4d8e4e7a79577c673 has 
> incorrect hash.
> root@test10:~ck #
> --snipp--
> 
> I get the is on all kinds of VM with different patchlevels of 10.1-RELEASE.  
> Some of them have /usr/src, some of them don't.
> 
> The hash seems to be different every time round.
> 
> Could this be an issue with one of the mirrors ?
> 
> Has anybody had success yet with an update to 10.2-RELEASE using 
> freebsd-update ?

This is like update.FreeBSD.org in some time give random files.
For examle:

=
Applying patches... done.
Fetching 10832 files... 
136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7 has incorrect 
hash.
[00:09:19] >> Error: Fail to upgrade system

# gzip -cd 
/poudriere/jails/i386/var/db/freebsd-update/136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.gz
 > 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.bad
# fetch 
http://update.FreeBSD.org/10.2-RELEASE/i386/f/136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.gz
# gzip -cd 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.gz 
> 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.god
# ls -ltr 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7*
-rw-r--r--  1 root  wheel   713 Aug 13 03:24 
136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.gz
-rw-r--r--  1 root  wheel  4960 Aug 17 18:37 
136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.bad
-rw-r--r--  1 root  wheel  1278 Aug 17 18:38 
136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.god
# cat 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.god
/*
 * Copyright 2013 Eukr'ea Electromatique 
 *
 * This program is free software; you can redistribute it and/or
 * modify it under the terms of the GNU General Public License
 * as published by the Free Software Foundation; either version 2
 * of the License, or (at your option) any later version.
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 */

#include "imx25-eukrea-mbimxsd25-baseboard.dts"

/ {
model = "Eukrea MBIMXSD25 with the DVI-VGA Display";
[...]
# cat 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.bad
/*
 * Copyright (C) 2014 Thomas Petazzoni 
 * Copyright (C) 2009 Simon Guinot 
 *
 * This file is licensed under the terms of the GNU General Public
 * License version 2. This program is licensed "as is" without any
 * warranty of any kind, whether express or implied.
 */

/dts-v1/;

#include 
#include 
#include 
#include "orion5x-mv88f5182.dtsi"

/ {
model = "LaCie d2 Network";
compatible = "lacie,d2-network", "marvell,orion5x-88f5182", 
"marvell,orion5x";
[...]
==

secteam?
admins?
webmaster?

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: freebsd-update to 10.2-RELEASE broken ?

[Glen Barber]

On Mon, Aug 17, 2015 at 06:50:22PM +0300, Slawa Olhovchenkov wrote:
> On Sun, Aug 16, 2015 at 07:16:03PM +0200, Christian Kratzer wrote:
> 
> > Hi,
> > 
> > I have been trying to update several of my FreeBSD 10.1 amd64 VM to 
> > 10.2-RELEASE with freebsd-update and have been failing with an incorrect 
> > hash error.
> > 
> > This is what happens with a plain vanilla 10.1-RELEASE vm when I try to 
> > update to 10.2-RELEASE
> > 
> > Fetching 2356 files... 
> > 068eb594e5f6b97554750a8321892e4c229a6f26455f40e4d8e4e7a79577c673 has 
> > incorrect hash.
> > root@test10:~ck #
> > --snipp--
> > 
> > I get the is on all kinds of VM with different patchlevels of 10.1-RELEASE. 
> >  Some of them have /usr/src, some of them don't.
> > 
> > The hash seems to be different every time round.
> > 
> > Could this be an issue with one of the mirrors ?
> > 
> > Has anybody had success yet with an update to 10.2-RELEASE using 
> > freebsd-update ?
> 
> This is like update.FreeBSD.org in some time give random files.
> For examle:
> 
> =
> Applying patches... done.
> Fetching 10832 files... 
> 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7 has 
> incorrect hash.
> [00:09:19] >> Error: Fail to upgrade system
> 
> # gzip -cd 
> /poudriere/jails/i386/var/db/freebsd-update/136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.gz
>  > 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.bad
> # fetch 
> http://update.FreeBSD.org/10.2-RELEASE/i386/f/136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.gz
> # gzip -cd 
> 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.gz > 
> 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.god
> # ls -ltr 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7*
> -rw-r--r--  1 root  wheel   713 Aug 13 03:24 
> 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.gz
> -rw-r--r--  1 root  wheel  4960 Aug 17 18:37 
> 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.bad
> -rw-r--r--  1 root  wheel  1278 Aug 17 18:38 
> 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.god
> # cat 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.god
> /*
>  * Copyright 2013 Eukr'ea Electromatique 
>  *
>  * This program is free software; you can redistribute it and/or
>  * modify it under the terms of the GNU General Public License
>  * as published by the Free Software Foundation; either version 2
>  * of the License, or (at your option) any later version.
>  * This program is distributed in the hope that it will be useful,
>  * but WITHOUT ANY WARRANTY; without even the implied warranty of
>  * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>  * GNU General Public License for more details.
>  */
> 
> #include "imx25-eukrea-mbimxsd25-baseboard.dts"
> 
> / {
> model = "Eukrea MBIMXSD25 with the DVI-VGA Display";
> [...]
> # cat 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.bad
> /*
>  * Copyright (C) 2014 Thomas Petazzoni 
>  * Copyright (C) 2009 Simon Guinot 
>  *
>  * This file is licensed under the terms of the GNU General Public
>  * License version 2. This program is licensed "as is" without any
>  * warranty of any kind, whether express or implied.
>  */
> 
> /dts-v1/;
> 
> #include 
> #include 
> #include 
> #include "orion5x-mv88f5182.dtsi"
> 
> / {
> model = "LaCie d2 Network";
> compatible = "lacie,d2-network", "marvell,orion5x-88f5182", 
> "marvell,orion5x";
> [...]
> ==
> 
> secteam?
> admins?
> webmaster?

Secteam.  I've cc'd them.

Glen



pgpI7_7kiQ0UH.pgp
Description: PGP signature

Re: ix(intel) vs mlxen(mellanox) 10Gb performance

[Slawa Olhovchenkov]

On Mon, Aug 17, 2015 at 05:44:37PM +0200, Alban Hertroys wrote:

> On 17 August 2015 at 13:54, Slawa Olhovchenkov  wrote:
> > On Mon, Aug 17, 2015 at 01:49:27PM +0200, Alban Hertroys wrote:
> >
> >> On 17 August 2015 at 13:39, Slawa Olhovchenkov  wrote:
> >>
> >> > In any case, for 10Gb expect about 1200MGB/s.
> >>
> >> Your usage of units is confusing. Above you claim you expect 1200
> >
> > I am use as topic starter and expect MeGaBytes per second
> 
> That's a highly unusual way of writing MB/s.

I am know. This is do not care for me.

> There are standards for unit prefixes: k means kilo, M means Mega, G
> means Giga, etc. See:
> https://en.wikipedia.org/wiki/International_System_of_Units#Prefixes
> 
> >> million gigabytes per second, or 1.2 * 10^18 Bytes/s. I don't think
> >> any known network interface can do that, including highly experimental
> >> ones.
> >>
> >> I suspect you intended to claim that you expect 1.2GB/s (Gigabytes per
> >> second) over that 10Gb/s (Gigabits per second) network.
> >> That's still on the high side of what's possible. On TCP/IP there is
> >> some TCP overhead, so 1.0 GB/s is probably more realistic.
> >
> > TCP give 5-7% overhead (include retrasmits).
> > 10^9/8*0.97 = 1.2125
> 
> In information science, Bytes are counted in multiples of 2, not 10. A
> kb is 1024 bits or 2^10 b. So 10 Gb is 10 * 2^30 bits.

Interface speeds counted in multile of 10.
10Mbit ethernet have speed 10^7 bit/s.
64Kbit ISDN have speed 64000, not 65536.

> It's also not unusual to be more specific about that 2-base and use
> kib, Mib and Gib instead.
> 
> Apparently you didn't know that...
> 
> Also, if you take 5% off, you are left with (0.95 * 10 * 2^30) / 8 =
> 1.1875 B/s, not 0.97 * ... Your calculations were a bit optimistic.

May bug.
10^10/8*0.93 = 116250 = 1162.5

> Now I have to admit I'm used to use a factor of 10 to convert from b/s
> to B/s (that's 20%!), but that's probably no longer correct, what with
> jumbo frames and all.

Ok, may be topic started use software metered speed with MGBs as
1048576 per second. 116250/1048576 = 1108.64
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: CARp comatibility between 9 and 10

[Damien Fleuriot]

On 8 June 2015 at 12:13, Pete French  wrote:

> > I did this a while ago - and it actually worked.
> > (CARP between 9 and 10).
> >
> > I think I have each IP assigned its own VHID, though.
>
> Thanks for this, someone else has confimred that 10 orks with multiple IP's
> per VHID, so I am good to go.
>
> Do I still need to compile a separate kernel with pfsync and carp in
> it, or can these now be loaded as modules ? I cant remember when I started
> doing that, but I suspect it may no longer be necessary.
>
>

OK I'm late to reply to this, but a word of caution to those that would
upgrade from 8 to 10 or 9 to 10.

While CARP maintains compatibility between the two different OS releases,
you will lose pfsync capability.

That means when you switch over your CARP master, PF will drop all these
sessions that existed at host A but were not synched over to host B.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

[POSSIBLE BUG] 10-STABLE CARP erroneously becomes master on boot

[Damien Fleuriot]

Hello list,



I'm seeing this very peculiar behaviour between 2 10-STABLE boxes.

Host A is CARP Master with advskew 20 and runs 10.2-BETA1 from 10/07
Host B is CARP Backup with advskew 150 and runs 10.2-PRERELEASE from 12/08


When I configure CARP in rc.conf on host B, it becomes Master on boot, and
host A remains Master as well.
When I force a state change on host B (ifconfig vlanx vhid y state backup),
it transitions to Backup then again to Master.

When I comment out the CARP configuration in rc.conf , and configure CARP
manually on host B's interfaces after it boots, it correctly becomes and
remains Backup.



Below is the excerpt from rc.conf pertaining to CARP configuration, the
only difference between the 2 hosts being their advskew.

Host A
== BEGIN

ifconfig_vlan410_alias0="vhid 110 pass passhere advskew 20 alias
10.104.10.251/32"

== END

Host B
== BEGIN

ifconfig_vlan410_alias0="vhid 110 pass passhere advskew 150 alias
10.104.10.251/32"

== END



Additionally, the sysctls for CARP for both boxes :

== BEGIN

net.inet.carp.ifdown_demotion_factor: 240
net.inet.carp.senderr_demotion_factor: 240
net.inet.carp.demotion: 0
net.inet.carp.log: 1
net.inet.carp.preempt: 1
net.inet.carp.allow: 1
net.pfsync.carp_demotion_factor: 240

== END



The hosts can see each other correctly, as evidenced by manually
configuring the CARP aliases on Host B after it boots, and it retaining
Backup status.


Has anyone experienced the same problem ?



I'm thinking of upgrading Host A to 10.2-PRERELEASE so they're on the same
revision level, but I'm afraid it may break CARP on boot as on Host B.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: [POSSIBLE BUG] 10-STABLE CARP erroneously becomes master on boot

[Freddie Cash]

On Aug 17, 2015 9:22 AM, "Damien Fleuriot"  wrote:
>
> Hello list,
>
>
>
> I'm seeing this very peculiar behaviour between 2 10-STABLE boxes.
>
> Host A is CARP Master with advskew 20 and runs 10.2-BETA1 from 10/07
> Host B is CARP Backup with advskew 150 and runs 10.2-PRERELEASE from 12/08
>
>
> When I configure CARP in rc.conf on host B, it becomes Master on boot, and
> host A remains Master as well.
> When I force a state change on host B (ifconfig vlanx vhid y state
backup),
> it transitions to Backup then again to Master.
>
> When I comment out the CARP configuration in rc.conf , and configure CARP
> manually on host B's interfaces after it boots, it correctly becomes and
> remains Backup.
>
>
>
> Below is the excerpt from rc.conf pertaining to CARP configuration, the
> only difference between the 2 hosts being their advskew.
>
> Host A
> == BEGIN
>
> ifconfig_vlan410_alias0="vhid 110 pass passhere advskew 20 alias
> 10.104.10.251/32"
>
> == END
>
> Host B
> == BEGIN
>
> ifconfig_vlan410_alias0="vhid 110 pass passhere advskew 150 alias
> 10.104.10.251/32"
>
> == END

Put the IP first, and the vhid stuff last in rc.conf for things to work the
most reliably. And drop the extra alias.

ifconfig_vlan410_alias0="inet 10.104.10.251/32 vhid 110 pass passhere
advskew 150"

CARP requires that all IPs on an interface that are part of the same vhid
to be listed (added) in the exact same order for the vhid to be considered
"the same". That one trips me up all the time when manually adding an IP to
a CARP pair, and then later rebooting one box as they both think they're
master for that interface, while being a mix of master/backup for the other
interfaces.

Cheers,
Freddie
(running CARP on 2 10-CURRENT boxes and 2 10.1-p13 boxes)
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: [POSSIBLE BUG] 10-STABLE CARP erroneously becomes master on boot

[Damien Fleuriot]

On 17 August 2015 at 18:32, Freddie Cash  wrote:

>
> On Aug 17, 2015 9:22 AM, "Damien Fleuriot"  wrote:
> >
> > Hello list,
> >
> >
> >
> > I'm seeing this very peculiar behaviour between 2 10-STABLE boxes.
> >
> > Host A is CARP Master with advskew 20 and runs 10.2-BETA1 from 10/07
> > Host B is CARP Backup with advskew 150 and runs 10.2-PRERELEASE from
> 12/08
> >
> >
> > When I configure CARP in rc.conf on host B, it becomes Master on boot,
> and
> > host A remains Master as well.
> > When I force a state change on host B (ifconfig vlanx vhid y state
> backup),
> > it transitions to Backup then again to Master.
> >
> > When I comment out the CARP configuration in rc.conf , and configure CARP
> > manually on host B's interfaces after it boots, it correctly becomes and
> > remains Backup.
> >
> >
> >
> > Below is the excerpt from rc.conf pertaining to CARP configuration, the
> > only difference between the 2 hosts being their advskew.
> >
> > Host A
> > == BEGIN
> >
> > ifconfig_vlan410_alias0="vhid 110 pass passhere advskew 20 alias
> > 10.104.10.251/32"
> >
> > == END
> >
> > Host B
> > == BEGIN
> >
> > ifconfig_vlan410_alias0="vhid 110 pass passhere advskew 150 alias
> > 10.104.10.251/32"
> >
> > == END
>
> Put the IP first, and the vhid stuff last in rc.conf for things to work
> the most reliably. And drop the extra alias.
>
> ifconfig_vlan410_alias0="inet 10.104.10.251/32 vhid 110 pass passhere
> advskew 150"
>
> CARP requires that all IPs on an interface that are part of the same vhid
> to be listed (added) in the exact same order for the vhid to be considered
> "the same". That one trips me up all the time when manually adding an IP to
> a CARP pair, and then later rebooting one box as they both think they're
> master for that interface, while being a mix of master/backup for the other
> interfaces.
>
> Cheers,
> Freddie
> (running CARP on 2 10-CURRENT boxes and 2 10.1-p13 boxes)
>

Cheers Freddie, will try and keep the thread up to date on the results.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: 10.2: ntp update breaks DCF77 clock

[Konstantin Belousov]

On Sun, Aug 16, 2015 at 10:49:57PM -0700, Cy Schubert wrote:
> qemu-sbruno) doesn't support all our supported platforms, especially the 
> multitude of ARM platforms, so holes in our auto-generated config.h support 
> will exist.

I believe that the userspace arm ABI is not that variable.  There could
be little/bit endian variants, and we seems to get hw-floating point
ABI.  This is definitely much less than the variations of the platform
each of which requires specific kernel.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: 10.2: ntp update breaks DCF77 clock

[Christian Weisgerber]

On 2015-08-16, Ian Lepore  wrote:

> I wonder: is there a reason to not enable all (or most of) the refclocks
> in base and in ports?

I guess it isn't very elegant to enable clock drivers if nobody
knows whether they work or whether the hardware was ever produced
in series or whether external services they rely on still exist.

As a rough guess I'd say NMEA GPS clocks are the most popular, maybe
also by way of gpsd and SHM, some RAWDCF here in Europe, a few
institutional or corporate users with MEINBERG clocks, and then
things are getting mighty thin on the ground...  Not coincidentally
this looks quite a bit like the list of clock drivers originally
enabled (r268351).

-- 
Christian "naddy" Weisgerber  na...@mips.inka.de
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Swap Questions

[Antony Uspensky]

On Sat, 15 Aug 2015, Tim Daneliuk wrote:


So -L does fix the problem - sort of.  The machine picks up the file as
additional swap on boot just fine.  HWOEVER, when I try to reboot or shut
down the host, I get a panic telling me some noise about not being able
to shutdown swap for some reason.


Try to swapoff (by hands) before shutdown.
Shutdown sequence, I think, unmounts carrying disk before swapping off 
a carried file. If I am right, -L should be processed on shutdown also.

Just a guess.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Swap Questions

[Tim Daneliuk]

On 08/17/2015 12:53 PM, Antony Uspensky wrote:
> On Sat, 15 Aug 2015, Tim Daneliuk wrote:
> 
>> So -L does fix the problem - sort of.  The machine picks up the file as
>> additional swap on boot just fine.  HWOEVER, when I try to reboot or shut
>> down the host, I get a panic telling me some noise about not being able
>> to shutdown swap for some reason.
> 
> Try to swapoff (by hands) before shutdown.
> Shutdown sequence, I think, unmounts carrying disk before swapping off a 
> carried file. If I am right, -L should be processed on shutdown also.
> Just a guess.

Yes, that did it.  

But, isn't this kind of an operational bug?  Shouldn't the shutdown logic
do the swapoff before the unmount if it sees files being used for swap?

i.e. Should I enter this as a bug report?

The only reason this matters - and it's a pretty big reason - is for production
servers when someone logs in remotely, becomes root, and issued "reboot".  The
machine hangs at the panic and never comes back ... something you do not see
unless you are in a console of some sort ...
-- 

Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ix(intel) vs mlxen(mellanox) 10Gb performance

[Rick Macklem]

Daniel Braniss wrote:
> 
> > On Aug 17, 2015, at 3:21 PM, Rick Macklem  wrote:
> > 
> > Daniel Braniss wrote:
> >> 
> >>> On Aug 17, 2015, at 1:41 PM, Christopher Forgeron 
> >>> wrote:
> >>> 
> >>> FYI, I can regularly hit 9.3 Gib/s with my Intel X520-DA2's and FreeBSD
> >>> 10.1. Before 10.1 it was less.
> >>> 
> >> 
> >> this is NOT iperf/3 where i do get close to wire speed,
> >> it’s NFS writes, i.e., almost real work :-)
> >> 
> >>> I used to tweak the card settings, but now it's just stock. You may want
> >>> to
> >>> check your settings, the Mellanox may just have better defaults for your
> >>> switch.
> >>> 
> > Have you tried disabling TSO for the Intel? With TSO enabled, it will be
> > copying
> > every transmitted mbuf chain to a new chain of mbuf clusters via.
> > m_defrag() when
> > TSO is enabled. (Assuming you aren't an 82598 chip. Most seem to be the
> > 82599 chip
> > these days?)
> > 
> 
> hi Rick
> 
> how can i check the chip?
> 
Haven't a clue. Does "dmesg" tell you? (To be honest, since disabling TSO 
helped,
I'll bet you don't have a 82598.)

> > This has been fixed in the driver very recently, but those fixes won't be
> > in 10.1.
> > 
> > rick
> > ps: If you could test with 10.2, it would be interesting to see how the ix
> > does with
> >the current driver fixes in it?
> 
> I new TSO was involved!
> ok, firstly, it’s 10.2 stable.
> with TSO enabled, ix is bad, around 64MGB/s.
> disabling TSO it’s better, around 130
> 
Hmm, could you check to see of these lines are in sys/dev/ixgbe/if_ix.c at 
around
line#2500?
  /* TSO parameters */
2572 ifp->if_hw_tsomax = 65518;
2573 ifp->if_hw_tsomaxsegcount = IXGBE_82599_SCATTER;
2574 ifp->if_hw_tsomaxsegsize = 2048;

They are in stable/10. I didn't look at releng/10.2. (And if they're in a #ifdef
for FreeBSD11, take the #ifdef away.)
If they are there and not ifdef'd, I can't explain why disabling TSO would help.
Once TSO is fixed so that it handles the 64K transmit segments without copying 
all
the mbufs, I suspect you might get better perf. with it enabled?

Good luck with it, rick

> still, mlxen0 is about 250! with and without TSO
> 
> 
> > 
> >>> On Mon, Aug 17, 2015 at 6:41 AM, Slawa Olhovchenkov  >>> > wrote:
> >>> On Mon, Aug 17, 2015 at 10:27:41AM +0300, Daniel Braniss wrote:
> >>> 
>  hi,
>   I have a host (Dell R730) with both cards, connected to an HP8200
>   switch at 10Gb.
>   when writing to the same storage (netapp) this is what I get:
>   ix0:~130MGB/s
>   mlxen0  ~330MGB/s
>   this is via nfs/tcpv3
>  
>   I can get similar (bad) performance with the mellanox if I increase
>   the file size
>   to 512MGB.
> >>> 
> >>> Look like mellanox have internal beffer for caching and do ACK
> >>> acclerating.
> >>> 
>   so at face value, it seems the mlxen does a better use of resources
>   than the intel.
>   Any ideas how to improve ix/intel's performance?
> >>> 
> >>> Are you sure about netapp performance?
> >>> ___
> >>> freebsd-...@freebsd.org  mailing list
> >>> https://lists.freebsd.org/mailman/listinfo/freebsd-net
> >>> 
> >>> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org
> >>> "
> >>> 
> >> 
> >> ___
> >> freebsd-stable@freebsd.org mailing list
> >> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> >> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
> 
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"