Re: High priority for the tx queue in netmap

[Vincenzo Maffione]

Hi,
  I think with ethtool on Linux you can play at least with the weight to
give to each TX queue for the round-robin algorithm. Of course not all the
cards implement this. I think Intel 10Gbit cards support it.

Cheers,
  VIncenzo

2017-01-24 8:17 GMT+01:00 Sepherosa Ziehau :

> On Tue, Jan 24, 2017 at 3:00 PM, Xiaoye Sun  wrote:
> > I'm using the typical intel 10 Gbps nic.
> > Does ethtool have related configuration command?
>
> You can check Intel's spec on their website for the chip you use.
> They have very good documentation.  IIRC, the default behavior is to
> round-robin TX queue on packet boundary (after TSO segementation).
>
> Don't know about Linux driver's state, but you have the detailed
> document, you can always change the code as you like for your own
> stuffs.
>
> Thanks,
> sephe
>
> --
> Tomorrow Will Never Die
> ___
> freebsd-net@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
>



-- 
Vincenzo Maffione
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] D8963: ifnet: introduce event handlers for ifup/ifdown events

[Phabricator]

This revision was automatically updated to reflect the committed changes.
Closed by commit rS312687: ifnet: introduce event handlers for ifup/ifdown 
events (authored by dexuan).

CHANGED PRIOR TO COMMIT
  https://reviews.freebsd.org/D8963?vs=23652&id=24374#toc

REPOSITORY
  rS FreeBSD src repository

CHANGES SINCE LAST UPDATE
  https://reviews.freebsd.org/D8963?vs=23652&id=24374

REVISION DETAIL
  https://reviews.freebsd.org/D8963

AFFECTED FILES
  head/sys/net/if.c
  head/sys/sys/eventhandler.h

EMAIL PREFERENCES
  https://reviews.freebsd.org/settings/panel/emailpreferences/

To: decui_microsoft.com, hselasky, cem, np, kmacy, kib, honzhan_microsoft.com, 
howard0su_gmail.com, jhb, ae, delphij, royger, glebius, gnn, rwatson, 
sepherosa_gmail.com
Cc: garga, freebsd-net-list
diff --git a/head/sys/sys/eventhandler.h b/head/sys/sys/eventhandler.h
--- a/head/sys/sys/eventhandler.h
+++ b/head/sys/sys/eventhandler.h
@@ -284,4 +284,11 @@
 EVENTHANDLER_DECLARE(swapon, swapon_fn);
 EVENTHANDLER_DECLARE(swapoff, swapoff_fn);
 
+/* ifup/ifdown events */
+#define IFNET_EVENT_UP		0
+#define IFNET_EVENT_DOWN	1
+struct ifnet;
+typedef void (*ifnet_event_fn)(void *, struct ifnet *ifp, int event);
+EVENTHANDLER_DECLARE(ifnet_event, ifnet_event_fn);
+
 #endif /* _SYS_EVENTHANDLER_H_ */
diff --git a/head/sys/net/if.c b/head/sys/net/if.c
--- a/head/sys/net/if.c
+++ b/head/sys/net/if.c
@@ -59,6 +59,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include 
 #include 
@@ -2218,6 +2219,7 @@
 if_down(struct ifnet *ifp)
 {
 
+	EVENTHANDLER_INVOKE(ifnet_event, ifp, IFNET_EVENT_DOWN);
 	if_unroute(ifp, IFF_UP, AF_UNSPEC);
 }
 
@@ -2230,6 +2232,7 @@
 {
 
 	if_route(ifp, IFF_UP, AF_UNSPEC);
+	EVENTHANDLER_INVOKE(ifnet_event, ifp, IFNET_EVENT_UP);
 }
 
 /*

___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: inheriting fib from an interface

[Eugene Grosbein]

On Mon, Jan 23, 2017 at 05:31:22PM -0800, Don Lewis wrote:

> Let's say that I have an application running on a server that is
> connected to the Internet via two different ISPs and is using IP
> addresses (ISP A:10.0.0.10 and ISP B:192.168.1.10) delegated by those
> two ISPs on it's two interfaces.  Responses to requests sent to
> 10.0.0.10 should be sent via ISP A, and responses to requests sent to
> 192.168.1.10 should be ISB B.
> 
> There are a couple of different ways that I can think of to do this:
> 
> 1) Put the server behind another FreeBSD box that uses policy-based
>  routing to forward the outbound packets to the desired ISP.  My
>  understanding is that this only works for packet forwarding and not
>  for locally generated packets.

Single command "ipfw add 2000 fwd $ispgw2 ip from $ip2 to any out xmit 
$isp1_iface"
works for locally generated packets too.

It "fixes" outgoing routing path for packets
from IP belonging to "non-default" ISP2 when default route points to ISP1.
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: RFC: ethctl

[Borja Marcos]

> On 20 Jan 2017, at 19:36, Garrett Wollman  
> wrote:
> 
> In article  you write:
>> Eg, I don't see why we need another tool for some of this missing
>> "ethtool" functionality; it seems like most of it would naturally fit
>> into ifconfig.
> 
> From the end-user perspective, I agree with Drew.  Most of this stuff
> should just be part of ifconfig.

Actually I have a concern regarding the ethtool inspiration. I am sure it’s not 
the only factor at play,
but I think that the decision to include options such as autonegotiation or 
forced speed/duplex modes in
ifconfig helped FreeBSD drivers to be more consistent. In the old days when 
Fast Ethernet ruled and so many
devices got negotiation wrong my few interactions with Linux systems were 
really frustrating. Either the ethtool
package wasn’t installed, or it was the wrong version, or the driver just 
didn’t support ethtool.

More recently I faced a bug in an Intel driver: media detection didn’t work 
properly for 10 GbE DA cables. 
The interface worked, but the it didn’t get marked as full duplex, which meant 
that lagg rightfully refused to use it in
LACP mode (LACP mandates full duplex interfaces).

Debugging it on FreeBSD was a somewhat straightforward task. It was easy to 
determine why lagg refused to use it,
ifconfig indeed reflected the “unknown duplexness” and it gave me a good 
pointer to the issue (driver problem).

My coworkers told me that they were facing a similar problem on some Linux 
systems and I tried to have a look at it.
But looking at the whole ifconfig/ethtool stuff I believe (note, _I believe_, I 
didn’t have time to pay more attention to it and
it wasn’t critical for us anyway) that ethtool didn’t report the actual 
interface status from the operating system point of view,
but it rather polled the network interface itself and represented its own 
picture.

So, while the same issue was probably causing LACP not to work, which would 
mean that the interface wasn’t considered
full-duplex, the system’s ifconfig did not show anything about it, and ethtool 
probably deducted by itself that the interface
was actually full duplex, after all half-duplex 10 GbE doesn’t exist.

The moral of the story is: ifconfig being tightly coupled to the OS is more 
likely to be made to present the proper information.
What about an ethtool-class command? I guess there is a risk that, being more 
close to hardware, in the future it might be subject
to a similar problem to that one I believe ethtool to have. 

I understand that it wouldn’t be a Linux-style optional package, but part of 
the system, which should remove the concerns
about driver support. But, anyway, given that ifconfig is already supporting 
plenty of media/driver options, wouldn’t it be better
to refine it a bit and leave the handful of esoteric operations to a driver 
specific tool?

At least it’s cleaner I think. Just my opinion, based in part on beliefs and, I 
admit it, on having been burned by linuxisms.







Borja.




___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: High priority for the tx queue in netmap

[Xiaoye Sun]

I'm using the typical intel 10 Gbps nic.
Does ethtool have related configuration command?


On Tue, Jan 24, 2017 at 12:57 AM Sepherosa Ziehau 
wrote:

> On Tue, Jan 24, 2017 at 4:09 AM, Xiaoye Sun  wrote:
>
> > Hi,
>
> >
>
> > In my application, there are multiple tx queues in the OS, and the netmap
>
> > program uses one tx queue. I wondering if I can give the netmap tx queue
>
> > higher priority so that the NIC always sends the packets from this queue
> if
>
> > netmap put any packet to it.
>
>
>
> You need to check your NIC chip vendor for this.
>
>
>
> Thanks,
>
> sephe
>
>
>
>
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: HEADS-UP: IFLIB implementations of sys/dev/e1000 em, lem, igb pending

[Sean Bruno]

On 01/23/17 23:31, Olivier Cochard-Labbé wrote:
> On Tue, Jan 24, 2017 at 2:40 AM, Sean Bruno  > wrote:
> 
> 
> 
> Which set of configs from your test suite are you using for this?
> Specifically, what packet size are you slamming across?
> 
> https://github.com/ocochard/netbenches/tree/master/pktgen.configs
> 
> 
> 
> ​Because I'm in the point of view of a Telco, I'm measuring the «worst»
> case, this mean with the smallest frame size.
> Here is the exact pkt-gen command line I'm using:
> - 60 byte Ethernet frame size (excluding the 4 CRC bytes)
> - 2000 UDP flows (20 IP sources * 100 IP destinations)
> 
> pkt-gen -U -i igb2 -f tx -n 8000 -l 60 -d 198.19.10.1:2000-198.19.10.20 
> -D 00:0d:b9:41:ca:3d -s 198.18.10.1:2000-198.18.10.100 -w 4
> 
> ​Option -U is available on a patched netmap version [1]: It fix the
> checksum calculation when using source/destination IP range on NIC that
> didn't enable HW CHKSUM in netmap mode and IPv6 support.
> 
> [1]
> https://github.com/ocochard/BSDRP/blob/master/BSDRPcur/patches/freebsd.pkt-gen.ae-ipv6.patch
> 


Did you increase the number of rx/tx rings to 8 and the number of
descriptors to 4k in your tests or just the defaults?

sean



signature.asc
Description: OpenPGP digital signature

Re: RTL8153 Gigabit Ethernet USB Adapter

[diffusae]

Hi!

Thanks for your fast reply.

On 23.01.2017 21:24, Hans Petter Selasky wrote:
> On 01/23/17 21:06, diffusae wrote:
>> Hi!
>>
>> Maybe a noobs question but I am mostly familiar with Linux.
>>
>> Currently there is no driver for the RTL8153 Gigabit Ethernet Adapter.
>>
>> Bus 001 Device 004: ID 0bda:8153 Realtek Semiconductor Corp.
>>
>> https://www.freebsd.org/relnotes/CURRENT/hardware/support.html
>>
>> RealTek has a Unix (Linux) driver here:
>>
>> http://www.realtek.com/downloads/downloadsView.aspx?Langid=1&PNid=56&PFid=56&Level=5&Conn=4&DownTypeID=3&GetDown=false#RTL8153
>>
>>
>> How can I compile a custome kernel with this driver?
>>
>> I am using FreeBSD 11.0-STABLE (RPI-B) #0 r308738
>>
> 
> Hi,
> 
> Have a look in sys/dev/usb/net and see if you find any similar devices.
> I think your driver is already supported. You need to:
> 
> 1) kldload usb_quirk
> 2) kldload if_ure
> 3) replug your device and it should attach (FreeBSD-12 at least)
> 
> grep -r 8153 /usr/src/sys/dev/usb
> 
> --HPS

That was the key. Yes, you're right. It's supported and uses the if_cdce
driver, but there is no mediaopt extention with ifconfig. Nevertheless
it is connected with one 1 Gbit/s. So, that's fine.

It was due to a "faulty" USB OTG cable of the RPI. If I plug the device
connector until the end, it isn't detected, because it has no power. If
I slighly move it back for some mm, that the power is connected and the
device will be detected:

usbconfig -d 0.2 dump_device_desc
ugen0.2:  at usbus0, cfg=1 md=HOST spd=HIGH
(480Mbps) pwr=ON (180mA)

Strange chinese adaptors ... :-)

So, thanks again

Best regards,
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: HEADS-UP: IFLIB implementations of sys/dev/e1000 em, lem, igb pending

[Olivier Cochard-Labbé]

On Tue, Jan 24, 2017 at 3:17 PM, Sean Bruno  wrote:

>
>
> Did you increase the number of rx/tx rings to 8 and the number of
> descriptors to 4k in your tests or just the defaults?
>

Tuning are same as described in my previous email (rxd|txd=2048, rx|tx
process_limit=-1, max_interrupt_rate=16000).
[root@apu2]~# sysctl hw.igb.
hw.igb.tx_process_limit: -1
hw.igb.rx_process_limit: -1
hw.igb.num_queues: 0
hw.igb.header_split: 0
hw.igb.max_interrupt_rate: 16000
hw.igb.enable_msix: 1
hw.igb.enable_aim: 1
hw.igb.txd: 2048
hw.igb.rxd: 2048


But I've did a new benchs with default setting, and the performance drop is
now about -25% :

x head r311848 packets-per-second (default settings)
+ head r311849 packets-per-second (default settings)
+--+
|+ |
|+   x |
|+   xx|
|++  xx|
|A||
|A||
+--+
N   Min   MaxMedian   AvgStddev
x   5618711621135  619930.5  619840.8 951.83787
+   5467389468740467778  467864.8 550.40322
Difference at 95.0% confidence
-151976 +/- 1133.9
-24.5186% +/- 0.150581%
(Student's t, pooled s = 777.476)
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: HEADS-UP: IFLIB implementations of sys/dev/e1000 em, lem, igb pending

[Sean Bruno]

On 01/24/17 08:27, Olivier Cochard-Labbé wrote:
> On Tue, Jan 24, 2017 at 3:17 PM, Sean Bruno  > wrote:
> 
> 
> 
> Did you increase the number of rx/tx rings to 8 and the number of
> descriptors to 4k in your tests or just the defaults?
> 
> 
> Tuning are same as described in my previous email (rxd|txd=2048, rx|tx
> process_limit=-1, max_interrupt_rate=16000).
> [root@apu2]~# sysctl hw.igb.
> hw.igb.tx_process_limit: -1
> hw.igb.rx_process_limit: -1
> hw.igb.num_queues: 0
> hw.igb.header_split: 0
> hw.igb.max_interrupt_rate: 16000
> hw.igb.enable_msix: 1
> hw.igb.enable_aim: 1
> hw.igb.txd: 2048
> hw.igb.rxd: 2048
> 
> 

Oh, I think you missed my note on these.  In order to adjust txd/rxd you
need to tweak the iflib version of these numbers.  nrxds/ntxds should be
adjust upwards to your value of 2048.  nrxqs/ntxqs should be adjust
upwards to 8, I think, so you can test equivalent settings to the legacy
driver.

Specifically, you may want to adjust these:

dev.em.0.iflib.override_nrxds: 0
dev.em.0.iflib.override_ntxds: 0

dev.em.0.iflib.override_nrxqs: 0
dev.em.0.iflib.override_ntxqs: 0

sean

> But I've did a new benchs with default setting, and the performance drop
> is now about -25% :
> 
> x head r311848 packets-per-second (default settings)
> + head r311849 packets-per-second (default settings)
> +--+
> |+ |
> |+   x |
> |+   xx|
> |++  xx|
> |A||
> |A||
> +--+
> N   Min   MaxMedian   AvgStddev
> x   5618711621135  619930.5  619840.8 951.83787
> +   5467389468740467778  467864.8 550.40322
> Difference at 95.0% confidence
> -151976 +/- 1133.9
> -24.5186% +/- 0.150581%
> (Student's t, pooled s = 777.476)
> 



signature.asc
Description: OpenPGP digital signature

Re: HEADS-UP: IFLIB implementations of sys/dev/e1000 em, lem, igb pending

[Bruce Evans]

On Tue, 24 Jan 2017, Sean Bruno wrote:


On 01/24/17 08:27, Olivier Cochard-Labb?? wrote:

On Tue, Jan 24, 2017 at 3:17 PM, Sean Bruno mailto:sbr...@freebsd.org>> wrote:

Did you increase the number of rx/tx rings to 8 and the number of
descriptors to 4k in your tests or just the defaults?

Tuning are same as described in my previous email (rxd|txd=2048, rx|tx
process_limit=-1, max_interrupt_rate=16000).
[root@apu2]~# sysctl hw.igb.
hw.igb.tx_process_limit: -1
hw.igb.rx_process_limit: -1
hw.igb.num_queues: 0
hw.igb.header_split: 0
hw.igb.max_interrupt_rate: 16000
hw.igb.enable_msix: 1
hw.igb.enable_aim: 1
hw.igb.txd: 2048
hw.igb.rxd: 2048


Oh, I think you missed my note on these.  In order to adjust txd/rxd you
need to tweak the iflib version of these numbers.  nrxds/ntxds should be
adjust upwards to your value of 2048.  nrxqs/ntxqs should be adjust
upwards to 8, I think, so you can test equivalent settings to the legacy
driver.

Specifically, you may want to adjust these:

dev.em.0.iflib.override_nrxds: 0
dev.em.0.iflib.override_ntxds: 0

dev.em.0.iflib.override_nrxqs: 0
dev.em.0.iflib.override_ntxqs: 0


That is painful.

My hack to increase the ifq length also no longer works:

X Index: if_em.c
X ===
X --- if_em.c   (revision 312696)
X +++ if_em.c   (working copy)
X @@ -1,3 +1,5 @@
X +int em_qlenadj = -1;
X +

-1 gives a null adjustment; 0 gives a default (very large ifq), and other
values give a non-null adustment.

X  /*-
X   * Copyright (c) 2016 Matt Macy 
X   * All rights reserved.
X @@ -2488,7 +2490,10 @@
X 
X  	/* Single Queue */

X  if (adapter->tx_num_queues == 1) {
X -   if_setsendqlen(ifp, scctx->isc_ntxd[0] - 1);
X +   if (em_qlenadj == 0)
X + em_qlenadj = imax(2 * tick, 0) * 15 / 10;
X + // lem_qlenadj = imax(2 * tick, 0) * 42 / 100;
X +   if_setsendqlen(ifp, scctx->isc_ntxd[0] + em_qlenadj);
X if_setsendqready(ifp);
X   }
X

I don't want larger hardware queues, but sometimes want larger software
queues.  ifq's used to give them.  The if_setsenqlen() call is still there.
but no longer gives them.

The large queues are needed for backet blasting benchmarks since select()
doesn't work for udp sockets, so if the queues fill up then the benchmarks
must busy-wait or sleep waiting for them to drain, and timeout granularity
tends to prevent short sleeps from working so the queues run dry while
sleeping unless the queues are very large.

Bruce___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: RFC: ethctl

[Adrian Chadd]

On 19 January 2017 at 19:58, Kevin Bowling  wrote:
> Greetings,
>
> I'm casting a wide net in cc, try to keep the noise minimal but we need
> some input from a variety of HW vendors.
>
> I have heard from several vendors the need for a NIC configuration tool.
>  Chelsio ships a cxgb/cxgbetool in FreeBSD as one example.  There is
> precedence for some nod toward a basic unified tool in Linux ethtool.
>
> From your perspective,
> 1) What are the common requirements?
> 2) What are specialized requirements? For instance as a full TCP offload
> card Chelsio needs things others wont
> 3) What should it _not_ do?  Several of you have experience doing Ethernet
> driver dev on many platforms so we should attempt to avoid repeating past
> design mistakes.
>
> I expect we can achieve some level of inversion so the device specific code
> can live close to the driver and plug into the ethctl framework.  It should
> be general enough to add completely new top level commands, so vendors can
> implement HW specific features.  On the other hand, we should attempt to
> hook into common core for features every NIC provides, with a focus on
> iflib.
>
> I will fund Matt Macy to do the overall design and implementation.
>
> Regards,
> Kevin Bowling, on behalf of Limelight Networks for this effort

Hi,

ethtool isn't exactly complicated. It's just vaguely standardized.

When I was hoping (hah!) to do it, partly for wifi but partly for
ethernet, my goals were:

* generic interface for counter statistics, versus sysctl;
* "default" mib space for known counter statistics, versus the small
set we have now and then the very large vendor space that ethtool
(linux) and sysctl (freebsd) exposes;
* generic interface for configuring things like RSS mapping, L2/L3
rules for a NIC based on the NIC capability and what queue(s) they map
to, versus vendor tools;
* vendor specific extensions, which hopefully (!) are implemented as
generic-ish plugins where required, or just strings that can be passed
through to the driver and registered via a command hook;
* and importantly - some kind of optional debug control, because every
driver does something different for debugging / tracing, and boy it'd
be nice if it all was like wifi (wlandebug -i 
+/-(feature))

A lot of what's in linux ethtool is in freebsd's ifconfig, albeit not
in a reusable/runtime-extensible fashion. It'd be nice to include say,
many more vendor counters in a somewhat generic fashion, versus how we
currently do things.

I think it'd be a good starting point to have an ethtool control iflib
drivers, so any driver using iflib can benefit from what's
implemented.

2c,

-adrian
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: inheriting fib from an interface

[Don Lewis]

On 24 Jan, Eugene Grosbein wrote:
> On Mon, Jan 23, 2017 at 05:31:22PM -0800, Don Lewis wrote:
> 
>> Let's say that I have an application running on a server that is
>> connected to the Internet via two different ISPs and is using IP
>> addresses (ISP A:10.0.0.10 and ISP B:192.168.1.10) delegated by those
>> two ISPs on it's two interfaces.  Responses to requests sent to
>> 10.0.0.10 should be sent via ISP A, and responses to requests sent to
>> 192.168.1.10 should be ISB B.
>> 
>> There are a couple of different ways that I can think of to do this:
>> 
>> 1) Put the server behind another FreeBSD box that uses policy-based
>>  routing to forward the outbound packets to the desired ISP.  My
>>  understanding is that this only works for packet forwarding and not
>>  for locally generated packets.
> 
> Single command "ipfw add 2000 fwd $ispgw2 ip from $ip2 to any out xmit 
> $isp1_iface"
> works for locally generated packets too.
> 
> It "fixes" outgoing routing path for packets
> from IP belonging to "non-default" ISP2 when default route points to ISP1.

Thanks, that looks very promising!

___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"