Re: making SCTP loadable and removing it from GENERIC
> On 10. Jul 2020, at 02:06, Eugene Grosbein wrote:
>
> 10.07.2020 2:44, Doug Hardie wrote:
>
>>> On 9 July 2020, at 08:13, Mark Johnston wrote:
>>>
>>> Hi,
>>>
>>> I spent some time working on making it possible to load the SCTP stack
>>> as a kernel module, the same as we do today with IPSec. There is one
>>> patch remaining to be committed before that can be done in head. One
>>> caveat is that the module can't be unloaded, as some work is needed to
>>> make this safe. However, this obviously isn't a regression.
>>>
>>> The work is based on the observations that:
>>> 1) the in-kernel SCTP stack is not widely used (I know that the same
>>> code is used in some userland applications), and
>>> 2) the SCTP stack is quite large, most FreeBSD kernel developers are
>>> unfamiliar with it, and bugs in it can easily lead to security holes.
>>>
>>> Michael has done a lot of work to fix issues in the SCTP code,
>>> particularly those found by syzkaller, but given that in-kernel SCTP has
>>> few users (almost certainly fewer than IPSec), it seems reasonable to
>>> require users to opt in to having an SCTP stack with a simple "kldload
>>> sctp". Thus, once the last patch is committed I would like to propose
>>> removing "options SCTP" from GENERIC kernel configs in head, replacing
>>> it with "options SCTP_SUPPORT" to enable sctp.ko to be loaded.
>>>
>>> I am wondering if anyone has any objections to or concerns about this
>>> proposal. Any feedback is appreciated.
>>
>> I have a number of systems using SCTP. It is a key part of a distributed
>> application. As a user of SCTP, I have a slight objection to removing it
>> from the kernel. It would require me to remember when setting up a new
>> system to enable that. I am not likely to remember. What is going to
>> happen if you run an application that uses SCTP and the module is not
>> loaded? What will remind me how to fix the issue? I am not likely to
>> remember about this 6 months from now.
>
> If an application starts with superuser privileges (as root), it is allowed
> to perform the check
> and load the module if needed:
>
> int
> modload(const char *name)
> {
>if (modfind(name) < 0)
>if (kldload(name) < 0 || modfind(name) < 0) {
>warn("%s: module not found", name);
>return 0;
>}
>return 1;
> }
> ...
> modload("sctp");
>
> This works for both cases of sctp built into the kernel and already loaded as
> module.
Hi Eugene,
you are completely right. However, it requires that the program needs to run
with root privileges just to be able to communicate.
In the context of userland stack, this is one of the most important issues.
In case of SCTP, this is needed to open a raw socket to send/recv SCTP packets.
This is one of the reasons why you use UDP encapsulation...
Best regards
Michael
>
> Alternatively, if an application already has rc.d startup script, you don't
> even need to change
> application source code but add required_modules="sctp" to the script, see
> rc.subr(8),
> then sctp.ko would be loaded automagically if it was not loaded yet and not
> present in the kernel.
Interesting, I did not know that. Thanks for sharing.
Best regards
Michael
>
>
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: making SCTP loadable and removing it from GENERIC
> On 10 July 2020, at 02:39, Michael Tuexen wrote: > > Hi Eugene, > > you are completely right. However, it requires that the program needs to run > with root privileges just to be able to communicate. > In the context of userland stack, this is one of the most important issues. > In case of SCTP, this is needed to open a raw socket to send/recv SCTP > packets. > This is one of the reasons why you use UDP encapsulation... I see RFC 6951 on UDP encapsulation and understand there are situations where that would be needed. However, my replication processes do run as root. Just for fun, I started them as non-root and SCTP worked just fine. I didn't see any raw sockets in a ktrace of the processes. 76330 replicate CALL socket(PF_INET6,0x5,IPPROTO_SCTP) 76330 replicate RET socket 5 -- Doug ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: making SCTP loadable and removing it from GENERIC
> On 10. Jul 2020, at 12:29, Doug Hardie wrote: > >> On 10 July 2020, at 02:39, Michael Tuexen wrote: >> >> Hi Eugene, >> >> you are completely right. However, it requires that the program needs to run >> with root privileges just to be able to communicate. >> In the context of userland stack, this is one of the most important issues. >> In case of SCTP, this is needed to open a raw socket to send/recv SCTP >> packets. >> This is one of the reasons why you use UDP encapsulation... > > I see RFC 6951 on UDP encapsulation and understand there are situations where > that would be needed. However, my replication processes do run as root. > Just for fun, I started them as non-root and SCTP worked just fine. I didn't > see any raw sockets in a ktrace of the processes. > > 76330 replicate CALL socket(PF_INET6,0x5,IPPROTO_SCTP) > 76330 replicate RET socket 5 Hi Doug, sorry, I wasn't clear. If you use an SCTP kernel stack, you don't need root privileges, except for binding to a port smaller than 1024. This is the same for TCP and UDP. If you want to use a userland SCTP stack, the program must be able to send and receive SCTP packets. That is done via a raw socket, which need root privileges to be opened. Since you are using a kernel stack, you don't see any raw socket interactions, but the opening of an SCTP socket. The FreeBSD SCTP sources are also the basis of the SCTP userland stack usrsctp. That is why I have some experience with people running userland SCTP stacks. One of the issues they report, is that they don't want to run an application with root privileges just to do communication. I agree with that. To avoid using root privileges just for network communication, the solution is to use a kernel stack or a userland stack with raw sockets, which can be realised by using UDP encapsulation. If you run you application as root due to other constraints, there is no problem for you to also load a kernel module. So you do not have the problem I was referring to. Best regards Michael > > -- Doug > ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: poor performance with Intel X520 card
Hello This interface is 14.8 Mpps, but such capacity is only possible without a firewall performing filtering. The more firewall rules on your router, the less forwarding capacity the card will have, due to having to process the packet in CPU to match the rules and then forward the packet. In the link that follows, a little is talked about: https://wiki.freebsd.org/NetworkPerformanceTuning Here are some more useful links on FreeBSD and network tuning https://calomel.org/freebsd_network_tuning.html https://people.freebsd.org/~olivier/talks/2017_EuroBSDCon-Tuning_FreeBSD_for_routing_and_firewalling.pdf Em sex., 10 de jul. de 2020 às 03:45, Patrick Lamaiziere escreveu: > > Hello, > > That is mostly for the record but it looks like the intel X520 is not > very good and generates a high level of interrupts. > > On a router / firewall with 500 Kpps in input (dropped by pf) is enough to > put the CPUs at > 100% busy. > > We use FreeBSD 11.3 on a machine with 12 CPU Intel(R) Xeon(R) CPU E5-2643 v3 > @ 3.40GHz (3400.07-MHz K8-class CPU) > FreeBSD/SMP: Multiprocessor System Detected: 12 CPUs > FreeBSD/SMP: 2 package(s) x 6 core(s) > Multi threading is disabled. > > ix0: port > 0x3020-0x303f mem 0x9230-0x923f,0x92404000-0x92407fff irq 34 at > device 0.0 numa-domain 0 on pci5 > ix0: Using MSI-X interrupts with 9 vectors > ix0: Ethernet address: a0:36:9f:93:84:10 > ix0: PCI Express Bus: Speed 5.0GT/s Width x8 > ix0: netmap queues/slots: TX 8/2048, RX 8/2048 > > I've set fews tunable in loader.conf but I don't remember why > loader.conf > # cartes ix > #hw.ix.max_interrupt_rate=10 > #hw.ix.enable_aim=0 > hw.ix.tx_process_limit=-1 > hw.ix.rx_process_limit=-1 > #hw.ix.num_queues=6 > hw.intr_storm_threshold=9000 > > Well, do you think another NIC cards can help to reach a better pps rate ? I > think 500 Kpps is quite low for such a machine. > > Thanks, best regards. > > ___ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org" ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: poor performance with Intel X520 card
On Fri, Jul 10, 2020 at 8:45 AM Patrick Lamaiziere wrote: > Hello, > > That is mostly for the record but it looks like the intel X520 is not > very good and generates a high level of interrupts. > > On a router / firewall with 500 Kpps in input (dropped by pf) is enough to > put the CPUs at > 100% busy. > > Hi Patrick, yes 500 Kpps is quite low: Do you have a very complex long pf rule set? A 8 core Atom C2758 with an old Intel 10G 82599 is able to reach about 1.6Mpps (with one pf rule), so I would expect more on your setup. https://github.com/ocochard/netbenches/blob/master/Atom_C2758_8Cores-Intel_82599/forwarding-pf-ipfw/results/fbsd12-stable.r354440.BSDRP.1.96/README.md So, try this: - Identify the bottleneck: pmcstat and flamegraph are the tools for that; - Use FreeBSD -head or a 12-stable minimum but not less; - You should follow instruction here: https://wiki.freebsd.org/10gFreeBSD/Router Regards, Olivier ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
[Bug 238520] [sctp] Fatal trap 9: general protection fault while in kernel mode
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=238520 Mark Johnston changed: What|Removed |Added CC||ma...@freebsd.org Resolution|--- |FIXED Status|In Progress |Closed -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
[Bug 224218] Kernel panic in SCTP/IpV6 server mode
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=224218 Mark Johnston changed: What|Removed |Added CC||ma...@freebsd.org Resolution|--- |Overcome By Events Status|In Progress |Closed --- Comment #25 from Mark Johnston --- Closing since the default stack size was increased on i386. The two major offenders in SCTP, sctp_auth_get_cookie_params() and sctp_load_addresses_from_init() are still there. They both allocate 3 512-byte buffers on the stack. I can't see an easy way to fix that; all three buffers are used to temporarily store data until we know the combined size of the data, at which point a buffer to store all of it is allocated. It might be possible to avoid the temporary buffers by using m_pulldown() to ensure that the parameter headers are contiguous, and then use m_copydata() to copy data into the key buffer once we know the combined length. This is a bit tricky to get right and I have no setup to test such a change. However, it would shave 1536 bytes off the stack frame and avoid some extra copying. -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: making SCTP loadable and removing it from GENERIC
On Thu, Jul 09, 2020 at 11:13:00AM -0400, Mark Johnston wrote: > Hi, > > I spent some time working on making it possible to load the SCTP stack > as a kernel module, the same as we do today with IPSec. There is one > patch remaining to be committed before that can be done in head. One > caveat is that the module can't be unloaded, as some work is needed to > make this safe. However, this obviously isn't a regression. > > The work is based on the observations that: > 1) the in-kernel SCTP stack is not widely used (I know that the same >code is used in some userland applications), and > 2) the SCTP stack is quite large, most FreeBSD kernel developers are >unfamiliar with it, and bugs in it can easily lead to security holes. > > Michael has done a lot of work to fix issues in the SCTP code, > particularly those found by syzkaller, but given that in-kernel SCTP has > few users (almost certainly fewer than IPSec), it seems reasonable to > require users to opt in to having an SCTP stack with a simple "kldload > sctp". Thus, once the last patch is committed I would like to propose > removing "options SCTP" from GENERIC kernel configs in head, replacing > it with "options SCTP_SUPPORT" to enable sctp.ko to be loaded. > > I am wondering if anyone has any objections to or concerns about this > proposal. Any feedback is appreciated. As a follow-up, here is the proposed change now that requisite code has been committed to head: https://reviews.freebsd.org/D25611 I will wait for a week or so for feedback. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
[Bug 224218] Kernel panic in SCTP/IpV6 server mode
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=224218 --- Comment #26 from Michael Tuexen --- (In reply to Mark Johnston from comment #25) Increasing the stack size is a workaround. The plan was to rewrite the handling such that only one buffer is needed. That is why I left the bug open. Since it is closed now, such an optimisation does not seem to be wanted anymore. -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
[Bug 224218] Kernel panic in SCTP/IpV6 server mode
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=224218 Mark Johnston changed: What|Removed |Added Resolution|Overcome By Events |--- Status|Closed |Open --- Comment #27 from Mark Johnston --- (In reply to Michael Tuexen from comment #26) Sorry. I closed the bug only because the submitter's original problem was resolved and I was going through some old bug reports. Since you plan to work on the bug I reopened the PR. -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
[Bug 224218] Kernel panic in SCTP/IpV6 server mode
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=224218 --- Comment #28 from Michael Tuexen --- (In reply to Mark Johnston from comment #27) OK, great. I think reducing the stack space worth the effort. It is not that hard, will improve also the handling of pathological parameter configurations. I haven't done this yet, because I also must extend packetdrill to have a way to test all the strange disallowed combinations. And packetdrill is still missing support for the Authentication extension... -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Deadlocks when using pf tags or socket owner matching
On 8 Jul 2020, at 12:52, Kajetan Staszkiewicz wrote: I have forgot to mention my system: it's FreeBSD 11.3-RELEASE-p9 I have also managed to replicate this (or a similar) issue on a test system built with lock debugging and I got this: Jul 8 10:32:07 hwlb-aw-01 kernel: lock order reversal: Jul 8 10:32:07 hwlb-aw-01 kernel: 1st 0x81850760 pf rulesets (pf rulesets) @ /usr/home/kajetan.staszkiewicz/freebsd.git/sys/netpfil/pf/pf.c:6006 Jul 8 10:32:07 hwlb-aw-01 kernel: 2nd 0xf8011f7028a8 tcpinp (tcpinp) @ /usr/home/kajetan.staszkiewicz/freebsd.git/sys/netinet/in_pcb.c:1994 This lock order reversal is almost certainly the reason for the deadlock you report in your previous e-mail. The problem relates to the UID filtering functionality, so not using that feature will avoid the deadlock. I believe I’ve seen previous reports about issues in this area as well. I’ll try to put this on my short-sh term todo list, but I can’t promise anything. Free time is exceptionally rare these days. Best regards, Kristof ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
[Bug 206932] Realtek 8111 card stops responding under high load in netmap mode
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206932 Mark Linimon changed: What|Removed |Added Resolution|--- |Overcome By Events Status|In Progress |Closed -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
[Bug 189088] Assigning the same IP to multiple interfaces in different FIBs creates a host route for only one.
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=189088 Mark Linimon changed: What|Removed |Added Status|In Progress |Open --- Comment #6 from Mark Linimon --- Not currently being worked on. -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
