Ruslan Ermilov wrote:
I'm sorry, I mis-wrote. My ng_tee is actually modified to only passes packets to the r2l/l2r hooks if they are connected, otherwise packets are passed directly to the left/right hooks (so it's an optional divert), so there is no m_dup anymore in my modified ng_tee.Hi Guy,
On Fri, Feb 04, 2005 at 11:03:31AM -0600, Guy Helmer wrote:
A while back, Maxim Konovalov made a commit to usr.sbin/ngctl/main.c to increase its socket receive buffer size to help 'ngctl list' deal with a big number of nodes, and Ruslan Ermilov responded that setting sysctls net.graph.recvspace=200000 and net.graph.maxdgram=200000 was a good idea on a system with a large number of nodes.The bottleneck must be in ng_tee(4) -- the latter uses m_dup(9) when
I'm getting what I consider to be sub-par performance under FreeBSD 5.3 from a userland program using ngsockets connected into ng_tee to play with packets that are traversing a ng_bridge, and I finally have an opportunity to look into this. I say "sub-par" because when we've tested this configuration using three 2.8GHz Xeon machines with Gigabit Ethernet interfaces at 1000Mbps full-duplex, we obtained peak performance of a single TCP stream of about 12MB/sec through the bridging machine as measured by NetPIPE and netperf.
a duplicate is needed, which is very expensive as it has to create a
writable copy of the entire mbuf chain (the original chain is DMA'ed
into the host memory by the network card).
Thanks, Ruslan. Yes, I do need to pass all the traffic down through my userland daemon. Since I'm just beginning to work with Netgraph, I was wondering if there was something simple or obvious that I was missing, or if there was a known performance issue with one of the nodes I'm using (as you pointed out with ng_tee).I'm wondering if bumping the recvspace should help, if changing the ngsocket hook to queue incoming data should help, if it would be best to replace ngsocket with a memory-mapped interface, or if anyone has any other ideas that would help performance.If you absolutely need to see *all* GigE traffic in userland, then
it's going to be troublesome. If not, filter it with ng_bpf(4).
I assumed that the bridging and trip through userland would only add latency to the connection, but the result of the performance test seemed to indicate that there is either a bottleneck I need to solve or my testing methodology was flawed.
Thanks again, Guy
_______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "[EMAIL PROTECTED]"
