On 05/20/2015 09:50 PM, Todd Bezenek wrote:
I'm pushing 10,000 frames per second of UDP traffic through a Linux
system with a bridge configured between two 1GbE ports.
Iptables is installed and running, but the default rule is ACCEPT with
no other rules.
When I make the packets 1179 bytes in size (total size includes
Ethernet header, etc.), I see the latency of most packets between 60
and 200 usec. When I change the size to 1180 bytes, I start seeing
about 1% of the packets with latencies larger than 300 usec, and some
as high as 500+ usec.
Any ideas what buffer size, cache line count, TLB reach, etc. might be
causing this dramatic change?
Cut-and-paste to my terminal is not working, so limited info here:
Kernel: 3.12.20 x86-64
Processor: Intel E3845 (Atom, 4 cores)
Network interface driver: Intel igb
Thank you for any helpful pointers.
-Todd
The igb driver defaults to an adaptive interrupt moderation scheme. It
is possible that what you are seeing could be due to that, or it could
possibly be that the time between packets is becoming large enough that
your CPU is starting to go into deeper sleep states.
My first suggestion would be to try disabling sleeps states beyond C1.
To do that you could boot your kernel with the parameter
"processor.max_cstate=1" or "idle=poll". Either one should prevent the
CPU from going into a sleep state but it will consume more power.
You also might try modifying the interrupt moderation to a fixed value
of 10 or more via "ethtool -C <dev> rx-usecs <x>" to see if the behavior
goes away after that. If the device is the cause it might point to a
possible glitch in the interrupt moderation scheme or possibly lost
interrupts for the device or driver.
- Alex
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
