Hello all,
While we were implementing new AQMs (Codel, PIE, FQ-Codel and FQ-PIE) in
FreeBSD targeting ipfw+Dummynet, we noticed timing issue with Dummynet when we
increase kernel timer interrupt Hz (changing the value of kern.hz in
/boot/loader.conf) to 3000 or higher. The issue is that spikes (upward and
downward) appear in RTT when we use FreeBSD+ipfw+Dummynet as a router with high
Hz value.
We did simple experiment (configuration listed below) with six runs and
Hz={1000, 2000, 3000, 3250, 3500, 4000} to demonstrate the issue. I have
attached two RTT vs Time graphs for our experiments. You can see in these
graphs everything is smooth when Hz ~ <=3000, and the issue starts to appear
when Hz ~>=3250.
As part of figuring out why this happen, I found that removing C_DIRECT_EXEC
and C_HARDCLOCK flags from callout_reset_sbt() call(in dn_reschedule()
function) causes the issue with high kernel interrupt hz (>3000 in my case) to
disappear even when kern.hz=10000.
I did another two experiment (kern.hz=10000, with and without the flags
mentioned above) and I logged dn_cfg.curr_time from dummynet_task() (after the
adjustments code). Then I calculated the difference (delta) between
curr_time[i] and curr_time[i-1] of the log file (the logged dn_cfg.curr_time).
I found that when kern.hz=10000 and C_DIRECT_EXEC and C_HARDCLOCK flags are
set, the delta values vary from 0 to ~600, while when C_DIRECT_EXEC and
C_HARDCLOCK flags are unset, the delta values vary from 1 to ~80. It seems like
the high jumps of delta values when the flags are set causes the issue. I have
attached a graph that compares delta values for the two experiments.
Does anyone have thoughts on what we can test next to narrow down the
root-cause of these unusual timing jumps?
Our configurations:
Router PC: Intel(R) Core(TM)2 Duo @ 2.33GHz processer, 4GiB RAM.
FreeBSD 10.1 (also I tried FreeBSD11-CURRENT-r289420).
ipfw+Dummynet: 10mbit traffic shaping , emulated delay 30ms (RTT=60ms), queue
size is 100 slices, droptail
Hosts: Linux 3.17
Traffic: One TCP flow, iperf, TCP NewReno
NICs: Intel PRO/1000 GT
Testbed experiments under the control of TEACUP
(http://caia.swin.edu.au/tools/teacup/)
Regards,
Rasool Al-Saadi
_______________________________________________
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
