Greetings,
In a private e-mail Martin Pelikan told me how he had solved this on
their network with many OpenVPN users. With his permission, I offer
this summary and solution to the list for the archives.
Martin wrote:
"I don't personally favor OpenVPN, but this ENOBUFS problem we solved on
our 1000+ users shaping DNS server only by brute force:
queue lowdelay on em2 bandwidth 4Mb priority 5 qlimit 600 hfsc( red ecn
realtime 256Kb upperlimit 10Mb )
The qlimit of 100 was way too small and it doesn't need more bandwidth.
I haven't seen the error in log for quite a long time. And in our setup
replacing hfsc is not an option - I even run a customized kernel with
1024 queues per one level instead of the default 64."
Based on that I reverted back to the hfsc scheduler and bumped the
qlimit levels to the following. Since I've done this, I'm no longer
seeing the "No buffer space available" error from OpenVPN, nor am I
getting any dropped packets or dropped bytes from any of the queues.
match in all scrub (no-df max-mss 1440)
altq on $ext_if bandwidth $ext_bw hfsc queue { main }
queue main bandwidth 99% priority 7 qlimit 6000 hfsc \
(realtime 20%, linkshare 99%) \
{ q_pri, q_web, q_mail, q_def, q_dns }
queue q_pri bandwidth 4% priority 7 qlimit 300 hfsc \
(red ecn realtime 0, linkshare 4%)
queue q_web bandwidth 50% priority 6 qlimit 1000 hfsc \
(red ecn realtime 30% linkshare 50%)
queue q_def bandwidth 30% priority 5 qlimit 1000 hfsc \
(default red ecn realtime (100Kb 3000 50Kb) linkshare 30%)
queue q_mail bandwidth 10% priority 1 qlimit 2000 hfsc \
(red ecn realtime (30Kb 3000 15Kb) linkshare 10%)
queue q_dns bandwidth 3% priority 4 qlimit 1000 hfsc \
(red ecn realtime (80Kb 5000 40Kb) linkshare 3%)
match out on $ext_if from $localnet nat-to $carp_if \
queue (q_def, q_pri) tag DEFAULT
match out on $ext_if proto tcp to port { www https } \
queue (q_web, q_pri) tag WEB
match out on $ext_if proto tcp to port smtp \
queue (q_mail, q_pri) tag MAIL
match out on $ext_if proto { tcp udp } to port domain \
queue (q_dns, q_pri) tag DNS
match out on $ext_if proto icmp \
queue (q_dns, q_pri) tag ICMP
Additionally, these rules put the OpenVPN traffic on the default queue
instead of the dns queue:
pass quick on tun0 from any to any queue q_def
pass quick on tun1 from any to any queue q_def
pass quick on tun2 from any to any queue q_def
Thanks again to all who replied!
Jeff Ross
