Hello, All !

We tried to use netmap-ipfw in production (as filtering bridge) for traffic sanity and bandwidth limitation.
And meet a problem. Will be explaned below.

CPU: i5-4690 CPU @ 3.50GHz
RAM: 8GB x 1800Mhz
NET: Intel DA 520 (2 x 10Gbps)

kipfw starts as:
/usr/local/netmap-ipfw/kipfw netmap:ix0 netmap:ix1

ruleset:

00100 allow ip from 192.168.254.0/24 to 192.168.254.0/24
00200 allow ip from any to 192.168.0.0/16 - incoming (for customers) traffic goes without touching
00400 pipe 665 udp from 192.168.0.0/16 to any dst-port 6881
00500 pipe 666 tcp from 192.168.0.0/16 to any tcpflags syn
00600 deny tcp from table(25) to any dst-port 25
00700 deny tcp from 192.168.0.0/16 to table(26) dst-port 25
00750 allow ip from 192.168.0.0/16 to any - this rule we have to use (explaned below) 00800 pipe 10 ip from 192.168.0.0/16 to any - main rule for this bridge
65535 allow ip from any to any

pipes:
# BW for packets with SYN flag and UDP-6881
${fw} pipe 665 config mask src-ip 0xffffffff bw 384Kbit/s
${fw} pipe 666 config mask src-ip 0xffffffff bw 64Kbit/s
# Outgoing BW for each IP
${fw} pipe 10 config mask src-ip 0xffffffff bw 5120Kbit/s

table 25 has about 100 IP's
table 26 has about 15 sub-networks

this bridge serves about 25K subscribers with IP's from network: 192.168.0.0/16

current traffic:
netstat -bdh -w1 -I ix1

         input            ix1           output
packets errs idrops bytes packets errs bytes colls drops 607K 0 0 753M 452K 0 88M 0 0 603K 0 0 750M 449K 0 87M 0 0 604K 0 0 751M 448K 0 88M 0 0 604K 0 0 747M 452K 0 92M 0 0

all traffic:
netstat -bdh -w1

         input        (Total)           output
packets errs idrops bytes packets errs bytes colls drops 2M 0 0 1.6G 2M 0 1.6G 0 0 2M 0 0 1.6G 2M 0 1.6G 0 0


current CPU:
CPU 0: 31.1% user, 0.0% nice, 56.1% system, 5.1% interrupt, 7.7% idle CPU 1: 0.0% user, 0.0% nice, 0.5% system, 8.2% interrupt, 91.3% idle CPU 2: 0.0% user, 0.0% nice, 0.0% system, 4.6% interrupt, 95.4% idle CPU 3: 0.0% user, 0.0% nice, 0.5% system, 7.1% interrupt, 92.3% idle

THE Question:
is it normal for kipfw to eat so much resoures ?

660 root        99    0   873M   325M CPU0    0 272:03  91.46% kipfw

Also, the rule #750 I have to place into ruleset, cos without it kipfw begins to use all 100%

00750 allow ip from 192.168.0.0/16 to any
00800 pipe 10 ip from 192.168.0.0/16 to any - this rule is the main for using of this bridge,

it assigns the same outgoing bandwidth for each of IP addresses - 5120Kbit/s (5Mbps)


# BW for packets with SYN flag and UDP-6881
${fw} pipe 665 config mask src-ip 0xffffffff bw 384Kbit/s
${fw} pipe 666 config mask src-ip 0xffffffff bw 64Kbit/s
# Outgoing BW for each IP
${fw} pipe 10 config mask src-ip 0xffffffff bw 5120Kbit/s

With working rule #800 after 30-50 mins kipfw begins to use 100% in top -PHS and incoming (for users) traffic downs from 750Mbytes/s (about 6Gbit/s) to 330Mbytes/s (2.6Gbit/s), delay increases from 65ms to 250ms and high percentage of drops.

Is it real limit of using netmap-ipfw ? We can give any additional info if it will be usefull to expand limits of kipfw.

With regards and happy New Year !

Azamat B. Umurzakov
AkNet ISP


_______________________________________________
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Reply via email to