Hello group,
I'm hunting some bugs in FreeBSD's pf and I thought to ask here to
better undernstad how the global source tracking is supposed to work in
the reference implementation of pf. Or maybe it does not work, maybe
this ia s bug report.
I have the following ruleset on OpenBSD 7.3. The LB Nodes run a web
server serving just their hostname and have a static route towards the
LB Pool via the OpenBSD router with pf so that they can be clients too.
table <lb_nodes> {
10.64.2.31
10.64.2.32
10.64.2.33
}
table <lb_pool> {
10.64.2.30
}
pass in quick on vio0 \
inet proto tcp to <lb_pool> port 81 \
rdr-to <lb_nodes> port 80 round-robin sticky-address \
keep state (source-track global max-src-states 2)
pass in quick on vio0 \
inet proto tcp to <lb_pool> port 82 \
rdr-to <lb_nodes> port 80 round-robin sticky-address \
keep state (source-track global max-src-states 2)
pass in quick on vio0 \
inet proto tcp to <lb_pool> port 83 \
rdr-to <lb_nodes> port 80 round-robin sticky-address \
keep state (source-track global max-src-states 2)
pass out quick on vio0 \
inet proto tcp to <lb_nodes> port 80 \
nat-to vio0 \
keep state
Now let's make some connections from node 31:
[14:03:30] kajetan-test-aw-31 ~/ # curl http://10.64.2.30:81
kajetan-test-aw-31.test.ig.local
[14:04:33] kajetan-test-aw-31 ~/ # curl http://10.64.2.30:81
kajetan-test-aw-31.test.ig.local
[14:04:33] kajetan-test-aw-31 ~/ # curl http://10.64.2.30:81
^C
Timeout. So far so good.
Let's make more connections from the same source to another rule:
[14:04:39] kajetan-test-aw-31 ~/ # curl http://10.64.2.30:82
kajetan-test-aw-31.test.ig.local
[14:04:40] kajetan-test-aw-31 ~/ # curl http://10.64.2.30:82
kajetan-test-aw-31.test.ig.local
[14:04:40] kajetan-test-aw-31 ~/ # curl http://10.64.2.30:82
^C
For each destination port, and thus each rule, 2 connections have passed
and the 3rd one resulted in a timeout. But that's not what the manual
says. With global tracking "The number of states created by all rules
that use this option is limited" according to the pf.conf man page. And
yet tracking has been performed per rule:
10.64.2.31 ( states 2, connections 2, rate 0.0/0s )
age 00:00:05, 20 pkts, 1764 bytes, rule 1
10.64.2.31 rdr-to 10.64.2.31 ( states 2, connections 0, rate 0.0/0s )
age 00:00:05, 20 pkts, 1764 bytes, rule 1
10.64.2.31 ( states 2, connections 2, rate 0.0/0s )
age 00:00:07, 20 pkts, 1764 bytes, rule 0
10.64.2.31 rdr-to 10.64.2.31 ( states 2, connections 0, rate 0.0/0s )
age 00:00:07, 20 pkts, 1764 bytes, rule 0
I've looked around the code and it looks like the type of tracking is
ignored. All that is checked is the PFRULE_SRCTRACK, while
PFRULE_RULESRCTRACK is not:
pf_test_rule(…) {
…
if (r->rule_flag & PFRULE_SRCTRACK &&
pf_insert_src_node(&ctx.sns[PF_SN_NONE], r, PF_SN_NONE,
pd->af, pd->src, NULL, NULL) != 0) {
REASON_SET(&ctx.reason, PFRES_SRCLIMIT);
goto cleanup;
}
I've checked the commit history and at some point there was a flag
"global" in pf_insert_src_node but it seems to have never been fully
implemmented and now it's gone. I think that to fix this issue it would
be proper to bind the src nodes of type PF_SN_NONE to pf_default_rule if
PFRULE_RULESRCTRACK is not given and to the current rule if it is.
Another issue is with this limit:
max-src-nodes number
Limits the maximum number of source addresses which can
simultaneously have state table entries.
The description states that the number of source addresses is limited,
but it is not - it's the number of src nodes. For a ruleset similar to
above but with max-src-nodes:
pass in quick on vio0 \
inet proto tcp to <lb_pool> port 81 \
rdr-to <lb_nodes> port 80 round-robin sticky-address \
keep state (max-src-nodes 2)
every client creates 2 src nodes. One of the RDR type and one of the
NONE type. So the limit of clients is limit of nodes * 0.5. What if a
rule cointains a mix of rdr-to / nat-to / route-to, each with its own
src node? Maybe it would be better to have the limit checks in
pf_insert_src_node() and rule->src_nodes counter to apply only to the
node of PF_SN_NONE type.
--
| pozdrawiam / greetings | Powered by macOS, Debian and FreeBSD |
| Kajetan Staszkiewicz | www: http://vegeta.tuxpowered.net |
`------------------------^--------------------------------------'
