Hello. Thank you very much, Tom.
Just one correction, not related to the patch itself, but to the example
(should anyone be interested).
When you have NAT on the exit interface, upload is also affected and
filters in tcfilters based on source also won't work. My bad.
What would work:
1) Forget the tcfilters file in this case
2) Use MARK in forward chain to control upload and CONNMARK to control
download in /etc/shorewall/mangle file, like so:
#ACTION SOURCE DEST PROTO DPORT (...)
# limit uploads based on source, since we have NAT on the exit interface,
# tcfilters won't work
MARK(10):F 10.100.100.0/24 enp1s0
# limit downloads of connections started by LAN users to the Internet
# this only works with the aforementioned conntrack filter action when
# setting up mirred ifb, as it restores the connmark to the mark
# and LAN users' download traffic will get the 11:100 class (defined in
# tcclasses) applied
CONNMARK(11):F 10.100.100.0/24 enp1s0
(this, of course, will only fully work in shorewall 5.2.7)
Best regards.
On 08/07/20 15:55, Tom Eastep wrote:
> Hi Rodrigo,
>
> On 7/7/20 12:14 PM, Rodrigo Araujo wrote:
>> Hello.
>>
>> This is more like a pull request. If the users mailing list is not the
>> appropriate, I apologize and please tell me where should I submit it.
> Normally, proposed patches are submitted via the shorewall-devel mailing
> list.
>> One big problem when using ifb (redirected) interfaces for download QoS
>> control and NAT is that you can't usually create tcfilters based on a
>> internal destination address. That's because when packets enter the
>> outside interface, they are redirected to the ifb interface before
>> reverse NAT or any other netfilter hooks.
>>
>> However, fairly recent kernels (at least the one that comes with
>> RHEL/CentOS 7, which has now a few years) have a tc filter action named
>> connmark that helps to create a workaround, but shorewall doesn't
>> support it.
>>
>> Using the following (crude) patch I was able to take advantage of that
>> feature:
>>
>> --- /root/Tc.pm.bkp 2020-07-07 17:26:27.329581191 +0100
>> +++ /usr/lib/perl5/site_perl/Shorewall/Tc.pm 2020-07-07
>> 17:28:14.249592180 +0100
>> @@ -1865,7 +1865,7 @@
>> for my $rdev ( @{$devref->{redirected}} ) {
>> my $phyrdev = physical_name( $rdev );
>> emit ( "run_tc qdisc add dev $phyrdev handle ffff: ingress" );
>> - emit( "run_tc filter add dev $phyrdev parent ffff: protocol all
>> u32 match u32 0 0 action mirred egress redirect dev $device > /dev/null" );
>> + emit( "run_tc filter add dev $phyrdev parent ffff: protocol all
>> u32 match u32 0 0 action connmark action mirred egress redirect dev
>> $device > /dev/null" );
>> }
>>
>> for my $class ( @tcclasses ) {
>>
>>
>> Of course this should be optional. My suggestion is to have a new
>> possible OPTION in tcdevices named "connmark", only valid when
>> REDIRECTED INTERFACES is not empty. If present, the mirred filter is
>> generated as above suggested, and if not present it stays as the current
>> behaviour.
>>
>> Perhaps some extra code to determine if the tc filter action connmark is
>> supported and set a new capability would be nice, but since it's already
>> optional and should fail anyway if not supported, I don't think that
>> would be really necessary.Please improve as needed - my proposed patch
>> (against shorewall 5.2.5-2, and already with the option parsing) is
>> attached in this message.
>>
>> A minimal example configuration using this suggested feature follows.
>> For this scenario, there is a single 1 Gbps Internet connection. LAN
>> users shouldn't use more than 50 mbps for upload and 100 for download.
>> The rest should be available to our servers, who can also "borrow"
>> bandwidth from the LAN network when it's not in use.
>> Let's suppose that 10.100.100.0/24 corresponds to the lan network.
>>
>> Shorewall configuration files:
>>
>>
>> /etc/shorewall/zones:
>> #ZONE TYPE OPTIONS IN_OPTIONS OUT_OPTIONS
>> fw firewall
>> net ipv4
>> lan ipv4 routeback
>> srv ipv4 routeback
>>
>> /etc/shorewall/interfaces:
>> #ZONE INTERFACE OPTIONS
>> net enp1s0
>> lan enp2s0
>> srv enp3s0
>> # our ifb interface, shouldn't be necessary to declare but doesn't hurt
>> - ifb0
>>
>>
>> /etc/shorewall/policy:
>> #SOURCE DEST POLICY LOGLEVEL RATE CONNLIMIT
>> $FW all ACCEPT
>> lan net ACCEPT
>> srv net ACCEPT
>>
>>
>> /etc/shorewall/rules:
>> #ACTION SOURCE DEST PROTO DPORT SPORT
>> ORIGDEST RATE USER MARK CONNLIMIT TIME HEADERS
>> SWITCH HELPER
>> ?SECTION NEW
>> ACCEPT lan $FW tcp 22
>> (...)
>>
>>
>> /etc/shorewall/snat:
>> #ACTION SOURCE DEST PROTO PORT IPSEC MARK
>> USER SWITCH ORIGDEST PROBABILITY
>> MASQUERADE - enp1s0
>>
>>
>> /etc/shorewall/shorewall.conf:
>> (...)
>> TC_ENABLED=Internal
>>
>>
>> /etc/shorewall/tcdevices:
>> #NUMBER: IN-BANDWITH OUT-BANDWIDTH OPTIONS REDIRECTED
>> #INTERFACE INTERFACES
>> ## net upload
>> 10:enp1s0 - 1000mbit htb
>> ## net download
>> 11:ifb0 - 1000mbit htb,connmark enp1s0
>>
>>
>> /etc/shorewall/tcclasses:
>> #INTERFACE MARK RATE CEIL PRIO OPTIONS
>>
>> 10:5000 111 500kbit full 10
>> tcp-ack,tos-minimize-delay
>> 11:5000 110 500kbit full 10
>> tcp-ack,tos-minimize-delay
>>
>> 10:1000 100 full-50500 full 20 default
>> 11:1000 101 full-100500 full 20 default
>>
>> 10:50 10 50mbit 50mbit 101
>> flow=nfct-src
>> 11:100 11 100mbit 100mbit 101 flow=dst
>>
>>
>> /etc/shorewall/tcfilters:
>> #INTERFACE: SOURCE DEST PROTO DEST SOURCE
>> TOS LENGTH PRIORITY
>> #CLASS PORT(S) PORT(S)
>> ## limit LAN upload - works
>> 10:50 10.100.100.0/24
>> ## limit LAN download - DOESN'T WORK BECAUSE OF MASQUERADE ON enp1s0
>> !!!! (snat file)
>> #11:100 - 10.100.100.0/24
>>
>>
>> /etc/shorewall/mangle:
>> #ACTION SOURCE DEST PROTO DPORT
>> SPORT USER TEST LENGTH TOS CONNBYTES HELPER
>> PROBABILITY DSCP SWITCH
>> ## limit downloads of connections started by LAN users to the Internet
>> ## this only works with the aforementioned conntrack filter action when
>> setting up mirred ifb, as it restores the connmark to the mark
>> ## and LAN users' download traffic will get the 11:100 class (defined in
>> tcclasses) applied
>> CONNMARK(11) 10.100.100.0/24 -
> In order for this to work with Shorewall's Multi-ISP facility, you
> probably would want:
>
> CONNMARK(11):T 10.100.100.0/24 { TEST=!0x/0xff }
>
> See below.
>
>>
>> Only problem I see is when using more than one Internet provider, having
>> configured them in the /etc/shorewall/providers. Perhaps in that case
>> one is able to use xmark masks to have the best of both worlds? Any ideas?
> The Shorewall TC code uses the low-order TC_BITS bits of the fwmark and
> masks off the left-most bits. So the fact that the 'connmark' tc filler
> action restores the entire connmark is fine. In the case of multiple
> providers, the provider-generator PREROUTING rules will re-restore the
> provider field from the connection mark into the packet mark.
>
>> Anyway, in hope that you also find this a useful feature... is there any
>> chance that shorewall officially supports this in a near future release? :)
>>
> I'll schedule it for 5.2.7.
>
> Thanks,
>
> -Tom
>
>
> _______________________________________________
> Shorewall-users mailing list
> Shorewall-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/shorewall-users
signature.asc
Description: OpenPGP digital signature
_______________________________________________ Shorewall-users mailing list Shorewall-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/shorewall-users
