Hi,
Here's the summary of the IRC meeting.
---
COMMUNITY MEETING
Place: #openvpn-meeting on libera.chat
Date: Wed 23rd February 2022
Time: 10:30 CET (9:30 UTC)
Planned meeting topics for this meeting were here:
<https://community.openvpn.net/openvpn/wiki/Topics-2022-02-23>
Your local meeting time is easy to check from services such as
<http://www.timeanddate.com/worldclock>
SUMMARY
cron2, d12fk, lev, mattock, ordex and plaisthos participated in this
meeting.
---
Noted that both Trac and Pwm are unable to send emails right now, for
unrelated reasons. Mattock will try to get Sendgrid SMTP credentials
this week to resolve both of these issues (plus get credentials for
Buildbot).
---
Talked about making --fragment pushable. Since we do not modify the
actual buffers anymore, pushing --fragment should be possible now. We
also can't kill --fragment as long as we support tap - people do use
openvpn to link together ethernet segments.
---
Talked about --mtu-disc. It works for "over IPv4" now, and the missing
code for "over IPv6" is on the list for review and ACK. That option
seems rarely used as it has been broken at least since 2.4.3 and never
worked for IPv6 [and we had not heard about it].
---
Talked about the OpenVPN control channel and noted it can't handle
smaller MTU paths. At the moment openvpn assumes one control packet per
TLS record and one TLS record per packet. This is the reason why TLS
record splitting breaks OpenVPN.
---
Talked about --mtu-test, one of the other exotic OpenVPN --mtu-*
options. It exchanges a number of packets of varying sizes over OCC,
trying to figure out if "large packets" get eaten by gremlins on the way
(like, fragmenting routers). The implemention was rewritten as part of
the frame stuff, and lost the "varying sizes" part. So, that needs to be
brought back.
---
Talked about --mssfix. The new 2.6 behaviour is "--mssfix defaults to
'1492 mtu'" (good), and "if --tun-mtu != 1500" is configured, mssfix
defaults to *off* (not good). Cron2 proposed that a good default might
be "if tun-mtu != 1500, then use the inner MTU for mssfix".
--
Full chatlog attached
(11.28.49) mattock: hi
(11.29.01) mattock: my IRC blew up for no particular reason, but now it is fixed
(11.29.49) cron2: hah
(11.30.05) cron2: you were just hiding to avoid my questions about trac and
e-mail sending...
(11.30.18) mattock: that's being pushed internally
(11.30.34) mattock: we're going to get a bunch of SMTP credentials that will
send email through Sendgrid
(11.30.42) mattock: that will fix our email sending issues in Trac and pwm
(11.31.52) cron2: "we're going to get" sounds about as exciting as "one day,
community might have IPv6"
(11.32.01) cron2: this is broken since over a week now
(11.32.43) cron2: anyway... meeting
(11.33.45) d12fk: hi
(11.34.04) cron2: hi :)
(11.34.35) lev__: guten tag
(11.34.50) ordex: sorry - had a thing to put to sleep
(11.35.04) plaisthos: hey all
(11.35.46) mattock: ordex: a small human?
(11.36.10) ordex: yeah
(11.36.11) ordex: quite small
(11.37.20) d12fk: very cooperative
(11.38.43) cron2: mattock: can you explain to the SMTP credential authorities
that they broke a production service and some people are getting really really
annoyed at Corp Ops?
(11.39.03) cron2: like "can we not just move this to someone competent" level
annoyed
(11.39.31) mattock: I will relay your annoyance and bump up the priority of the
ticket
(11.40.02) plaisthos: cron2: trust us, they are breaking not only community
stuff :/
(11.40.17) ***ordex listens to Limp Bizkit - Breaking stuff
(11.40.20) plaisthos: we are as annoyed as you are
(11.40.25) mattock: breaking is one thing, getting things fixed quickly is
another
(11.40.51) mattock: they only broke 50% of our email delivery, the rest was
just legacy configuration that broke by itself
(11.40.52) cron2: plaisthos: meh :(
(11.42.23) cron2: anyway, shall we look at the agenda? :-)
(11.43.58) d12fk: yeah why not ;-)
(11.43.59) mattock: ok, there was a response from the sendgrid person
(11.44.12) mattock: I'm trying to schedule a meeting with her tomorrow or on
Friday
(11.45.09) cron2: that is the right spirit, something as trivial as SMTP
credentials must never be solved just by an e-mail (or corp wiki or whatnot),
if a meeting is possible
(11.45.15) plaisthos: you need a meeting for credentilas?
(11.45.43) cron2: plaisthos: OpenVPN Corp is practising for "We Will Be A
MegaCorp One Day", I think
(11.46.21) cron2: one of my consulting customers is, indeed, a DAX company, and
there is no way to get anything done without meetings with 10+ people in there
that do not server any other purpose than to agree who to invite for the next
meeting
(11.46.26) plaisthos: I not yet have to fill out forms where I detail what
expense goes into what cost object
(11.46.53) mattock: plaisthos: it seems I do need a meeting, yes
(11.47.11) cron2: that is the nicer part of working with a DAX company... "this
is only 5 millions? yeah, just order it, we do not need to ask for board
approval for such small sums"
(11.47.36) mattock: yeah :)
(11.47.38) cron2: (unfortunately, they refuse to give these millions to *me*)
(11.47.54) cron2: a-ny-way... agenda...?
(11.48.08) ordex: cron2: maybe I can help with that ;p
(11.48.39) cron2: with the agenda or with the millions? ;-)
(11.49.01) mattock: https://community.openvpn.net/openvpn/wiki/Topics-2022-02-23
(11.53.29) ordex: the latter :p
(11.56.29) cron2: I'm all ears ;-)
(11.57.43) cron2: but until then...
(11.57.53) cron2: I have ideas and wishes about fragment, and mtu, and mssfix...
(11.58.18) cron2: mostly plaisthos, but also ordex/lev due to the DCO aspect
(11.58.37) cron2: plaisthos: *bump*
(11.58.39) plaisthos: My current idea/plan is to change the default for P2MP
server
(11.58.48) cron2: can we start on top?
(12.00.23) cron2: there is a list of topics in, well, the Topics page, and I'd
like to discuss them in an orderly way
(12.00.58) plaisthos: oh sorry
(12.01.23) plaisthos: hm fragment pushable might be possible
(12.01.56) plaisthos: but it is the question if we want to invest into that or
not
(12.02.23) ordex: mah, shouldn't --fragment just die?
(12.02.29) plaisthos: but since we do not modify the actual buffers anymore,
fragment pushable should be possible now.
(12.03.04) cron2: this is what I thought
(12.03.51) cron2: and we can't kill fragment as long as we support tap - people
do use openvpn to link together ethernet segments
(12.04.06) cron2: lan-bridge-tap-openvpn-tap-bridge-lan
(12.04.47) cron2: and "LAN has 1500 mtu" needs "some way to transport this
across internet-1500", so either --fragment, or fragment external UDP packets,
which might or might not work well
(12.05.31) ordex: uhmpff
(12.05.32) ordex: ok
(12.05.39) cron2: for tun-based "routed" VPNs, most traffic is TCP today, so
mssfix works beautifully
(12.06.13) cron2: ok, anyway, i'll have a look into making fragment pushable
(12.06.47) lev__: is this something we really need in 2.6
(12.06.47) cron2: next, --mtu-disc - this is basically (now) an info item only.
It works for "over IPv4" now, and the missing code for "over IPv6" is on the
list for review and ACK
(12.07.14) cron2: lev__: I assume it's trivial now, just changing the option
permission mask
(12.07.56) cron2: so, mostly testing what happens. If it needs larger rewrites
(like, initializing the fragment buffer), maybe not
(12.08.39) cron2: --mtu-disc was broken at least since 2.4.3 and never worked
for IPv6... so this is not exactly one of the most-used options out there :-)
(12.08.53) ordex: yeah
(12.09.06) ordex: one of those easter eggs included in openvpn :P
(12.09.22) plaisthos: and it is --fragment only anyway
(12.09.26) plaisthos: which is not even documented
(12.09.42) plaisthos: or only enabled if fragment is also enabled
(12.09.46) plaisthos: one of those two
(12.09.53) cron2: but, to raise awareness: testing with --mtu-disc on paths
that are arbitrarily limited to MTU 1300 or less, brought up the issue of "our
control channel cannot handle smaller-MTU paths"
(12.10.09) plaisthos: yes
(12.10.24) plaisthos: Requires refactoring of our TLS stuff
(12.10.30) cron2: plaisthos: not sure what it does in 2.5 and earlier :-) - now
it sets mssfix + fragment to "$value mtu" which is really nice :-)
(12.10.54) plaisthos: at the moment openvpn assumes one control packet per TLS
record and one TLS record per packet
(12.11.32) plaisthos: The same reason why TLS record splitting breaks OpenVPN
(12.11.32) cron2: and some of these are just big (not even the IV_ stuff, it
seems to be the basic certificate stuff that is already producing fairly big
packets)
(12.12.02) plaisthos: for the basic certificate stuff that is managable
(12.12.19) plaisthos: you can force the TLS library to use smaller packets
(12.12.24) plaisthos: but yeah
(12.12.54) plaisthos: probably needs to be looked into at some point
(12.13.09) cron2: I don't think it's a hard problem today, but for doing funky
stuff like "openvpn over an openvpn-dco session with mtu 1400" it might become
an issue
(12.13.27) cron2: but, wanted to mention it, so people have heard about it
(12.13.37) cron2: next, --mtu-test :-)
(12.15.32) cron2: so, this is one of the other exotic OpenVPN --mtu-* options -
it does exchange a number of packets of varying sizes over OCC, trying to
figure out if "large packets" get eaten by gremlins on the way (like,
fragmenting routers)
(12.15.57) plaisthos: yeah, we need to figure out what we want it to do
(12.15.58) cron2: it needed to be rewritten as part of the frame stuff, and
lost the "varying sizes" part :-) - so this needs a followup patch to bring it
back
(12.16.16) plaisthos: the old code is also very wacky since it does not account
for cbc round up
(12.16.32) cron2: plaisthos: I think it should do what it tried before - figure
out what size of packets can be sent/received
(12.17.13) cron2: right now it seems to be only informational, but it could go
into setting --mssfix/--fragment?
(12.17.29) cron2: or maybe just log as diagnostic tool
(12.17.48) plaisthos: the problem is that it figures out what the largest
payload that arrives at the other side
(12.18.07) plaisthos: so it does have no idea if that payload causes external
udp fragmentation
(12.18.36) cron2: true, but if fragmentation works, that's good enough for me
(12.18.42) plaisthos: Or with the new code, since we have headroom, you might
even end up with something that the other side's tun does not accept
(12.18.46) cron2: so, some networks will eat fragments -> --mtu-test will
discover
(12.18.56) cron2: other will handle fragments fine -> --mtu-test will be happy
(12.19.26) cron2: I think we should document it as "what size of packets does
your network path handle (possibly fragmented)"
(12.19.33) plaisthos: it will happily tell you that the macos on other side can
accept 1600 packets but if you send a 1600 byte packet, the macos utun will
drop it
(12.19.38) cron2: if you combine with --mtu-disc yes, openvpn will send DF
(12.20.00) plaisthos: cron2: fragmented UDP is always DF
(12.20.08) cron2: no?
(12.20.11) plaisthos: I never seen a packet without DF
(12.20.25) cron2: in IPv4, fragments can be fragmented further on the path
(12.20.28) plaisthos: cron2: DF as in "router on the path may not fragment this
packet"
(12.20.33) cron2: yes :-)
(12.20.56) cron2: but I think you misunderstood me
(12.20.57) plaisthos: do we really have packets without DF?
(12.21.06) cron2: normal openvpn will send without DF
(12.21.13) cron2: openvpn with --mtu-disc yes will send with DF
(12.21.18) plaisthos: I thought otherwise
(12.21.26) plaisthos: well okay
(12.21.41) cron2: all my t_client "ping -s 1440" tests would fail miserably
otherwise :-)
(12.22.08) plaisthos: cron2: do you have anything lower than 1500 mtu on the
path there?
(12.22.17) cron2: --mtu-disc yes -> DF -> router sends ICMP "packet too big!"
back -> error
(12.22.32) cron2: plaisthos: no, but that also affects locally generated
packets > outgoing MTU
(12.22.49) cron2: (so you can see the effect if you have a local route "via
$gateway mtu 1400")
(12.23.03) cron2: ((on linux, --mtu-disc is really linux-only))
(12.24.40) mattock: 6 minutes left, --mssfix?
(12.25.04) cron2: yeah, let's quickly explain what I had in mind
(12.25.11) plaisthos: cron2: hm my openvpn connection really has DF not set but
the return path packets somehow got DF
(12.25.20) plaisthos: seems to be a whole can of worms
(12.25.29) cron2: it is
(12.26.14) cron2: anyway, --mssfix. The new 2.6 behavioir is "--mssfix
defaults to '1492 mtu'" (good), and "if --tun-mtu != 1500" is configured,
mssfix defaults to *off* (not good)
(12.26.33) cron2: I find mssfix tremendously useful, especially if there are
MTU jumps on the whole network path
(12.27.07) cron2: so, what I think would be a good default is "if tun-mtu !=
1500, then use the inner MTU for mssfix"
(12.27.27) plaisthos: cron2: that happens automatically
(12.27.32) cron2: for TCP sessions originating left and right of the tunnel,
they will have that correct MSS value already (because, interface MTU)
(12.27.40) plaisthos: ah
(12.27.42) cron2: but for TCP sessions originating elsewhere, not
(12.27.46) plaisthos: I understand what you mean
(12.27.52) cron2: like, 1500 LAN -> 1400 tun -> 1500 LAN
(12.28.06) cron2: or maybe even 9000 LAN -> 8000 tun -> 9000 LAN (for whatever
funky reason)
(12.28.23) plaisthos: cron2: yeah I completely overlooked that case
(12.28.59) plaisthos: I think before we had a fixed mssfix that was not
dependent on MTU size and that was nonsense for != 1500 cases
(12.29.00) cron2: I'm just not sure if our new code likes this case, or wants
to go through hoops with "external packet size" to reach the "trivial" mss
setting
(12.29.40) cron2: plaisthos: yes, not forcing the old default for >1500 tuns is
a good thing. For <1500, it won't have changed much (it never raises the MSS),
but it wasn't useful either
(12.32.55) cron2: .oO(since we lost everyone else on the way, I think, maybe we
need to postpone the rest to +2 weeks... I won't be here next week, it's
snowboarding time again)
(12.33.19) plaisthos: hm
(12.33.35) plaisthos: mssfix would be then tun-mtu - 20? %)
(12.33.47) plaisthos: I am getting confused
(12.33.48) cron2: mattock: relevant for you - we discussed the CVE patch with
dazo, and I could do a release "this week" or "after mar 5"
(12.33.55) plaisthos: argh blergh
(12.34.03) plaisthos: mssfix uses the outside
(12.34.09) d12fk: not lost, just listening
(12.34.37) mattock: yep, me too
(12.34.41) cron2: plaisthos: yes, the config value is "outside", which does not
help much here. But at some point there is an "inside" value :-)
(12.34.47) mattock: trying to understand if there is an agreement --mssfix
(12.34.50) plaisthos: cron2: I think we probably need some flag that says,
clamp mss to mtu
(12.35.15) plaisthos: like mssfix_encap bool
(12.35.29) plaisthos: or make that mssfix_flags and then have a CAMP_TO_MTU
(12.35.36) plaisthos: CLAMP_TO_MTU
(12.35.45) ***cron2 is in the MTU camp :)
(12.35.59) mattock: cron2: "this week" is quite soon, I may have to fix my
msibuilder VM (spectre mitigation + openssl3), so after 5th march sounds better
(12.36.03) plaisthos: and the the calc mssfix calc method just does if
(clamp_flag) mssfix = tunmtu, done
(12.36.41) cron2: that could be an internal flag, which is set iff(mssfix
default + tun_mtu != 1500)
(12.36.56) cron2: exposing another external flag sounds like a good way for
user confusion
(12.37.18) plaisthos: nah external is fine
(12.37.26) plaisthos: mssfix 400 fixed
(12.37.28) plaisthos: or something
(12.37.43) lev__: speaking of mssfix, is this something we need to add to
dco/dco-win ?
(12.38.11) plaisthos: if the flag fixed is used, OpenVPN will override the MSS
value with this value if the value is larger
(12.38.20) cron2: but I wouldn't want to do my own math to come up with the
"400"... if I have --tun-mtu 1400, the computer is smarter than me and can
calcultat the right MSS
(12.38.47) plaisthos: maybe you have some reason to have a specific MSS value %)
(12.39.07) plaisthos: because while your tunnel is okay you want to abuse
OpenVPN to fix your unrelated mss problem
(12.39.34) cron2: haha, swiss army knife again ;-)
(12.39.46) cron2: lev__, ordex: yes, I think the DCO path needs to handle mssfix
(12.40.14) cron2: if we say "we will only offer a smaller MTU" *and* "but no
mssfix", we'll see breakage for TCP sessions
(12.41.08) mattock: I need to get some lunch now, summary is waiting for the
conclusion :)
_______________________________________________
Openvpn-devel mailing list
Openvpn-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/openvpn-devel
