[Bug 260973] pf: firewall rules stop matching when vnet jails share interface names with the host
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260973
Bug ID: 260973
Summary: pf: firewall rules stop matching when vnet jails share
interface names with the host
Product: Base System
Version: 13.0-STABLE
Hardware: Any
OS: Any
Status: New
Severity: Affects Some People
Priority: ---
Component: kern
Assignee: b...@freebsd.org
Reporter: tho...@gibfest.dk
Hello,
I've been building a new vnet jailhost on 13 and I am hitting a weird issue
where pf stops permitting traffic it clearly has rules to allow after
interfaces inside vnet jails are renamed to the same name as the host interface
with the pf rule.
This is on FreeBSD nuc1.servers.bornhack.org 13.0-STABLE FreeBSD 13.0-STABLE #1
stable/13-d208638c5: Wed Jan 5 13:32:08 UTC 2022
r...@nuc1.servers.bornhack.org:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64
The complete ruleset is pretty complex but I've managed to cook it down to a
few lines:
[tykling@nuc1 ~]$ cat testpf.conf
block log all
set skip on lo0
pass in quick on { em0 } proto { tcp } from { 85.235.250.87 } to { (em0) } port
{ 22 }
[tykling@nuc1 ~]$
The host has an em0 interface:
[tykling@nuc1 ~]$ ifconfig em0
em0: flags=8863 metric 0 mtu 1500
options=481049b
ether 1c:69:7a:ab:fe:be
inet 85.209.118.130/28 broadcast 85.209.118.143
inet6 fe80::1e69:7aff:feab:febe%em0/64 scopeid 0x1
inet6 2a09:94c4:55d1:7680::82/64
media: Ethernet autoselect (1000baseT )
status: active
nd6 options=21
[tykling@nuc1 ~]$
The issue seems to be triggered by renaming epair interfaces inside vnet jails
to the same name as an interface on the host.
The above pf ruleset works and keeps working if I don't start any vnet jails.
It also keeps working if I start vnet jails but don't rename interfaces. It
also keeps working if I start vnet jails but rename the interfaces to something
other than em0.
Existing states established before the issue happens keep working (I am working
remote via ssh on the server), but new states seem to just ignore the permit
rule on em0, and the traffic gets blocked even though a rule should permit it:
06:08:46.357935 rule 0/0(match): block in on em0: 85.235.250.87.40108 >
85.209.118.130.22: Flags [S], seq 909787121, win 65535, options [mss
1460,nop,wscale 6,sackOK,TS val 799486870 ecr 0], length 0
06:08:47.358590 rule 0/0(match): block in on em0: 85.235.250.87.40108 >
85.209.118.130.22: Flags [S], seq 909787121, win 65535, options [mss
1460,nop,wscale 6,sackOK,TS val 799487870 ecr 0], length 0
06:08:49.557897 rule 0/0(match): block in on em0: 85.235.250.87.40108 >
85.209.118.130.22: Flags [S], seq 909787121, win 65535, options [mss
1460,nop,wscale 6,sackOK,TS val 799490070 ecr 0], length 0
A wild guess as to the reason might be a race leading to some confusion over
which em0 interface is which?
Some more observations:
- It didn't seem to happen with just one vnet jail when I tried narrowing it
down. Enabling and starting three more made the problem occur almost instantly.
- Rebooting with four jails plus the above ruleset enabled means never getting
any contact to the server at all (ie. the problem manifests from boot).
- Results with two jails were less consistent. The number of jails/interface
renames seem to play a role in whether or not the issue is triggered.
- A "service jail restart" will trigger it almost instantly if it doesn't
happen right away.
- Renaming interfaces to something other than "em0" also works without any
issues.
I hope reproducing will be possible, I've included the jail.conf file for one
of the jails below:
[tykling@nuc1 ~]$ cat /var/run/jail.syslog1_servers_bornhack_org.conf
# Generated by rc.d/jail at 2022-01-06 08:19:08
syslog1_servers_bornhack_org {
host.hostname = "syslog1.servers.bornhack.org";
path = "/usr/jails/syslog1.servers.bornhack.org";
vnet;
vnet.interface = "epair2b";
exec.clean;
exec.system_user = "root";
exec.jail_user = "root";
exec.prestart += "ifconfig epair2a destroy 2>/dev/null || true &&
ifconfig epair2 create up && ifconfig epair2a up && ifconfig bridge1 addm
epair2a up";
exec.start += "/sbin/ifconfig epair2b name em0 && ifconfig em0
10.1.0.3/24 && ifconfig em0 inet6 2a09:94c4:55d1:76A0::3/64";
exec.start += "route add -inet default 10.1.0.1";
exec.start += "route add -inet6 default 2a09:94c4:55d1:76A0::1";
exec.poststop += "ifconfig bridge1 deletem epair2a && ifconfig epair2a
destroy";
exec.start += "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown jail";
exec.consolelog =
"/var/log/jail_syslog1_servers_bornhack_org_console.log";
mount.fstab = "/etc/fstab.syslog1_servers_bornhack_org";
allow.set_hostname = 0;
allow.sysvipc = 0;
enforce_statfs = "2";
}
[tykling@n
[Bug 260962] sysutils/openzfs fsck.zfs(8) does not detect OpenZFS pool degradation
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260962 --- Comment #2 from Andriy Gapon --- > because some Linux distributions expect But FreeBSD is not one. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 260973] pf: firewall rules stop matching when vnet jails share interface names with the host
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260973 --- Comment #1 from Thomas Steen Rasmussen / Tykling --- Maybe related https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=185619 Also, I forgot to mention, at some point yesterday while trying 100 things I saw the em0 on the host having multiple ether and hwaddr entries, the mac addresses were like the ones you see on epair interfaces. I have a screenshot of it if anyone is interested. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 260973] pf: firewall rules stop matching when vnet jails share interface names with the host
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260973 --- Comment #2 from Thomas Steen Rasmussen / Tykling --- This statement - Rebooting with four jails plus the above ruleset enabled means never getting any contact to the server at all (ie. the problem manifests from boot). is not true, my testing was off. The problem only shows up when vnet jails with the same interface names as on the host are stopped/restarted. This also explains why I had such a hard time reproducing it right after a reboot. It only happens when a jail has been started and is then stopped (or restarted) This fits the problem description in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=185619 perfectly -- You are receiving this mail because: You are the assignee for the bug.
[Bug 260979] libproc can fail to find debug symbols for jailed processes
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260979 Bug ID: 260979 Summary: libproc can fail to find debug symbols for jailed processes Product: Base System Version: CURRENT Hardware: Any OS: Any Status: New Severity: Affects Some People Priority: --- Component: bin Assignee: b...@freebsd.org Reporter: ma...@freebsd.org libproc knows about the .gnu_debuglink extension and will use it to find external debug files so that it can use them to do symbol resolution. When debugging a problem with lwhsu, we found that this doesn't quite work if the process is jailed but the debugger is running in the jail host. What libproc does is look up the path of a mapped file, check it for .gnu_debuglink, and then try to read /usr/lib/debug/.debug. On the jail host, this path should really be /usr/lib/debug/. In particular, libproc will fail to find the debug info. gdb has the same problem it seems. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 260962] sysutils/openzfs fsck.zfs(8) does not detect OpenZFS pool degradation
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260962 --- Comment #3 from Graham Perrin --- (In reply to Andriy Gapon from comment #2) Sure; if it's installed to FreeBSD then it should either: a) work (simply) as it does on Linux; or b) fail with an appropriate error. Thanks -- You are receiving this mail because: You are the assignee for the bug.
[Bug 260954] databases/mysqwl57-server: boot Problem
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260954 Jochen Neumeister changed: What|Removed |Added Summary|waiting for runnimg MySQL |databases/mysqwl57-server: ||boot Problem Status|New |Closed CC||jon...@freebsd.org Resolution|--- |Not Enough Information --- Comment #8 from Jochen Neumeister --- I can't see any bug message here. Please use this system in the future only if you want to report a bug. And then describe this problem clearly, and do not post umpteen responses in a row, thank you. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 260954] databases/mysqwl57-server: boot Problem
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260954 Jochen Neumeister changed: What|Removed |Added Assignee|b...@freebsd.org|jon...@freebsd.org -- You are receiving this mail because: You are the assignee for the bug.
[Bug 260979] libproc can fail to find debug symbols for jailed processes
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260979 Ed Maste changed: What|Removed |Added CC||ema...@freebsd.org --- Comment #1 from Ed Maste --- When looking for debug data where debuglink specifies ls.debug and the build-id is abcdef1234 gdb attempts to access, in order: /usr/lib/debug/.build-id/ab/cdef1234.debug /usr/bin/ls.debug /usr/bin/.debug/ls.debug /usr/lib/debug/usr/bin/ls.debug (from https://sourceware.org/gdb/onlinedocs/gdb/Separate-Debug-Files.html) gdb supports "set debug-file-directory directories" I don't see an obvious way this could be handled automatically. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 260979] libproc can fail to find debug symbols for jailed processes
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260979 --- Comment #2 from Ed Maste --- (by "this" I mean looking up standalone debug data in a jail-specific /usr/lib/debug directory) -- You are receiving this mail because: You are the assignee for the bug.
[Bug 224496] mpr and mps drivers seems to have issues with large seagate drives
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=224496 Alexander Motin changed: What|Removed |Added CC||m...@freebsd.org --- Comment #51 from Alexander Motin --- (In reply to Daniel Austin from comment #48) I see mpsutil/mprutil tools recently got support for `set ncq` subcommand, that should allow to disable NCQ. I've merged that into TrueNAS 12.0-U7. I haven't tried that myself, and it makes me shiver inside from its inefficiency, but I can believe that it may reduce maximum command latency in some scenarios, or may be even avoid command timeouts in situations when the disks or the HBAs can't schedule or process the NCQ commands reasonably. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 185619] [VNET] Name conflict not checked when a child vnet goes away and returns its interface(s) back to the parent
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=185619 Thomas Steen Rasmussen / Tykling changed: What|Removed |Added CC||tho...@gibfest.dk --- Comment #3 from Thomas Steen Rasmussen / Tykling --- This seems like it might be the same issue as https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260973 -- You are receiving this mail because: You are the assignee for the bug.
[Bug 260896] The 14-CURRENT system hangs if kern.vt.splash_cpu is set to 1 in loader.conf
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260896 --- Comment #3 from Oleg --- This issue occurs on both of my computers. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 260979] libproc can fail to find debug symbols for jailed processes
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260979 Mark Johnston changed: What|Removed |Added Status|New |Open --- Comment #3 from Mark Johnston --- (In reply to Ed Maste from comment #2) I'm not sure how best to deal with it either. We could look up the root path for the target process, and 1) prefix /usr/lib/debug, 2) strip that prefix from paths of mapped files, but none of this would be 100% reliable. It would perhaps be sufficient to have a way to explicitly specify the debug file prefix as a dtrace(1) option. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 224496] mpr and mps drivers seems to have issues with large seagate drives
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=224496 --- Comment #52 from Daniel Austin --- (In reply to Alexander Motin from comment #51) If it helps, it seems to be enclosure related... I can use NCQ on my disks and controller in another enclosure (e.g. I have an Areca 8 bay that's fine)... but when using my QNAP jbod enclosure (TL-D1600S) I get errors when NCQ is enabled. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 260963] vt newcons: mouse paste incorrectly adds newlines
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260963 Ed Maste changed: What|Removed |Added CC||ema...@freebsd.org --- Comment #1 from Ed Maste --- > If one marks a line that goes across more than one screen line, vt *always* > inserts a newline at each physical screen line end of the copy source. I just booted an old image I had around (12.0-CURRENT r327524) in QEMU, and sc behaves the same way. This would be good to address nonetheless. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 260963] vt newcons: mouse paste incorrectly adds newlines
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260963 --- Comment #2 from Ed Maste --- I cannot find the referenced report on hackers - could you please add a link to the mailing list archive? I want to make sure I'm correctly trying to reproduce the differing behaviour between sc and vt. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 260257] CAM_IO_STATS does not track MEDIUM ERRORs
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260257 --- Comment #8 from Alan Somers --- (In reply to Robert Wing from comment #7) So far this patch seems to be working. I've installed it on one system that's experienced two MEDIUM ERRORs since. I'll check on it daily. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 260399] freebsd-update: Downloading patches often fails repeatedly
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260399 --- Comment #9 from Some Signup --- My apologies for the delay in trying to update my machines. It's such a pain that I put it off until I can't put it off any longer. If I run freebsd-update with --debug I get to see thousands of errors downloading files. I can post them all here, but I don't think it's necessary. A couple of sample but unrelated entries: http://ipv4.aws.portsnap.freebsd.org/bp/8585c48bb921235425ee2865596c60b70aab551562ade66b65902e35b979a1f8-6cf7851b2ddd1d39ac9345fb4efe852f25da266ce85a6a3906f808c6287c81e6: 404 Error (ignored) http://ipv4.aws.portsnap.freebsd.org/f/a9be29debdb0c5186913b752ad4145f5221bd8944a58abbca1de03084d2acadf.gz: 400 Error (ignored) Some of these errors are wrong - the files exist, and in other cases the files are indeed missing. A couple of examples: curl http://ipv4.aws.portsnap.freebsd.org/bp/7f7a3e981fa345c60fc3dff57ed14af3b76c69d374cfaf50bbedcb9bbed2c013-85791eee88708d116494f0051e1e6930f227c9569f3d42d7658f8d01aee65a47 404 Not Found 404 Not Found nginx/1.20.1 curl --output thisoneisok.gz http://ipv4.aws.portsnap.freebsd.org/f/a9be29debdb0c5186913b752ad4145f5221bd8944a58abbca1de03084d2acadf.gz % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 869 100 8690 0 4456 0 --:--:-- --:--:-- --:--:-- 4456 So there appear to be two errors here: genuinely missing files, and files which exist but freebsd-update treats as missing. Interestingly, when I tried in a browser to get to 'http://ipv4.aws.portsnap.freebsd.org/bp' I see an error in my browser trying to get to 'http://ipv4.aws.portsnap.freebsd.org:8080/bp/'. I did not add this 8080 to the URL so I don't know where it's coming from (definitely not something locally, no proxies). It would appear to be a redirection coming from AWS. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug 260993] [regression] [clang] stable/12 buildworld broken: fatal error: 'clang/Basic/DiagnosticCommonKinds.inc' file not found
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260993 Bug ID: 260993 Summary: [regression] [clang] stable/12 buildworld broken: fatal error: 'clang/Basic/DiagnosticCommonKinds.inc' file not found Product: Base System Version: 12.3-STABLE Hardware: Any OS: Any Status: New Keywords: regression Severity: Affects Some People Priority: --- Component: misc Assignee: b...@freebsd.org Reporter: eu...@freebsd.org CC: d...@freebsd.org I have amd64 stable/12 system updated on 8 December 2020. I updated stable/12 sources with gitup on yesterday, 6 January 2022 and attempted to rebuild some NanoBSD image. This box successfully performed the build last time with fresh stable/12 sources on early December 2021. Now same build fails: Building /usr/obj/nanobsd.gw/usr/src/amd64.amd64/lib/clang/liblldb/Core/ModuleList.o --- Core/ModuleList.o --- In file included from /usr/src/contrib/llvm-project/lldb/source/Core/ModuleList.cpp:34: In file included from /usr/src/contrib/llvm-project/clang/include/clang/Driver/Driver.h:12: In file included from /usr/src/contrib/llvm-project/clang/include/clang/Basic/Diagnostic.h:17: /usr/src/contrib/llvm-project/clang/include/clang/Basic/DiagnosticIDs.h:71:10: fatal error: 'clang/ Basic/DiagnosticCommonKinds.inc' file not found #include "clang/Basic/DiagnosticCommonKinds.inc" ^~~ 1 error generated. *** [Core/ModuleList.o] Error code 1 make[6]: stopped in /usr/src/lib/clang/liblldb Here is full list of knobs used for build: CONF_BUILD=' LOADER_BZIP2_SUPPORT= MODULES_WITH_WORLD= WITHOUT_ACCT= WITHOUT_AMD= WITHOUT_APM= WITHOUT_ASSERT_DEBUG= WITHOUT_ATM= WITHOUT_AUDIT= WITHOUT_AUTHPH= WITHOUT_AUTOFS= WITHOUT_BINUTILS= WITHOUT_BLACKLIST= WITHOUT_BLACKLIST_SUPPORT= WITHOUT_BLUETOOTH= WITHOUT_CALENDAR= WITHOUT_CAPSICUM= WITHOUT_CCD= WITHOUT_CLANG= WITHOUT_CPP= WITHOUT_CTM= WITHOUT_CUSE= WITHOUT_CVS= WITHOUT_DEBUG_FILES= WITHOUT_DMAGENT= WITHOUT_DICT= WITHOUT_ED_CRYPTO= WITHOUT_EXAMPLES= WITHOUT_FINGER= WITHOUT_FDT= WITHOUT_FLOPPY= WITHOUT_FREEBSD_UPDATE= WITHOUT_GAMES= WITHOUT_GCC= WITHOUT_GCOV= WITHOUT_GDB= WITHOUT_GPIO= WITHOUT_GROFF= WITHOUT_GSSAPI= WITGOUT_HESIOD= WITHOUT_HTML= WITHOUT_HYPERV= WITHOUT_INFO= WITHOUT_IPFILTER= WITHOUT_KERBEROS= WITHOUT_KERNEL_SYMBOLS= WITHOUT_LIB32= WITHOUT_MAN= WITHOUT_MAN_UTILS= WITHOUT_NDIS= WITHOUT_NETCAT= WITHOUT_NLS= WITHOUT_NLS_CATALOGS= WITHOUT_NS_CACHING= WITHOUT_OBJC= WITHOUT_PC_SYSINSTALL= WITHOUT_PF= WITHOUT_PORTSNAP= WITHOUT_PROFILE= WITHOUT_QUOTAS= WITHOUT_RCS= WITHOUT_RESCUE= WITHOUT_ROUTED= WITHOUT_SHAREDOCS= WITHOUT_SVNLITE= WITHOUT_SYSTEM_COMPILER= WITHOUT_TALK= WITHOUT_TESTS= WITHOUT_TEXTPROC= ' Attaching unmodified make.conf used by build process, it's full log and nanobsd environment out of its build directory. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 260993] [regression] [clang] stable/12 buildworld broken: fatal error: 'clang/Basic/DiagnosticCommonKinds.inc' file not found
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260993 --- Comment #1 from Eugene Grosbein --- Created attachment 230772 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=230772&action=edit full build log is over 1000KB, so it comes compressed -- You are receiving this mail because: You are the assignee for the bug.
[Bug 260993] [regression] [clang] stable/12 buildworld broken: fatal error: 'clang/Basic/DiagnosticCommonKinds.inc' file not found
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260993 --- Comment #2 from Eugene Grosbein --- Created attachment 230773 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=230773&action=edit nanobsd make.conf -- You are receiving this mail because: You are the assignee for the bug.
[Bug 260993] [regression] [clang] stable/12 buildworld broken: fatal error: 'clang/Basic/DiagnosticCommonKinds.inc' file not found
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260993 --- Comment #3 from Eugene Grosbein --- Created attachment 230774 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=230774&action=edit nanobsd environment -- You are receiving this mail because: You are the assignee for the bug.
[Bug 260399] freebsd-update: Downloading patches often fails repeatedly
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260399 --- Comment #10 from Colin Percival --- Some quick notes -- sorry, juggling a bunch of stuff right now and won't have time to debug this properly until later: 1. Getting 404 errors in response to /bp/ requests is normal and harmless. Portsnap and FreeBSD Update use "opportunistic patching" -- if there's a patch available it will be used but if not they'll fall back to downloading the complete new file. 2. The redirect to :8080 is a Colin screwup -- I have varnish listening on port 80 and nginx serving up the bits from port 8080. Thanks for pointing this out; I'll dig into the nginx configuration to figure out what I did wrong there. 3. The one thing which concerns me here is the HTTP 400 errors; can you reproduce those with HTTP headers? -- You are receiving this mail because: You are on the CC list for the bug.
[Bug 260993] lib/clang/liblldb: stable/12 fails to build: DiagnosticIDs.h:71:10: fatal error: 'clang/ Basic/DiagnosticCommonKinds.inc' file not found
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260993 Kubilay Kocak changed: What|Removed |Added Assignee|b...@freebsd.org|toolch...@freebsd.org CC||toolch...@freebsd.org Keywords||needs-qa Summary|[regression] [clang]|lib/clang/liblldb: |stable/12 buildworld|stable/12 fails to build: |broken: fatal error:|DiagnosticIDs.h:71:10: |'clang/Basic/DiagnosticComm |fatal error: 'clang/ |onKinds.inc' file not found |Basic/DiagnosticCommonKinds ||.inc' file not found Status|New |Open -- You are receiving this mail because: You are the assignee for the bug.
