https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=298878
Bug ID: 298878
Summary: linux(4): epoll_pwait(2)/epoll_pwait2(2) with a
sigmask leave a stale signal mask, deadlocking Bun
apps (Claude Code, opencode)
Product: Base System
Version: 15.1-RELEASE
Hardware: Any
OS: Any
Status: New
Severity: Affects Some People
Priority: ---
Component: kern
Assignee: [email protected]
Reporter: [email protected]
CC: [email protected], [email protected],
[email protected]
Created attachment 275141
--> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=275141&action=edit
Tested patch fixing this issue with proposed commit message
linux_epoll_wait_ts() in sys/compat/linux/linux_event.c saves the
caller's signal mask in a local variable, but still sets TDP_OLDMASK and
schedules the TDA_SIGSUSPEND AST:
if (uset != NULL) {
error = kern_sigprocmask(td, SIG_SETMASK, uset,
&omask, 0);
...
td->td_pflags |= TDP_OLDMASK;
ast_sched(td, TDA_SIGSUSPEND);
}
...
if (uset != NULL)
error = kern_sigprocmask(td, SIG_SETMASK, &omask,
NULL, 0);
On return to user mode, ast_sigsuspend() (or postsig(), if a signal is
being delivered) installs td->td_oldsigmask. This function never writes
it, so it still holds the mask the thread had at its last sigsuspend(2)
(or pselect/ppoll with a mask). After any epoll_pwait(2)/epoll_pwait2(2)
call with a non-NULL sigmask the thread can silently end up with an old,
unrelated signal mask. In addition, the final kern_sigprocmask()
overwrites "error", so EINTR (and any other error) is returned to user
space as 0.
This goes back to 80c7315d17ce ("Restore signal mask in epoll_pwait.",
r314311, 2017), which changed &td->td_oldsigmask to &omask but kept
TDP_OLDMASK. main and stable/15 are identical to 15.1-RELEASE here;
stable/14 has the same code.
Real-world impact: Bun-compiled Linux binaries deadlock
-------------------------------------------------------
Bun's event loop calls epoll_pwait2() with an empty sigmask on every
iteration. JavaScriptCore suspends threads for conservative GC stack
scanning by sending a signal (Bun uses SIGPWR) and waiting on a
semaphore until the target thread's handler acknowledges. That handler
runs with SIGPWR blocked and calls sigsuspend(2), so after the first GC
suspend of the main thread its td_oldsigmask is {SIGPWR}. The next
epoll_pwait2() blocks SIGPWR on the main thread, and the next GC
suspend request never completes. State of a hung process:
- procstat -kk: every thread sleeps in linux_sys_futex -> umtxq_sleep
(plus the idle poll/epoll threads)
- procstat -j: the main thread has exactly one signal blocked, 98
(LINUX_SIGPWREMU = Linux SIGPWR), and it is pending there ("PB")
- user-space stacks: the main thread is in pthread_cond_wait() under
JSC heap code (waiting for the collector); a JSC heap thread is in
sem_wait() right after pthread_kill(), i.e. the thread-suspend
handshake, waiting for the SIGPWR handler to run on the main thread
Seen with Claude Code (Anthropic's CLI, a Bun-compiled Linux binary run
via Linuxulator). Versions 2.1.269 and later (they embed Bun 1.4.3)
freeze shortly after start, after roughly the same delay every time,
whether idle, typing or working; kill -9 is the only way out. Versions
up to 2.1.268 (Bun 1.4.1) are not affected in practice: in a GC stress
test under Bun 1.4.1 the collector never signalled the main thread (no
tgkill or sigsuspend at all), so td_oldsigmask never gets poisoned.
misc/claude-code is currently at 2.1.268, so port users will run into
this with the next update. Users who install the newest Linux binary
from npm directly are affected now. We do that because the port lags
behind upstream, with a script along these lines:
VERSION="${1:-latest}"
npm install -g "@anthropic-ai/claude-code-linux-x64@${VERSION}" --force
# /compat/linux/tmp must be 1777 for non-root users
chmod 1777 /compat/linux/tmp /compat/linux/var/tmp
cat > /usr/local/bin/claude << EOT
#!/bin/sh
exec /usr/local/lib/node_modules/@anthropic-ai/claude-code-linux-x64/claude
"\$@"
EOT
This is very likely also the cause of bug 296441 (opencode deadlocks
with all threads in umtxq_sleep, avoided with BUN_JSC_useJIT=0) and of
the Claude Code hangs reported in bug 298047 comment 7.
Test program
------------
Attached epoll_sigmask_test.c is freestanding (raw syscalls, no libc),
so it builds on FreeBSD with base clang and ld.lld; build instructions
are at the top of the file. It supports amd64 and i386 (the i386 build
goes through linux.ko, which compiles the same linux_event.c).
15.1-RELEASE-p3, stock code (i386 build, stock linux.ko):
PASS after sigsuspend() and sigprocmask(SIG_SETMASK, {}): mask
0x0000000000000000
FAIL after epoll_pwait(timeout=0, sigmask={}): mask 0x0000000020000200,
expected 0x0000000000000000
FAIL after epoll_pwait2(timeout=0, sigmask={}): mask 0x0000000020000200,
expected 0x0000000000000000
FAIL epoll_pwait2(timeout=NULL, sigmask={}) interrupted by SIGALRM: returned
0, expected -4 (EINTR); SIGALRM handler calls: 1
FAIL after the interrupted epoll_pwait2(), mask {SIGUSR1}: mask
0x0000000020000200, expected 0x0000000000000200
FAILED
15.1-RELEASE-p3 with the attached patch (amd64 build, patched linux64.ko):
PASS after sigsuspend() and sigprocmask(SIG_SETMASK, {}): mask
0x0000000000000000
PASS after epoll_pwait(timeout=0, sigmask={}): mask 0x0000000000000000
PASS after epoll_pwait2(timeout=0, sigmask={}): mask 0x0000000000000000
PASS epoll_pwait2(timeout=NULL, sigmask={}) interrupted by SIGALRM: returned
-4
PASS after the interrupted epoll_pwait2(), mask {SIGUSR1}: mask
0x0000000000000200
OK
(An earlier version of the test showed the same stale mask,
0x20000200, with the amd64 build and the stock linux64.ko.)
Fix
---
The attached patch saves the old mask in td_oldsigmask and lets the AST
restore it, exactly as kern_pselect() and kern_poll_kfds() have done
since ccb973da1f1b ("kern: restore signal mask before ast() for
pselect/ppoll"): TDA_SIGSUSPEND if the wait was interrupted, so the
signal is delivered with the temporary mask and the handler frame saves
the correct mask, and TDA_PSELECT otherwise. This matches Linux, which
restores the saved mask unless epoll_pwait returns -EINTR. It also stops
EINTR from being turned into 0.
Tested on 15.1-RELEASE-p3 amd64: the test program passes, and Claude
Code 2.1.283 runs normally with the patched linux64.ko (with the stock
module it froze every time). The patch applies to main and stable/15.
stable/14 has no TDA_PSELECT; the equivalent change there would mirror
its kern_poll_kfds() (save into td_oldsigmask; on EINTR set TDP_OLDMASK
and schedule TDA_SIGSUSPEND, otherwise restore td_oldsigmask directly).
The analysis, patch and test program were prepared with the help of an
AI assistant (Claude by Anthropic); I reviewed them and tested them on
my system.
See Also:
---------
296441 opencode deadlocks at startup
298047 claude-code hangs, comment 7
Blocks
------
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=247219
--
You are receiving this mail because:
You are the assignee for the bug.
