Source: mypy
X-Debbugs-Cc: [email protected]
Version: 2.1.0-2
Severity: important
Tags: patch fixed-upstream
Forwarded: https://github.com/mypyc/mypyc/issues/1192
Control: block 1143005 by -1
Dear Maintainer,
mypyc 2.1.0 generates incorrect reference-counting code for some binary
bytes concatenations. In particular, an owned object used as the left
operand of bytes + bytes is not decref'd after the call to
CPyBytes_Concat.
The issue is caused by the bytes + bytes primitive being declared with:
steals=[True, False]
However, CPyBytes_Concat does not steal or decref either operand. It
allocates and returns a new bytes object. Consequently, the reference-count
pass omits the decref for an owned left operand.
A minimal reproducer is:
class A:
def f(self) -> bytes:
return (1).to_bytes(4, "big") + (2).to_bytes(4, "big")
When compiled with mypyc 2.1.0, repeated calls to this method cause stable
and reproducible memory growth. Chained concatenations also leak
intermediate concatenation results, which amplifies the problem.
Upstream fixed this in the following commit:
https://github.com/python/mypy/commit/db331b44ac2b4ef35b138cd5ecba7731c656ca4c
Upstream issue:
https://github.com/mypyc/mypyc/issues/1192
The functional change consists only of removing the incorrect
steals=[True, False] annotation. The upstream commit also adds deterministic
IR regression tests verifying that the owned intermediate results are
decref'd.
The change applies cleanly to Debian's 2.1.0 package, does not depend on
other mypy 2.2 changes, and does not alter the runtime ABI or the signature
or implementation of CPyBytes_Concat.
This bug currently blocks the upload of python3-slhdsa, tracked as Debian
bug #1143005, because that package uses mypyc and contains a
high-frequency bytes-concatenation path affected by this leak.
There are also real-world upstream projects that naturally exercise the
same pattern:
LL-mtproto uses mypyc to compile protocol serialization code containing
owned and chained bytes concatenations.
mysql-mimic uses mypyc to compile packet and type helpers containing
owned bytes concatenations.
Neither project currently appears to be packaged in Debian, so I mention
them only as evidence that the affected pattern occurs in real mypyc
workloads, not as affected Debian reverse dependencies.
Although the fix is included in upstream mypy 2.2, I could not find a
publicly visible, ready-to-upload Debian packaging update for 2.2. I
therefore request that commit
db331b44ac2b4ef35b138cd5ecba7731c656ca4c be backported to the current
2.1.0 package rather than leaving the reproducible leak unfixed while
waiting for the next upstream-version upload.
I am happy to provide a Salsa merge request or a debdiff containing the
upstream patch and regression tests.
Regards,
Xu Colin.
-- System Information:
Debian Release: forky/sid
APT prefers unstable
APT policy:(500, 'unstable')
Architecture: amd64 (x86_64)
Kernel: Linux 6.18.33.2-microsoft-standard-WSL2 (SMP w/32 CPU threads;
PREEMPT)
Locale: LANG=C, LC_CTYPE=C.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: unable to detect
