Hi, On Fri, Aug 23, 2024 at 10:13:07PM +0200, Salvatore Bonaccorso wrote: > Control: tags -1 + moreinfo upstream > > hi, > > On Thu, Aug 22, 2024 at 08:09:06PM -0700, Forest wrote: > > Package: src:linux > > Version: 6.10.6-1 > > Severity: important > > X-Debbugs-Cc: fores...@nom.one > > > > Dear Maintainer, > > > > After upgrading from kernel 6.9.12 to 6.10.4, flatpak and ostree are now > > writing corrupt gpg signatures when exporting signed packages or signing > > their repository metadata/summary files, when the repository is on a cifs > > mount. Instead of writing signature data, null bytes are written in its > > place. > > > > No error is reported by the application or the kernel when it happens. > > The problem isn't revealed until something tries to use the repository, > > and finds signatures full of null bytes. Of course, this completely > > breaks affected flatpak repositories. > > > > A kernel bisect reveals this: > > 3ee1a1fc39819906f04d6c62c180e760cd3a689d is the first bad commit > > commit 3ee1a1fc39819906f04d6c62c180e760cd3a689d > > Author: David Howells <dhowe...@redhat.com> > > Date: Fri Oct 6 18:29:59 2023 +0100 > > cifs: Cut over to using netfslib > > > > I was unable to determine whether the problem is fixed in kernel > > 6.11.0-rc4, due to even worse cifs problems in that version. > > > > An strace of flatpak (which uses libostree) hints that the problem might > > be triggered by the following sequence of events: > > > > - create a temp file > > - write signature data to the temp file > > - memory map the temp file > > - close the temp file > > - unlink the temp file > > - read the previously written signature data from the memory mapping > > > > My investigation so far can be found in these bug reports: > > https://github.com/flatpak/flatpak/issues/5911 > > https://github.com/ostreedev/ostree/issues/3288 > > > > I am not familiar with those projects' code, so the triggering sequence > > of events is merely a hypothesis for now. > > > > However, I can consistently reproduce the problem by passing a path > > located on a cifs mount (along with a gpg key ID) to this script: > > > > > > #!/bin/sh > > set -e > > > > if [ "$#" -lt 2 ] || [ "$1" = "-h" ] ; then > > echo "usage: $(basename "$0") <repo-dir> <gpg-key-id>" > > exit 2 > > fi > > > > repo=$1 > > keyid=$2 > > src="./foo" > > > > echo "creating ostree repo at $repo" > > ostree init --repo="$repo" > > > > echo "creating test tree at $src" > > mkdir -p "$src" > > echo hi > "$src"/hello > > > > ostree config --repo="$repo" set core.min-free-space-percent 1 > > ostree commit --repo="$repo" --branch=foo --gpg-sign="$keyid" "$src" > > > > if ostree show --repo="$repo" foo; then > > echo --- > > echo success! > > else > > echo --- > > ostree show --repo="$repo" --print-detached-metadata-key=ostree.gpgsigs > > foo > > echo failure! > > echo look for null bytes in the above commit signature > > fi > > Since you identified this as upstream regression including the > breaking commit, can you please report this upstream (and please do > keep us in the loop). > > Since the breaking commit is 3ee1a1fc3981 ("cifs: Cut over to using > netfslib") so you can report it to: > > David Howells <dhowe...@redhat.com> > Steve French <sfre...@samba.org> > Shyam Prasad N <nspmangal...@gmail.com> > Rohith Surabattula <rohiths.m...@gmail.com> > Jeff Layton <jlay...@kernel.org> > linux-c...@vger.kernel.org > ne...@lists.linux.dev > linux-fsde...@vger.kernel.org > linux...@kvack.org > > make sure to include sta...@vger.kernel.org and ideally the regression > mailing list: regressi...@lists.linux.dev
One thing I forgot about the regressions list: https://www.kernel.org/doc/html/latest/admin-guide/reporting-regressions.html can help making sure the regression tracking has as well the correct metadata from start about where the issue is introduced, using the '#regzbot introduced' command. Regards, Salvatore
