On 24/11/17 10:08PM, David Kalnischkies wrote: > Control: retitle -1 apt: FTBFS on Arch Linux with libapt test failures > Control: severity -1 wishlist > > Hi Christian Heusel,
Hello David,
> as said on IRC, we are happy to receive and help with bug reports
> here in the BTS, but I had to dial back on the severity quite a bit as
> this is not applying to Debian and hence not release-critical to Debian
> (apt is not in any danger of being removed from testing or a future
> release for obvious reasons, but it could still confuse unsuspecting
> users and automatic processes working with and against bugreports).
>
> No worries, severity has hardly any practical meaning for us so your
> bug report isn't handled and dealt with slower or faster regardless
> of it being tagged serious or wishlist.
>
sure, that is not an issue for me! I think I just selected this severity
as it's descriptions sounded the most fitting when using reportbug.
(Fun fact, in the Arch bugtracker the severity field means almost
nothing aswell :D)
>
> But enough of the pretext, lets start with the on topic:
>
> Am Sun, Nov 17, 2024 at 08:15:21PM +0100, schrieb Christian Heusel:
> > * What was the outcome of this action?
> > Building the package yields test failures for 2.9.11
>
> Did you try with earlier versions or is 2.9.11 the first you try?
Yes I tried earlier versions and they all worked fine. The last version
that worked was the previous release 2.9.10 but I package it for [a
while now] (since 2.9.2).
In the meantime we had another release (2.9.12) which fixed the
URITest.AutoProxyTest issue, so let's just ignore that for now. The
crash in the other tests persists though!
I also took the time and bisected between 2.9.10 and 2.9.11 which commit
introduced the failure and I landed on the following:
commit b430a53e45b6d676142af7da78564de3bd97ef5f (HEAD)
Author: наб <nabijaczlew...@nabijaczleweli.xyz>
Date: Tue Nov 12 19:50:56 2024 +0100
methods/http.cc: APT::StringView -> std::string_view
>
> The tests in question themselves haven't changed in months.
> The code they test might have, but to figure that out it would be
> very beneficial to know if that was "always" the case or a recent
> change.
>
> > * What outcome did you expect instead?
> > The application builds without test failures
>
> Can you perhaps provide some instructions so that we with Debian as
> a starting base could reach a point where we can reproduce this failure
> which preferably doesn't involve "install Arch on a spare machine" or
> (at least for me, others might have other opinions) the word "Docker".
Docker or distrobox is of course the easy route, getting a working Arch
Linux chroot is not too hard aswell and is sufficient to repro this bug
(although we need to do a ton of trickery that wouldn't be neccessary on
a regular Arch system since things aren't set up):
# apt install arch-install-scripts pacman-package-manager archlinux-keyring
# curl
https://gitlab.archlinux.org/archlinux/packaging/packages/pacman/-/raw/main/pacman.conf\?ref_type\=heads
> /etc/pacman.conf
# mkdir -pv /etc/pacman.d/
# echo "Server = https://geo.mirror.pkgbuild.com/\$repo/os/\$arch"; >
/etc/pacman.d/mirrorlist
# pacman-key --init && pacman-key --populate
# cd /path/to/chroots
# mkdir -pv archlinux-chroot
# pacstrap -KP archlinux-chroot base base-devel git devtools nvchecker less
gdb
# arch-chroot archlinux-chroot
># useradd -m -G wheel username
># echo "%wheel ALL=(ALL:ALL) NOPASSWD: ALL" >> /etc/sudoers
># sed -i "s/CheckSpace/#CheckSpace/" /etc/pacman.conf
># sudo -u username -i
># pkgctl repo clone --protocol=https apt
># cd apt
># pkgctl version upgrade
># gpg --import < keys/pgp/*
># export MAKEFLAGS="-j$(nproc)"
># makepkg --syncdeps
># cat /home/username/apt/src/build/Testing/Temporary/LastTest.log
># gdb src/build/test/libapt/libapt_test
With "native" Arch tooling this would have been just a few commands (we
do all our builds in clean chroots), but sadly our devtools arent't
available in debian.
> As in, a Debian chroot can be relatively easy bootstrapped even on
> foreign Linux distros with (c)debootstrap. I suppose something like this
> exists for Arch, too? And how can I check out your work so far and build
> that to see the error(s) myself?
>
>
> > [ FAILED ] URITest.AutoProxyTest (0 ms)
>
> That test has 4 checks and they all fail… they don't do much through,
> they just call a script, so that screams setup problem like that the
> script isn't executable or you are building on tmpfs with noexec or
> something like that.
As I have said previously, let's just ignore this then!
> > [ RUN ] TreeParserTest.ParseInvalid
> > /usr/include/c++/14.2.1/string_view:256: constexpr const
> > std::basic_string_view<_CharT, _Traits>::value_type&
> > std::basic_string_view<_CharT, _Traits>::operator[](size_type) const [with
> > _CharT = char; _Traits = std::char_traits<char>; const_reference = const
> > char&; size_type = long unsigned int]: Assertion '__pos < this->_M_len'
> > failed.
> > Aborted (core dumped)
>
> While the libapt code this is crashing in should be rewritten to not
> crash… (which I did once, but it was kinda ugly, so we discarded it, but
> now that a bunch of std::string_view recently landed perhaps…) anyway,
> this is a test that is trying to trigger an exception. Do you perhaps
> build with exceptions disabled?
No, our buildflags are the following[1]:
CFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions \
-Wp,-D_FORTIFY_SOURCE=3 -Wformat -Werror=format-security \
-fstack-clash-protection -fcf-protection \
-fno-omit-frame-pointer -mno-omit-leaf-frame-pointer"
CXXFLAGS="$CFLAGS -Wp,-D_GLIBCXX_ASSERTIONS"
But I guess the added -D_GLIBCXX_ASSERTIONS is relevant to trigger the
bug..? So somehow (just judging from the backtrace) there seems to be a
bounds issue in APT::Internal::PatternTreeParser::parseShortPattern with
the check that might now be caught since the switch to std::string_view..?
> Or was GoogleTest perhaps built without them? Which type of GoogleTest
> is that anyhow, the pre-built or the one that is build with apt.
It is using the system one:
$ ldd src/build/test/libapt/libapt_test | grep gtest
libgtest.so.1.15.2 => /usr/lib/libgtest.so.1.15.2 (0x00007f684c157000)
Also I think the test suite is not responsible given that one version of
the upstream code works and another one does not.
> Our buildsystem supports both in theory, but
> GoogleTest upstream tends to advice against building the goods and the
> tests differently – in Debian that doesn't usually happen, but in a more
> source-orientated (AUR stuff is built on user-systems, right?) this
> might be more of a factor, maybe?
>
> At least I assume that both "failures" are a red herring and the actual
> problem causing them is deeper and more related to how Arch is setup.
> Hence my interest in chroot instructions… if you can't reproduce it
> there, we have a first good lead.
I hope you can work with the above instructions, they are roughly based
on [this][2] wiki article.
> Best regards
>
> David Kalnischkies
Thanks for being awesome,
Chris
[0]:
https://gitlab.archlinux.org/archlinux/packaging/packages/apt/-/commits/main/?ref_type=HEADS
[1]:
https://gitlab.archlinux.org/archlinux/devtools/-/blob/master/config/makepkg/x86_64.conf?ref_type=heads#L43-57
[2]:
https://wiki.archlinux.org/title/Install_Arch_Linux_from_existing_Linux#Using_a_chroot_environment
signature.asc
Description: PGP signature
