I see this in your trace: > /builddir/build/BUILD/trafficserver-10.0.2-build/trafficserver-10.0.2/lib/swoc/include/swoc/Errata.h:1002
Looking at: https://github.com/apache/trafficserver-libswoc/blob/SolidWallOfCode-patch-1/code/include/swoc/Errata.h The file is under 1000 lines. Perhaps you are seeing a bug that was fixed, or are using an incompatible library? On Thursday, January 9, 2025 at 01:00:09 PM EST, Jered Floyd <je...@convivian.com> wrote: Following up to myself, removing -Wp,-D_GLIBCXX_ASSERTIONS from my CFLAGS resolves this so I believe this is a legitimate memory safety issue in Errata (or the test). I've opened an issue on GitHub to track: https://github.com/apache/trafficserver/issues/11948 --Jered ----- On Jan 8, 2025, at 3:31 PM, Jered Floyd je...@convivian.com wrote: > Happy new year! > > I finally added a debugger into my mock build environment to debug the > test_libswoc assertion failures I see trying to build trafficserver 10 for > Fedora/RHEL production. > > Here is the backtrace -- any help is appreciated since I'm not very clear on > what this library is supposed to be doing and docs.solidwallofcode.com no > longer exists. From the backtrace this looks like possibly bwformat() tries > to > deference an uninitialized note.severity (via an accessor function > Annotation.severity()) without first checking validity, which triggers the > enabled assertion. > > This all looks majorly refactored from 9.2.x so I don't have a point of > comparison for these modules. > > Thanks, > --Jered > > /usr/include/c++/14/optional:482: constexpr const _Tp& > std::_Optional_base_impl<_Tp, _Dp>::_M_get() const [with _Tp = > swoc::_1_5_12::Errata::Severity; _Dp = > std::_Optional_base<swoc::_1_5_12::Errata::Severity, true, true>]: Assertion > 'this->_M_is_engaged()' failed. > > Program received signal SIGABRT, Aborted. > 0x00007ffff7a7fe64 in __pthread_kill_implementation () from /lib64/libc.so.6 > Missing rpms, try: dnf --enablerepo='*debug*' install > libstdc++-debuginfo-14.2.1-6.fc42.x86_64 > glibc-debuginfo-2.40.9000-26.fc42.x86_64 libgcc-debuginfo-14.2.1-6.fc42.x86_64 > (gdb) bt > #0 0x00007ffff7a7fe64 in __pthread_kill_implementation () from > /lib64/libc.so.6 > #1 0x00007ffff7a2649e in raise () from /lib64/libc.so.6 > #2 0x00007ffff7a0e397 in abort () from /lib64/libc.so.6 > #3 0x00007ffff7c3e2f0 in std::__glibcxx_assert_fail(char const*, int, char > const*, char const*) () from /lib64/libstdc++.so.6 > #4 0x00007ffff7f93eb7 in > std::_Optional_base_impl<swoc::_1_5_12::Errata::Severity, > std::_Optional_base<swoc::_1_5_12::Errata::Severity, true, true> >::_M_get > (this=0x55555580c0ca) > at /usr/include/c++/14/optional:480 > #5 std::_Optional_base_impl<swoc::_1_5_12::Errata::Severity, > std::_Optional_base<swoc::_1_5_12::Errata::Severity, true, true> >::_M_get > (this=0x55555580c0ca) > at /usr/include/c++/14/optional:480 > #6 std::optional<swoc::_1_5_12::Errata::Severity>::operator*() const & > (this=0x55555580c0ca) at /usr/include/c++/14/optional:968 > #7 swoc::_1_5_12::Errata::Annotation::severity (this=0x55555580c0b8) at > /builddir/build/BUILD/trafficserver-10.0.2-build/trafficserver-10.0.2/lib/swoc/include/swoc/Errata.h:1002 > #8 swoc::_1_5_12::bwformat (bw=..., errata=...) at > /builddir/build/BUILD/trafficserver-10.0.2-build/trafficserver-10.0.2/lib/swoc/src/Errata.cc:155 > #9 0x0000555555627a62 in > swoc::_1_5_12::BufferWriter::print_nfv<swoc::_1_5_12::bwf::NameBinding const&, > swoc::_1_5_12::bwf::Format::TextViewExtractor> (this=0x7fffffffd4e0, > names=..., > ex=..., args=...) at > >/builddir/build/BUILD/trafficserver-10.0.2-build/trafficserver-10.0.2/lib/swoc/include/swoc/bwf_base.h:898 > #10 0x0000555555637593 in > swoc::_1_5_12::BufferWriter::print_v<swoc::_1_5_12::Errata const&> > (fmt=<synthetic pointer>..., this=0x7fffffffd4e0, args=std::tuple containing = > {...}) > at > >/builddir/build/BUILD/trafficserver-10.0.2-build/trafficserver-10.0.2/lib/swoc/include/swoc/bwf_base.h:612 > #11 swoc::_1_5_12::FixedBufferWriter::print_v<swoc::_1_5_12::Errata const&> > (this=0x7fffffffd4e0, fmt=..., args=std::tuple containing = {...}) > at > >/builddir/build/BUILD/trafficserver-10.0.2-build/trafficserver-10.0.2/lib/swoc/include/swoc/bwf_base.h:1221 > #12 swoc::_1_5_12::bwprint_v<swoc::_1_5_12::Errata > const&>(std::__cxx11::basic_string<char, std::char_traits<char>, > std::allocator<char> >&, swoc::_1_5_12::TextView, > std::tuple<swoc::_1_5_12::Errata const&> const&)::{lambda()#1}::operator()() > const (__closure=<synthetic pointer>) > at > >/builddir/build/BUILD/trafficserver-10.0.2-build/trafficserver-10.0.2/lib/swoc/include/swoc/bwf_base.h:1152 > #13 swoc::_1_5_12::bwprint_v<swoc::_1_5_12::Errata const&> (s="", fmt=..., > args=std::tuple containing = {...}) > at > >/builddir/build/BUILD/trafficserver-10.0.2-build/trafficserver-10.0.2/lib/swoc/include/swoc/bwf_base.h:1153 > #14 swoc::_1_5_12::bwprint<swoc::_1_5_12::Errata const&> (s="", fmt=...) at > /builddir/build/BUILD/trafficserver-10.0.2-build/trafficserver-10.0.2/lib/swoc/include/swoc/bwf_base.h:1180 > #15 operator() (__closure=<optimized out>, errata=...) at > /builddir/build/BUILD/trafficserver-10.0.2-build/trafficserver-10.0.2/lib/swoc/unit_tests/test_Errata.cc:71 > #16 std::__invoke_impl<void, test_Errata_init()::<lambda(const > swoc::_1_5_12::Errata&)>&, const swoc::_1_5_12::Errata&> (__f=...) at > /usr/include/c++/14/bits/invoke.h:61 > #17 std::__invoke_r<void, test_Errata_init()::<lambda(const > swoc::_1_5_12::Errata&)>&, const swoc::_1_5_12::Errata&> (__fn=...) at > /usr/include/c++/14/bits/invoke.h:111 > #18 std::_Function_handler<void(const swoc::_1_5_12::Errata&), > test_Errata_init()::<lambda(const swoc::_1_5_12::Errata&)> >::_M_invoke(const > std::_Any_data &, const swoc::_1_5_12::Errata &) > (__functor=..., __args#0=...) at >/usr/include/c++/14/bits/std_function.h:290 > #19 0x00007ffff7f92fc1 in std::function<void(swoc::_1_5_12::Errata > const&)>::operator() (this=<optimized out>, __args#0=...) at > /usr/include/c++/14/bits/std_function.h:591 > #20 swoc::_1_5_12::Errata::SinkWrapper::operator() (this=<optimized out>, > e=...) > at > /builddir/build/BUILD/trafficserver-10.0.2-build/trafficserver-10.0.2/lib/swoc/include/swoc/Errata.h:1318 > #21 swoc::_1_5_12::Errata::sink (this=0x7fffffffd5d0) at > /builddir/build/BUILD/trafficserver-10.0.2-build/trafficserver-10.0.2/lib/swoc/src/Errata.cc:57 > #22 0x00007ffff7f9305d in swoc::_1_5_12::Errata::~Errata > (this=this@entry=0x7fffffffd5d0, __in_chrg=<optimized out>) > at > >/builddir/build/BUILD/trafficserver-10.0.2-build/trafficserver-10.0.2/lib/swoc/src/Errata.cc:50 > #23 0x000055555562dfdc in C_A_T_C_H_T_E_S_T_0 () at > /builddir/build/BUILD/trafficserver-10.0.2-build/trafficserver-10.0.2/lib/swoc/unit_tests/test_Errata.cc:114 > #24 0x000055555559c891 in Catch::TestInvokerAsFunction::invoke > (this=<optimized > out>) at > /builddir/build/BUILD/trafficserver-10.0.2-build/trafficserver-10.0.2/lib/catch2/catch.hpp:14328 > #25 Catch::TestCase::invoke (this=<optimized out>) at > /builddir/build/BUILD/trafficserver-10.0.2-build/trafficserver-10.0.2/lib/catch2/catch.hpp:14167 > #26 Catch::RunContext::invokeActiveTestCase (this=0x7fffffffdc90) at > /builddir/build/BUILD/trafficserver-10.0.2-build/trafficserver-10.0.2/lib/catch2/catch.hpp:13027 > #27 0x00005555555bb257 in Catch::RunContext::runCurrentTest > (this=this@entry=0x7fffffffdc90, redirectedCout="", redirectedCerr="") > at > >/builddir/build/BUILD/trafficserver-10.0.2-build/trafficserver-10.0.2/lib/catch2/catch.hpp:13000 > #28 0x00005555555bb738 in Catch::RunContext::runTest > (this=this@entry=0x7fffffffdc90, testCase=...) > at > >/builddir/build/BUILD/trafficserver-10.0.2-build/trafficserver-10.0.2/lib/catch2/catch.hpp:12761 > #29 0x00005555555cc650 in Catch::(anonymous namespace)::TestGroup::execute > (this=0x7fffffffdc80) > at > >/builddir/build/BUILD/trafficserver-10.0.2-build/trafficserver-10.0.2/lib/catch2/catch.hpp:13354 > #30 Catch::Session::runInternal (this=this@entry=0x7fffffffdf30) at > /builddir/build/BUILD/trafficserver-10.0.2-build/trafficserver-10.0.2/lib/catch2/catch.hpp:13560 > #31 0x00005555555ccc5f in Catch::Session::run (this=this@entry=0x7fffffffdf30) > at > /builddir/build/BUILD/trafficserver-10.0.2-build/trafficserver-10.0.2/lib/catch2/catch.hpp:13516 > #32 0x000055555558f6a2 in Catch::Session::run<char> (this=0x7fffffffdf30, > argc=1, argv=0x7fffffffe218) > at > >/builddir/build/BUILD/trafficserver-10.0.2-build/trafficserver-10.0.2/lib/catch2/catch.hpp:13238 > #33 Catch::Session::run<char> (this=0x7fffffffdf30, argc=1, > argv=0x7fffffffe218) > at > /builddir/build/BUILD/trafficserver-10.0.2-build/trafficserver-10.0.2/lib/catch2/catch.hpp:13233 > #34 main (argc=1, argv=0x7fffffffe218) at > /builddir/build/BUILD/trafficserver-10.0.2-build/trafficserver-10.0.2/lib/swoc/unit_tests/unit_test_main.cc:36 > > > > > ----- On Dec 13, 2024, at 5:01 PM, Jered Floyd je...@convivian.com wrote: > >> Hi Chris, >> >> Sorry for the delay in getting back to you. >> >> I've disabled to the verify_* tests for the time being; the way that rpmbuild >> works is that build/install happens to a build-user-owned DESTDIR so files >> can >> be gathered for packaging. Simply installing the previous version doesn't >> help >> because the build user doesn't have access to /etc/trafficserver (due to >> default permissions). The koji build system sets up BuildRequires based on >> the >> spec file, but I don't think there's a way I can ask it to do something as >> root >> (like changing the directory permissions). I'll investigate further. >> >> I'll return to the linking error later, but right now the problem I've run >> into >> is an assertion failure in libswoc; see the end of the build log here: >> https://kojipkgs.fedoraproject.org//work/tasks/48/126810048/build.log >> >> 5/63 Test #1: test_libswoc ...........................Subprocess >> aborted***Exception: 0.15 sec >> /usr/include/c++/14/optional:482: constexpr const _Tp& >> std::_Optional_base_impl<_Tp, _Dp>::_M_get() const [with _Tp = >> swoc::_1_5_12::Errata::Severity; _Dp = >> std::_Optional_base<swoc::_1_5_12::Errata::Severity, true, true>]: Assertion >> 'this->_M_is_engaged()' failed. >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> test_libswoc is a Catch v2.13.8 host application. >> Run with -? for options >> ------------------------------------------------------------------------------- >> Errata copy >> ------------------------------------------------------------------------------- >> /builddir/build/BUILD/trafficserver-10.0.2-build/trafficserver-10.0.2/lib/swoc/unit_tests/test_Errata.cc:84 >> ............................................................................... >> /builddir/build/BUILD/trafficserver-10.0.2-build/trafficserver-10.0.2/lib/swoc/unit_tests/test_Errata.cc:84: >> FAILED: >> {Unknown expression after the reported line} >> due to a fatal error condition: >> SIGABRT - Abort (abnormal termination) signal >> >> I'll do a local build in a mock environment and see if I can attach a gdb to >> this process, but any pointers on what might be going on? I haven't kept up >> with C++17... >> >> Thanks, >> --Jered >> >> ----- On Nov 17, 2024, at 11:25 AM, Chris McFarlen <ch...@mcfarlen.us> wrote: >> >>> Hi Jered, >> >>> You are correct there is a chicken-egg issue with the tests that verify >>> plugin >>> linkage (verify_.* tests). They use traffic_server's verify plugin command >>> and >>> the traffic_server initialization code wants the install directories to >>> exist. >>> I plan on fixing this before the next release. If installing before testing >>> isn't an option for your build, a workaround would be to comment or remove >>> the >>> bodies of the verify_remap_plugin and verify_global_plugin in >>> cmake/add_atsplugin.cmake. Just leave the function definitions in place. >> >>> I might be able to help with the linking issues as well if you can provide >>> some >>> steps to reproduce. >> >>> Thanks! >>> Chris >> >>> Sent with Proton Mail secure email. >> >>> On Saturday, November 16th, 2024 at 5:30 PM, Jered Floyd >>> <je...@convivian.com> >>> wrote: >> >>>> [This is specifically regarding trafficserver 10.0.2.] >> >>>> I've managed to get a successful build by disabling link time optimization >>>> features (default Fedora build flags "-flto=auto -ffat-lto-objects") -- >>>> that >>>> works in 9.2.6 so there's something else at play, but I'll look into that >>>> later. >> >>>> Now I'm running into ctest problems, which happen even if I build in a >>>> vanilla >>>> environment following the README.md build instructions. In particular, >>>> ctest >>>> doesn't use config files nor run or logdirs from the build environment. It >>>> appears that running "cd build; ctest" will not pass unless there is an >>>> existing install of traffic server installed. Have I missed something here? >> >>>> Thanks, >>>> --Jered >> >>>> ----- On Nov 14, 2024, at 11:56 AM, Jered Floyd je...@convivian.com wrote: >> >>>> > Hello! >> >>>> > I am trying to get ATS 10.x ready for Fedora rawhide/f42 and EPEL 10, >>>> > and am >>>> > running into some problems with CMake execution. I've banged my head >>>> > against it >>>> > for a few days so now I'm reaching out in hopes of finding an ATS CMake >>>> > expert. >> >>>> > What seems to be happening is that the build nearly completes, and then >>>> > linking >>>> > some of the last tests and application components fails due to missing >>>> > symbols. >>>> > This seems to be at least partly due to the library ordering, as a quick >>>> > hack >>>> > of repeating the library list (manually) gets a bit further through the >>>> > build >>>> > process. I'm not very familiar with CMake, so I'm not sure where to go >>>> > from >>>> > here. >> >>>> > You can find an entire build log here, where I've disabled build >>>> > parallelism for >>>> > clarity. Everything blows up with linker errors about 50% through the >>>> > file; I >>>> > recommend searching on "[100%] Linking CXX executable >>>> > test_PluginFactory" as >>>> > the subsequent link 2 lines later fails: >>>> > [ >>>> > https://kojipkgs.fedoraproject.org//work/tasks/5517/125855517/build.log | >>>> > https://kojipkgs.fedoraproject.org//work/tasks/5517/125855517/build.log ] >> >>>> > Any debugging help would be greatly appreciated! >> >>>> > Regards, > > > > > --Jered