https://github.com/ldionne updated https://github.com/llvm/llvm-project/pull/69498
>From 6f89b118ed56ad7a3af1996e19ccd30cc893c51e Mon Sep 17 00:00:00 2001 From: Louis Dionne <ldionn...@gmail.com> Date: Wed, 14 Jun 2023 17:49:22 -0700 Subject: [PATCH 1/8] [libc++] Fix the behavior of throwing `operator new` under -fno-exceptions In D144319, Clang tried to land a change that would cause some functions that are not supposed to return nullptr to optimize better. As reported in https://reviews.llvm.org/D144319#4203982, libc++ started seeing failures in its CI shortly after this change was landed. As explained in D146379, the reason for these failures is that libc++'s throwing `operator new` can in fact return nullptr when compiled with exceptions disabled. However, this contradicts the Standard, which clearly says that the throwing version of `operator new(size_t)` should never return nullptr. This is actually a long standing issue. I've previously seen a case where LTO would optimize incorrectly based on the assumption that `operator new` doesn't return nullptr, an assumption that was violated in that case because libc++.dylib was compiled with -fno-exceptions. Unfortunately, fixing this is kind of tricky. The Standard has a few requirements for the allocation functions, some of which are impossible to satisfy under -fno-exceptions: 1. `operator new(size_t)` must never return nullptr 2. `operator new(size_t, nothrow_t)` must call the throwing version and return nullptr on failure to allocate 3. We can't throw exceptions when compiled with -fno-exceptions In the case where exceptions are enabled, things work nicely. `new(size_t)` throws and `new(size_t, nothrow_t)` uses a try-catch to return nullptr. However, when compiling the library with -fno-exceptions, we can't throw an exception from `new(size_t)`, and we can't catch anything from `new(size_t, nothrow_t)`. The only thing we can do from `new(size_t)` is actually abort the program, which does not make it possible for `new(size_t, nothrow_t)` to catch something and return nullptr. This patch makes the following changes: 1. When compiled with -fno-exceptions, the throwing version of `operator new` will now abort on failure instead of returning nullptr on failure. This resolves the issue that the compiler could mis-compile based on the assumption that nullptr is never returned. This constitutes an API and ABI breaking change for folks compiling the library with -fno-exceptions (which is not the general public, who merely uses libc++ headers but use a shared library that has already been compiled). This should mostly impact vendors and other folks who compile libc++.dylib themselves. 2. When the library is compiled with -fexceptions, the nothrow version of `operator new` has no change. When the library is compiled with -fno-exceptions, the nothrow version of `operator new` will now check whether the throwing version of `operator new` has been overridden. If it has not been overridden, then it will use an implementation equivalent to that of the throwing `operator new`, except it will return nullptr on failure to allocate (instead of terminating). However, if the throwing `operator new` has been overridden, it is now an error NOT to also override the nothrow `operator new`. Indeed, there is no way for us to implement a valid nothrow `operator new` without knowing the exact implementation of the throwing version. rdar://103958777 Differential Revision: https://reviews.llvm.org/D150610 --- libcxx/docs/ReleaseNotes/18.rst | 23 +++ libcxx/include/CMakeLists.txt | 1 + libcxx/include/__overridable_function | 119 +++++++++++++ libcxx/src/new.cpp | 99 ++++++++--- ...new_not_overridden_fno_exceptions.pass.cpp | 59 ++++++ .../new_dont_return_nullptr.pass.cpp | 37 ++++ libcxx/test/support/check_assertion.h | 6 + libcxx/utils/generate_iwyu_mapping.py | 2 + libcxxabi/src/stdlib_new_delete.cpp | 168 +++++++++++------- 9 files changed, 426 insertions(+), 88 deletions(-) create mode 100644 libcxx/include/__overridable_function create mode 100644 libcxx/test/libcxx/language.support/support.dynamic/assert.nothrow_new_not_overridden_fno_exceptions.pass.cpp create mode 100644 libcxx/test/libcxx/language.support/support.dynamic/new_dont_return_nullptr.pass.cpp diff --git a/libcxx/docs/ReleaseNotes/18.rst b/libcxx/docs/ReleaseNotes/18.rst index 9e509db6359c4aa..405f1e172893bc1 100644 --- a/libcxx/docs/ReleaseNotes/18.rst +++ b/libcxx/docs/ReleaseNotes/18.rst @@ -142,6 +142,29 @@ LLVM 20 ABI Affecting Changes --------------------- +- When the shared/static library is built with ``-fno-exceptions``, the behavior of ``operator new`` was changed + to make it standards-conforming. In LLVM 17 and before, the throwing versions of ``operator new`` would return + ``nullptr`` upon failure to allocate, when the shared/static library was built with exceptions disabled. This + was non-conforming, since the throwing versions of ``operator new`` are never expected to return ``nullptr``, and + this non-conformance could actually lead to miscompiles in subtle cases. + + Starting in LLVM 18, the throwing versions of ``operator new`` will abort the program when they fail to allocate + if the shared/static library has been built with ``-fno-exceptions``. This is consistent with the behavior of all + other potentially-throwing functions in the library, which abort the program instead of throwing when ``-fno-exceptions`` + is used. + + Furthermore, when the shared/static library is built with ``-fno-exceptions``, users who override the throwing + version of ``operator new`` will now need to also override the ``std::nothrow_t`` version of ``operator new`` if + they want to use it. Indeed, this is because there is no way to implement a conforming ``operator new(nothrow)`` + from a conforming potentially-throwing ``operator new`` when compiled with ``-fno-exceptions``. In that case, using + ``operator new(nothrow)`` without overriding it explicitly but after overriding the throwing ``operator new`` will + result in an error. + + Note that this change only impacts vendors/users that build the shared/static library themselves and pass + ``-DLIBCXX_ENABLE_EXCEPTIONS=OFF``, which is not the default configuration. If you are using the default + configuration of the library, the libc++ shared/static library will be built with exceptions enabled, and + there is no change between LLVM 17 and LLVM 18, even for users who build their own code using ``-fno-exceptions``. + - The symbol of a non-visible function part of ``std::system_error`` was removed. This is not a breaking change as the private function ``__init`` was never referenced internally outside of the dylib. diff --git a/libcxx/include/CMakeLists.txt b/libcxx/include/CMakeLists.txt index 7d0defa26b0f73c..7d5a07d995e2675 100644 --- a/libcxx/include/CMakeLists.txt +++ b/libcxx/include/CMakeLists.txt @@ -570,6 +570,7 @@ set(files __numeric/transform_exclusive_scan.h __numeric/transform_inclusive_scan.h __numeric/transform_reduce.h + __overridable_function __random/bernoulli_distribution.h __random/binomial_distribution.h __random/cauchy_distribution.h diff --git a/libcxx/include/__overridable_function b/libcxx/include/__overridable_function new file mode 100644 index 000000000000000..81dd81bdc0ae8a2 --- /dev/null +++ b/libcxx/include/__overridable_function @@ -0,0 +1,119 @@ +// -*- C++ -*- +//===----------------------------------------------------------------------===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#ifndef _LIBCPP___OVERRIDABLE_FUNCTION +#define _LIBCPP___OVERRIDABLE_FUNCTION + +#include <__config> +#include <cstdint> + +#if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER) +# pragma GCC system_header +#endif + +// +// This file provides the std::__is_function_overridden utility, which allows checking +// whether an overridable function (typically a weak symbol) like `operator new` +// has been overridden by a user or not. +// +// This is a low-level utility which does not work on all platforms, since it needs +// to make assumptions about the object file format in use. Furthermore, it requires +// the "base definition" of the function (the one we want to check whether it has been +// overridden) to be annotated with the _LIBCPP_MAKE_OVERRIDABLE_FUNCTION_DETECTABLE macro. +// +// This currently works with Mach-O files (used on Darwin) and with ELF files (used on Linux +// and others). On platforms where we know how to implement this detection, the macro +// _LIBCPP_CAN_DETECT_OVERRIDDEN_FUNCTION is defined to 1, and it is defined to 0 on +// other platforms. The _LIBCPP_MAKE_OVERRIDABLE_FUNCTION_DETECTABLE macro is defined to +// nothing on unsupported platforms so that it can be used to decorate functions regardless +// of whether detection is actually supported. +// +// How does this work? +// ------------------- +// +// Let's say we want to check whether a weak function `f` has been overridden by the user. +// The general mechanism works by placing `f`'s definition (in the libc++ built library) +// inside a special section, which we do using the `__section__` attribute via the +// _LIBCPP_MAKE_OVERRIDABLE_FUNCTION_DETECTABLE macro. +// +// Then, when comes the time to check whether the function has been overridden, we take +// the address of the function and we check whether it falls inside the special function +// we created. This can be done by finding pointers to the start and the end of the section +// (which is done differently for ELF and Mach-O), and then checking whether `f` falls +// within those bounds. If it falls within those bounds, then `f` is still inside the +// special section and so it is the version we defined in the libc++ built library, i.e. +// it was not overridden. Otherwise, it was overridden by the user because it falls +// outside of the section. +// +// Important note +// -------------- +// +// This mechanism should never be used outside of the libc++ built library. In particular, +// attempting to use this within the libc++ headers will not work at all because we don't +// want to be defining special sections inside user's executables which use our headers. +// This is provided inside libc++'s include tree solely to make it easier to share with +// libc++abi, which needs the same mechanism. +// + +#if defined(_LIBCPP_OBJECT_FORMAT_MACHO) + +# define _LIBCPP_CAN_DETECT_OVERRIDDEN_FUNCTION 1 +# define _LIBCPP_MAKE_OVERRIDABLE_FUNCTION_DETECTABLE \ + __attribute__((__section__("__TEXT,__lcxx_override,regular,pure_instructions"))) + +_LIBCPP_BEGIN_NAMESPACE_STD +template <class _Ret, class... _Args> +_LIBCPP_HIDE_FROM_ABI bool __is_function_overridden(_Ret (*__fptr)(_Args...)) noexcept { + // Declare two dummy bytes and give them these special `__asm` values. These values are + // defined by the linker, which means that referring to `&__lcxx_override_start` will + // effectively refer to the address where the section starts (and same for the end). + extern char __lcxx_override_start __asm("section$start$__TEXT$__lcxx_override"); + extern char __lcxx_override_end __asm("section$end$__TEXT$__lcxx_override"); + + // Now get a uintptr_t out of these locations, and out of the function pointer. + uintptr_t __start = reinterpret_cast<uintptr_t>(&__lcxx_override_start); + uintptr_t __end = reinterpret_cast<uintptr_t>(&__lcxx_override_end); + uintptr_t __ptr = reinterpret_cast<uintptr_t>(__fptr); + + // Finally, the function was overridden if it falls outside of the section's bounds. + return __ptr < __start || __ptr > __end; +} +_LIBCPP_END_NAMESPACE_STD + +#elif defined(_LIBCPP_OBJECT_FORMAT_ELF) + +# define _LIBCPP_CAN_DETECT_OVERRIDDEN_FUNCTION 1 +# define _LIBCPP_MAKE_OVERRIDABLE_FUNCTION_DETECTABLE __attribute__((__section__("__lcxx_override"))) + +// This is very similar to what we do for Mach-O above. The ELF linker will implicitly define +// variables with those names corresponding to the start and the end of the section. +// +// See https://stackoverflow.com/questions/16552710/how-do-you-get-the-start-and-end-addresses-of-a-custom-elf-section +extern char __start___lcxx_override; +extern char __stop___lcxx_override; + +_LIBCPP_BEGIN_NAMESPACE_STD +template <class _Ret, class... _Args> +_LIBCPP_HIDE_FROM_ABI bool __is_function_overridden(_Ret (*__fptr)(_Args...)) noexcept { + uintptr_t __start = reinterpret_cast<uintptr_t>(&__start___lcxx_override); + uintptr_t __end = reinterpret_cast<uintptr_t>(&__stop___lcxx_override); + uintptr_t __ptr = reinterpret_cast<uintptr_t>(__fptr); + + return __ptr < __start || __ptr > __end; +} +_LIBCPP_END_NAMESPACE_STD + +#else + +# define _LIBCPP_CAN_DETECT_OVERRIDDEN_FUNCTION 0 +# define _LIBCPP_MAKE_OVERRIDABLE_FUNCTION_DETECTABLE /* nothing */ + +#endif + +#endif // _LIBCPP___OVERRIDABLE_FUNCTION diff --git a/libcxx/src/new.cpp b/libcxx/src/new.cpp index 033bba5c1fc95b6..7a3ad4136010611 100644 --- a/libcxx/src/new.cpp +++ b/libcxx/src/new.cpp @@ -7,6 +7,8 @@ //===----------------------------------------------------------------------===// #include <__memory/aligned_alloc.h> +#include <__overridable_function> +#include <cstddef> #include <cstdlib> #include <new> @@ -15,6 +17,10 @@ // The code below is copied as-is into libc++abi's libcxxabi/src/stdlib_new_delete.cpp // file. The version in this file is the canonical one. +inline void __throw_bad_alloc_shim() { std::__throw_bad_alloc(); } + +# define _LIBCPP_ASSERT_SHIM(expr, str) _LIBCPP_ASSERT(expr, str) + // ------------------ BEGIN COPY ------------------ // Implement all new and delete operators as weak definitions // in this shared library, so that they can be overridden by programs @@ -36,41 +42,61 @@ static void* operator_new_impl(std::size_t size) noexcept { return p; } -_LIBCPP_WEAK void* operator new(std::size_t size) _THROW_BAD_ALLOC { +_LIBCPP_MAKE_OVERRIDABLE_FUNCTION_DETECTABLE _LIBCPP_WEAK void* operator new(std::size_t size) _THROW_BAD_ALLOC { void* p = operator_new_impl(size); -# ifndef _LIBCPP_HAS_NO_EXCEPTIONS if (p == nullptr) - throw std::bad_alloc(); -# endif + __throw_bad_alloc_shim(); return p; } _LIBCPP_WEAK void* operator new(size_t size, const std::nothrow_t&) noexcept { +# ifdef _LIBCPP_HAS_NO_EXCEPTIONS +# if _LIBCPP_CAN_DETECT_OVERRIDDEN_FUNCTION + _LIBCPP_ASSERT_SHIM( + !std::__is_function_overridden(static_cast<void* (*)(std::size_t)>(&operator new)), + "libc++ was configured with exceptions disabled and `operator new(size_t)` has been overridden, " + "but `operator new(size_t, nothrow_t)` has not been overridden. This is problematic because " + "`operator new(size_t, nothrow_t)` must call `operator new(size_t)`, which will terminate in case " + "it fails to allocate, making it impossible for `operator new(size_t, nothrow_t)` to fulfill its " + "contract (since it should return nullptr upon failure)."); +# endif + + return operator_new_impl(size); +# else void* p = nullptr; -# ifndef _LIBCPP_HAS_NO_EXCEPTIONS try { -# endif // _LIBCPP_HAS_NO_EXCEPTIONS p = ::operator new(size); -# ifndef _LIBCPP_HAS_NO_EXCEPTIONS } catch (...) { } -# endif // _LIBCPP_HAS_NO_EXCEPTIONS return p; +# endif } -_LIBCPP_WEAK void* operator new[](size_t size) _THROW_BAD_ALLOC { return ::operator new(size); } +_LIBCPP_MAKE_OVERRIDABLE_FUNCTION_DETECTABLE _LIBCPP_WEAK void* operator new[](size_t size) _THROW_BAD_ALLOC { + return ::operator new(size); +} _LIBCPP_WEAK void* operator new[](size_t size, const std::nothrow_t&) noexcept { +# ifdef _LIBCPP_HAS_NO_EXCEPTIONS +# if _LIBCPP_CAN_DETECT_OVERRIDDEN_FUNCTION + _LIBCPP_ASSERT_SHIM( + !std::__is_function_overridden(static_cast<void* (*)(std::size_t)>(&operator new[])), + "libc++ was configured with exceptions disabled and `operator new[](size_t)` has been overridden, " + "but `operator new[](size_t, nothrow_t)` has not been overridden. This is problematic because " + "`operator new[](size_t, nothrow_t)` must call `operator new[](size_t)`, which will terminate in case " + "it fails to allocate, making it impossible for `operator new[](size_t, nothrow_t)` to fulfill its " + "contract (since it should return nullptr upon failure)."); +# endif + + return operator_new_impl(size); +# else void* p = nullptr; -# ifndef _LIBCPP_HAS_NO_EXCEPTIONS try { -# endif // _LIBCPP_HAS_NO_EXCEPTIONS p = ::operator new[](size); -# ifndef _LIBCPP_HAS_NO_EXCEPTIONS } catch (...) { } -# endif // _LIBCPP_HAS_NO_EXCEPTIONS return p; +# endif } _LIBCPP_WEAK void operator delete(void* ptr) noexcept { std::free(ptr); } @@ -107,43 +133,64 @@ static void* operator_new_aligned_impl(std::size_t size, std::align_val_t alignm return p; } -_LIBCPP_WEAK void* operator new(std::size_t size, std::align_val_t alignment) _THROW_BAD_ALLOC { +_LIBCPP_MAKE_OVERRIDABLE_FUNCTION_DETECTABLE _LIBCPP_WEAK void* +operator new(std::size_t size, std::align_val_t alignment) _THROW_BAD_ALLOC { void* p = operator_new_aligned_impl(size, alignment); -# ifndef _LIBCPP_HAS_NO_EXCEPTIONS if (p == nullptr) - throw std::bad_alloc(); -# endif + __throw_bad_alloc_shim(); return p; } _LIBCPP_WEAK void* operator new(size_t size, std::align_val_t alignment, const std::nothrow_t&) noexcept { +# ifdef _LIBCPP_HAS_NO_EXCEPTIONS +# if _LIBCPP_CAN_DETECT_OVERRIDDEN_FUNCTION + _LIBCPP_ASSERT_SHIM( + !std::__is_function_overridden(static_cast<void* (*)(std::size_t, std::align_val_t)>(&operator new)), + "libc++ was configured with exceptions disabled and `operator new(size_t, align_val_t)` has been overridden, " + "but `operator new(size_t, align_val_t, nothrow_t)` has not been overridden. This is problematic because " + "`operator new(size_t, align_val_t, nothrow_t)` must call `operator new(size_t, align_val_t)`, which will " + "terminate in case it fails to allocate, making it impossible for `operator new(size_t, align_val_t, nothrow_t)` " + "to fulfill its contract (since it should return nullptr upon failure)."); +# endif + + return operator_new_aligned_impl(size, alignment); +# else void* p = nullptr; -# ifndef _LIBCPP_HAS_NO_EXCEPTIONS try { -# endif // _LIBCPP_HAS_NO_EXCEPTIONS p = ::operator new(size, alignment); -# ifndef _LIBCPP_HAS_NO_EXCEPTIONS } catch (...) { } -# endif // _LIBCPP_HAS_NO_EXCEPTIONS return p; +# endif } -_LIBCPP_WEAK void* operator new[](size_t size, std::align_val_t alignment) _THROW_BAD_ALLOC { +_LIBCPP_MAKE_OVERRIDABLE_FUNCTION_DETECTABLE _LIBCPP_WEAK void* +operator new[](size_t size, std::align_val_t alignment) _THROW_BAD_ALLOC { return ::operator new(size, alignment); } _LIBCPP_WEAK void* operator new[](size_t size, std::align_val_t alignment, const std::nothrow_t&) noexcept { +# ifdef _LIBCPP_HAS_NO_EXCEPTIONS +# if _LIBCPP_CAN_DETECT_OVERRIDDEN_FUNCTION + _LIBCPP_ASSERT_SHIM( + !std::__is_function_overridden(static_cast<void* (*)(std::size_t, std::align_val_t)>(&operator new[])), + "libc++ was configured with exceptions disabled and `operator new[](size_t, align_val_t)` has been overridden, " + "but `operator new[](size_t, align_val_t, nothrow_t)` has not been overridden. This is problematic because " + "`operator new[](size_t, align_val_t, nothrow_t)` must call `operator new[](size_t, align_val_t)`, which will " + "terminate in case it fails to allocate, making it impossible for `operator new[](size_t, align_val_t, " + "nothrow_t)` " + "to fulfill its contract (since it should return nullptr upon failure)."); +# endif + + return operator_new_aligned_impl(size, alignment); +# else void* p = nullptr; -# ifndef _LIBCPP_HAS_NO_EXCEPTIONS try { -# endif // _LIBCPP_HAS_NO_EXCEPTIONS p = ::operator new[](size, alignment); -# ifndef _LIBCPP_HAS_NO_EXCEPTIONS } catch (...) { } -# endif // _LIBCPP_HAS_NO_EXCEPTIONS return p; +# endif } _LIBCPP_WEAK void operator delete(void* ptr, std::align_val_t) noexcept { std::__libcpp_aligned_free(ptr); } diff --git a/libcxx/test/libcxx/language.support/support.dynamic/assert.nothrow_new_not_overridden_fno_exceptions.pass.cpp b/libcxx/test/libcxx/language.support/support.dynamic/assert.nothrow_new_not_overridden_fno_exceptions.pass.cpp new file mode 100644 index 000000000000000..7aa51b365f74eba --- /dev/null +++ b/libcxx/test/libcxx/language.support/support.dynamic/assert.nothrow_new_not_overridden_fno_exceptions.pass.cpp @@ -0,0 +1,59 @@ +//===----------------------------------------------------------------------===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +// void* operator new(std::size_t, const std::nothrow_t&); +// void* operator new(std::size_t, std::align_val_t, const std::nothrow_t&); +// void* operator new[](std::size_t, const std::nothrow_t&); +// void* operator new[](std::size_t, std::align_val_t, const std::nothrow_t&); + +// This test ensures that we catch the case where `new` has been overridden but `new(nothrow)` +// has not been overridden, and the library is compiled with -fno-exceptions. +// +// In that case, it is impossible for libc++ to provide a Standards conforming implementation +// of `new(nothrow)`, so the only viable option is to terminate the program. + +// REQUIRES: has-unix-headers +// UNSUPPORTED: c++03 +// XFAIL: availability-verbose_abort-missing + +// We only know how to diagnose this on platforms that use the ELF or Mach-O object file formats. +// XFAIL: target={{.+}}-windows-{{.+}} + +// TODO: We currently don't have a way to express that the built library was +// compiled with -fno-exceptions, so if the library was built with support +// for exceptions but we run the test suite without exceptions, this will +// spuriously fail. +// REQUIRES: no-exceptions + +#include <cstddef> +#include <new> + +#include "check_assertion.h" + +// Override the throwing versions of operator new, but not the nothrow versions. +alignas(32) char DummyData[32 * 3]; +void* operator new(std::size_t) { return DummyData; } +void* operator new(std::size_t, std::align_val_t) { return DummyData; } +void* operator new[](std::size_t) { return DummyData; } +void* operator new[](std::size_t, std::align_val_t) { return DummyData; } + +void operator delete(void*) noexcept {} +void operator delete(void*, std::align_val_t) noexcept {} +void operator delete[](void*) noexcept {} +void operator delete[](void*, std::align_val_t) noexcept {} + +int main(int, char**) { + std::size_t size = 3; + std::align_val_t align = static_cast<std::align_val_t>(32); + EXPECT_DEATH((void)operator new(size, std::nothrow)); + EXPECT_DEATH((void)operator new(size, align, std::nothrow)); + EXPECT_DEATH((void)operator new[](size, std::nothrow)); + EXPECT_DEATH((void)operator new[](size, align, std::nothrow)); + + return 0; +} diff --git a/libcxx/test/libcxx/language.support/support.dynamic/new_dont_return_nullptr.pass.cpp b/libcxx/test/libcxx/language.support/support.dynamic/new_dont_return_nullptr.pass.cpp new file mode 100644 index 000000000000000..548046b0ab43d28 --- /dev/null +++ b/libcxx/test/libcxx/language.support/support.dynamic/new_dont_return_nullptr.pass.cpp @@ -0,0 +1,37 @@ +//===----------------------------------------------------------------------===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +// void* operator new(std::size_t); +// void* operator new(std::size_t, std::align_val_t); +// void* operator new[](std::size_t); +// void* operator new[](std::size_t, std::align_val_t); + +// This test ensures that we abort the program instead of returning nullptr +// when we fail to satisfy the allocation request. The throwing versions of +// `operator new` must never return nullptr on failure to allocate (per the +// Standard) and the compiler actually relies on that for optimizations. +// Returning nullptr from the throwing `operator new` can basically result +// in miscompiles. + +// REQUIRES: has-unix-headers +// REQUIRES: no-exceptions +// UNSUPPORTED: c++03, c++11, c++14 + +#include <cstddef> +#include <limits> +#include <new> + +#include "check_assertion.h" + +int main(int, char**) { + EXPECT_DEATH((void)operator new(std::numeric_limits<std::size_t>::max())); + EXPECT_DEATH((void)operator new(std::numeric_limits<std::size_t>::max(), static_cast<std::align_val_t>(32))); + EXPECT_DEATH((void)operator new[](std::numeric_limits<std::size_t>::max())); + EXPECT_DEATH((void)operator new[](std::numeric_limits<std::size_t>::max(), static_cast<std::align_val_t>(32))); + return 0; +} diff --git a/libcxx/test/support/check_assertion.h b/libcxx/test/support/check_assertion.h index 98dd95b11556e6c..34e41e8f0d8eaf8 100644 --- a/libcxx/test/support/check_assertion.h +++ b/libcxx/test/support/check_assertion.h @@ -10,6 +10,7 @@ #define TEST_SUPPORT_CHECK_ASSERTION_H #include <cassert> +#include <csignal> #include <cstdarg> #include <cstddef> #include <cstdio> @@ -257,9 +258,14 @@ void std::__libcpp_verbose_abort(char const* format, ...) { std::exit(DeathTest::RK_Terminate); } +[[noreturn]] inline void abort_handler(int) { + std::exit(DeathTest::RK_Terminate); +} + template <class Func> inline bool ExpectDeath(const char* stmt, Func&& func, AssertionInfoMatcher Matcher) { std::set_terminate(terminate_handler); + std::signal(SIGABRT, abort_handler); DeathTest DT(Matcher); DeathTest::ResultKind RK = DT.Run(func); auto OnFailure = [&](const char* msg) { diff --git a/libcxx/utils/generate_iwyu_mapping.py b/libcxx/utils/generate_iwyu_mapping.py index 343538a6cae4819..88840eff8f6e293 100644 --- a/libcxx/utils/generate_iwyu_mapping.py +++ b/libcxx/utils/generate_iwyu_mapping.py @@ -65,6 +65,8 @@ def generate_map(include): continue elif i == "__node_handle": public = ["map", "set", "unordered_map", "unordered_set"] + elif i == "__overridable_function": + continue elif i == "__pstl_algorithm": continue elif i == "__pstl_config_site.in": diff --git a/libcxxabi/src/stdlib_new_delete.cpp b/libcxxabi/src/stdlib_new_delete.cpp index 6c9990f063dde66..5bd9b5e58b36c6a 100644 --- a/libcxxabi/src/stdlib_new_delete.cpp +++ b/libcxxabi/src/stdlib_new_delete.cpp @@ -7,7 +7,10 @@ //===----------------------------------------------------------------------===// #include "__cxxabi_config.h" +#include "abort_message.h" #include <__memory/aligned_alloc.h> +#include <__overridable_function> +#include <cstddef> #include <cstdlib> #include <new> @@ -25,6 +28,20 @@ # error libc++ and libc++abi seem to disagree on whether exceptions are enabled #endif +inline void __throw_bad_alloc_shim() { +#ifndef _LIBCPP_HAS_NO_EXCEPTIONS + throw std::bad_alloc(); +#else + abort_message("bad_alloc was thrown in -fno-exceptions mode"); +#endif +} + +#define _LIBCPP_ASSERT_SHIM(expr, str) \ + do { \ + if (!expr) \ + abort_message(str); \ + } while (false) + // ------------------ BEGIN COPY ------------------ // Implement all new and delete operators as weak definitions // in this shared library, so that they can be overridden by programs @@ -46,64 +63,74 @@ static void* operator_new_impl(std::size_t size) noexcept { return p; } -_LIBCPP_WEAK -void* operator new(std::size_t size) _THROW_BAD_ALLOC { +_LIBCPP_MAKE_OVERRIDABLE_FUNCTION_DETECTABLE _LIBCPP_WEAK void* operator new(std::size_t size) _THROW_BAD_ALLOC { void* p = operator_new_impl(size); -#ifndef _LIBCPP_HAS_NO_EXCEPTIONS if (p == nullptr) - throw std::bad_alloc(); -#endif + __throw_bad_alloc_shim(); return p; } -_LIBCPP_WEAK -void* operator new(size_t size, const std::nothrow_t&) noexcept { +_LIBCPP_WEAK void* operator new(size_t size, const std::nothrow_t&) noexcept { +#ifdef _LIBCPP_HAS_NO_EXCEPTIONS +# if _LIBCPP_CAN_DETECT_OVERRIDDEN_FUNCTION + _LIBCPP_ASSERT_SHIM( + !std::__is_function_overridden(static_cast<void* (*)(std::size_t)>(&operator new)), + "libc++ was configured with exceptions disabled and `operator new(size_t)` has been overridden, " + "but `operator new(size_t, nothrow_t)` has not been overridden. This is problematic because " + "`operator new(size_t, nothrow_t)` must call `operator new(size_t)`, which will terminate in case " + "it fails to allocate, making it impossible for `operator new(size_t, nothrow_t)` to fulfill its " + "contract (since it should return nullptr upon failure)."); +# endif + + return operator_new_impl(size); +#else void* p = nullptr; -#ifndef _LIBCPP_HAS_NO_EXCEPTIONS try { -#endif // _LIBCPP_HAS_NO_EXCEPTIONS p = ::operator new(size); -#ifndef _LIBCPP_HAS_NO_EXCEPTIONS } catch (...) { } -#endif // _LIBCPP_HAS_NO_EXCEPTIONS return p; +#endif } -_LIBCPP_WEAK -void* operator new[](size_t size) _THROW_BAD_ALLOC { return ::operator new(size); } +_LIBCPP_MAKE_OVERRIDABLE_FUNCTION_DETECTABLE _LIBCPP_WEAK void* operator new[](size_t size) _THROW_BAD_ALLOC { + return ::operator new(size); +} -_LIBCPP_WEAK -void* operator new[](size_t size, const std::nothrow_t&) noexcept { +_LIBCPP_WEAK void* operator new[](size_t size, const std::nothrow_t&) noexcept { +#ifdef _LIBCPP_HAS_NO_EXCEPTIONS +# if _LIBCPP_CAN_DETECT_OVERRIDDEN_FUNCTION + _LIBCPP_ASSERT_SHIM( + !std::__is_function_overridden(static_cast<void* (*)(std::size_t)>(&operator new[])), + "libc++ was configured with exceptions disabled and `operator new[](size_t)` has been overridden, " + "but `operator new[](size_t, nothrow_t)` has not been overridden. This is problematic because " + "`operator new[](size_t, nothrow_t)` must call `operator new[](size_t)`, which will terminate in case " + "it fails to allocate, making it impossible for `operator new[](size_t, nothrow_t)` to fulfill its " + "contract (since it should return nullptr upon failure)."); +# endif + + return operator_new_impl(size); +#else void* p = nullptr; -#ifndef _LIBCPP_HAS_NO_EXCEPTIONS try { -#endif // _LIBCPP_HAS_NO_EXCEPTIONS p = ::operator new[](size); -#ifndef _LIBCPP_HAS_NO_EXCEPTIONS } catch (...) { } -#endif // _LIBCPP_HAS_NO_EXCEPTIONS return p; +#endif } -_LIBCPP_WEAK -void operator delete(void* ptr) noexcept { std::free(ptr); } +_LIBCPP_WEAK void operator delete(void* ptr) noexcept { std::free(ptr); } -_LIBCPP_WEAK -void operator delete(void* ptr, const std::nothrow_t&) noexcept { ::operator delete(ptr); } +_LIBCPP_WEAK void operator delete(void* ptr, const std::nothrow_t&) noexcept { ::operator delete(ptr); } -_LIBCPP_WEAK -void operator delete(void* ptr, size_t) noexcept { ::operator delete(ptr); } +_LIBCPP_WEAK void operator delete(void* ptr, size_t) noexcept { ::operator delete(ptr); } -_LIBCPP_WEAK -void operator delete[](void* ptr) noexcept { ::operator delete(ptr); } +_LIBCPP_WEAK void operator delete[](void* ptr) noexcept { ::operator delete(ptr); } -_LIBCPP_WEAK -void operator delete[](void* ptr, const std::nothrow_t&) noexcept { ::operator delete[](ptr); } +_LIBCPP_WEAK void operator delete[](void* ptr, const std::nothrow_t&) noexcept { ::operator delete[](ptr); } -_LIBCPP_WEAK -void operator delete[](void* ptr, size_t) noexcept { ::operator delete[](ptr); } +_LIBCPP_WEAK void operator delete[](void* ptr, size_t) noexcept { ::operator delete[](ptr); } #if !defined(_LIBCPP_HAS_NO_LIBRARY_ALIGNED_ALLOCATION) @@ -127,70 +154,87 @@ static void* operator_new_aligned_impl(std::size_t size, std::align_val_t alignm return p; } -_LIBCPP_WEAK -void* operator new(std::size_t size, std::align_val_t alignment) _THROW_BAD_ALLOC { +_LIBCPP_MAKE_OVERRIDABLE_FUNCTION_DETECTABLE _LIBCPP_WEAK void* +operator new(std::size_t size, std::align_val_t alignment) _THROW_BAD_ALLOC { void* p = operator_new_aligned_impl(size, alignment); -# ifndef _LIBCPP_HAS_NO_EXCEPTIONS if (p == nullptr) - throw std::bad_alloc(); -# endif + __throw_bad_alloc_shim(); return p; } -_LIBCPP_WEAK -void* operator new(size_t size, std::align_val_t alignment, const std::nothrow_t&) noexcept { +_LIBCPP_WEAK void* operator new(size_t size, std::align_val_t alignment, const std::nothrow_t&) noexcept { +# ifdef _LIBCPP_HAS_NO_EXCEPTIONS +# if _LIBCPP_CAN_DETECT_OVERRIDDEN_FUNCTION + _LIBCPP_ASSERT_SHIM( + !std::__is_function_overridden(static_cast<void* (*)(std::size_t, std::align_val_t)>(&operator new)), + "libc++ was configured with exceptions disabled and `operator new(size_t, align_val_t)` has been overridden, " + "but `operator new(size_t, align_val_t, nothrow_t)` has not been overridden. This is problematic because " + "`operator new(size_t, align_val_t, nothrow_t)` must call `operator new(size_t, align_val_t)`, which will " + "terminate in case it fails to allocate, making it impossible for `operator new(size_t, align_val_t, nothrow_t)` " + "to fulfill its contract (since it should return nullptr upon failure)."); +# endif + + return operator_new_aligned_impl(size, alignment); +# else void* p = nullptr; -# ifndef _LIBCPP_HAS_NO_EXCEPTIONS try { -# endif // _LIBCPP_HAS_NO_EXCEPTIONS p = ::operator new(size, alignment); -# ifndef _LIBCPP_HAS_NO_EXCEPTIONS } catch (...) { } -# endif // _LIBCPP_HAS_NO_EXCEPTIONS return p; +# endif } -_LIBCPP_WEAK -void* operator new[](size_t size, std::align_val_t alignment) _THROW_BAD_ALLOC { +_LIBCPP_MAKE_OVERRIDABLE_FUNCTION_DETECTABLE _LIBCPP_WEAK void* +operator new[](size_t size, std::align_val_t alignment) _THROW_BAD_ALLOC { return ::operator new(size, alignment); } -_LIBCPP_WEAK -void* operator new[](size_t size, std::align_val_t alignment, const std::nothrow_t&) noexcept { +_LIBCPP_WEAK void* operator new[](size_t size, std::align_val_t alignment, const std::nothrow_t&) noexcept { +# ifdef _LIBCPP_HAS_NO_EXCEPTIONS +# if _LIBCPP_CAN_DETECT_OVERRIDDEN_FUNCTION + _LIBCPP_ASSERT_SHIM( + !std::__is_function_overridden(static_cast<void* (*)(std::size_t, std::align_val_t)>(&operator new[])), + "libc++ was configured with exceptions disabled and `operator new[](size_t, align_val_t)` has been overridden, " + "but `operator new[](size_t, align_val_t, nothrow_t)` has not been overridden. This is problematic because " + "`operator new[](size_t, align_val_t, nothrow_t)` must call `operator new[](size_t, align_val_t)`, which will " + "terminate in case it fails to allocate, making it impossible for `operator new[](size_t, align_val_t, " + "nothrow_t)` " + "to fulfill its contract (since it should return nullptr upon failure)."); +# endif + + return operator_new_aligned_impl(size, alignment); +# else void* p = nullptr; -# ifndef _LIBCPP_HAS_NO_EXCEPTIONS try { -# endif // _LIBCPP_HAS_NO_EXCEPTIONS p = ::operator new[](size, alignment); -# ifndef _LIBCPP_HAS_NO_EXCEPTIONS } catch (...) { } -# endif // _LIBCPP_HAS_NO_EXCEPTIONS return p; +# endif } -_LIBCPP_WEAK -void operator delete(void* ptr, std::align_val_t) noexcept { std::__libcpp_aligned_free(ptr); } +_LIBCPP_WEAK void operator delete(void* ptr, std::align_val_t) noexcept { std::__libcpp_aligned_free(ptr); } -_LIBCPP_WEAK -void operator delete(void* ptr, std::align_val_t alignment, const std::nothrow_t&) noexcept { +_LIBCPP_WEAK void operator delete(void* ptr, std::align_val_t alignment, const std::nothrow_t&) noexcept { ::operator delete(ptr, alignment); } -_LIBCPP_WEAK -void operator delete(void* ptr, size_t, std::align_val_t alignment) noexcept { ::operator delete(ptr, alignment); } +_LIBCPP_WEAK void operator delete(void* ptr, size_t, std::align_val_t alignment) noexcept { + ::operator delete(ptr, alignment); +} -_LIBCPP_WEAK -void operator delete[](void* ptr, std::align_val_t alignment) noexcept { ::operator delete(ptr, alignment); } +_LIBCPP_WEAK void operator delete[](void* ptr, std::align_val_t alignment) noexcept { + ::operator delete(ptr, alignment); +} -_LIBCPP_WEAK -void operator delete[](void* ptr, std::align_val_t alignment, const std::nothrow_t&) noexcept { +_LIBCPP_WEAK void operator delete[](void* ptr, std::align_val_t alignment, const std::nothrow_t&) noexcept { ::operator delete[](ptr, alignment); } -_LIBCPP_WEAK -void operator delete[](void* ptr, size_t, std::align_val_t alignment) noexcept { ::operator delete[](ptr, alignment); } +_LIBCPP_WEAK void operator delete[](void* ptr, size_t, std::align_val_t alignment) noexcept { + ::operator delete[](ptr, alignment); +} #endif // !_LIBCPP_HAS_NO_LIBRARY_ALIGNED_ALLOCATION // ------------------ END COPY ------------------ >From 1b5144f9f8d18580b6ceca264ce033e7ede0907b Mon Sep 17 00:00:00 2001 From: Louis Dionne <ldionn...@gmail.com> Date: Tue, 14 Nov 2023 22:55:06 -0500 Subject: [PATCH 2/8] Improve diagnostic --- libcxx/src/new.cpp | 14 +++++++++----- libcxxabi/src/stdlib_new_delete.cpp | 14 +++++++++----- 2 files changed, 18 insertions(+), 10 deletions(-) diff --git a/libcxx/src/new.cpp b/libcxx/src/new.cpp index 7a3ad4136010611..5ca9d6b43da443f 100644 --- a/libcxx/src/new.cpp +++ b/libcxx/src/new.cpp @@ -58,7 +58,8 @@ _LIBCPP_WEAK void* operator new(size_t size, const std::nothrow_t&) noexcept { "but `operator new(size_t, nothrow_t)` has not been overridden. This is problematic because " "`operator new(size_t, nothrow_t)` must call `operator new(size_t)`, which will terminate in case " "it fails to allocate, making it impossible for `operator new(size_t, nothrow_t)` to fulfill its " - "contract (since it should return nullptr upon failure)."); + "contract (since it should return nullptr upon failure). Please make sure you override " + "`operator new(size_t, nothrow_t)` as well."); # endif return operator_new_impl(size); @@ -85,7 +86,8 @@ _LIBCPP_WEAK void* operator new[](size_t size, const std::nothrow_t&) noexcept { "but `operator new[](size_t, nothrow_t)` has not been overridden. This is problematic because " "`operator new[](size_t, nothrow_t)` must call `operator new[](size_t)`, which will terminate in case " "it fails to allocate, making it impossible for `operator new[](size_t, nothrow_t)` to fulfill its " - "contract (since it should return nullptr upon failure)."); + "contract (since it should return nullptr upon failure). Please make sure you override " + "`operator new[](size_t, nothrow_t)` as well."); # endif return operator_new_impl(size); @@ -150,7 +152,8 @@ _LIBCPP_WEAK void* operator new(size_t size, std::align_val_t alignment, const s "but `operator new(size_t, align_val_t, nothrow_t)` has not been overridden. This is problematic because " "`operator new(size_t, align_val_t, nothrow_t)` must call `operator new(size_t, align_val_t)`, which will " "terminate in case it fails to allocate, making it impossible for `operator new(size_t, align_val_t, nothrow_t)` " - "to fulfill its contract (since it should return nullptr upon failure)."); + "to fulfill its contract (since it should return nullptr upon failure). Please make sure you override " + "`operator new(size_t, align_val_t, nothrow_t)` as well."); # endif return operator_new_aligned_impl(size, alignment); @@ -178,8 +181,9 @@ _LIBCPP_WEAK void* operator new[](size_t size, std::align_val_t alignment, const "but `operator new[](size_t, align_val_t, nothrow_t)` has not been overridden. This is problematic because " "`operator new[](size_t, align_val_t, nothrow_t)` must call `operator new[](size_t, align_val_t)`, which will " "terminate in case it fails to allocate, making it impossible for `operator new[](size_t, align_val_t, " - "nothrow_t)` " - "to fulfill its contract (since it should return nullptr upon failure)."); + "nothrow_t)` to fulfill its contract (since it should return nullptr upon failure). Please make sure you " + "override " + "`operator new[](size_t, align_val_t, nothrow_t)` as well."); # endif return operator_new_aligned_impl(size, alignment); diff --git a/libcxxabi/src/stdlib_new_delete.cpp b/libcxxabi/src/stdlib_new_delete.cpp index 5bd9b5e58b36c6a..cba6c3968775ab2 100644 --- a/libcxxabi/src/stdlib_new_delete.cpp +++ b/libcxxabi/src/stdlib_new_delete.cpp @@ -79,7 +79,8 @@ _LIBCPP_WEAK void* operator new(size_t size, const std::nothrow_t&) noexcept { "but `operator new(size_t, nothrow_t)` has not been overridden. This is problematic because " "`operator new(size_t, nothrow_t)` must call `operator new(size_t)`, which will terminate in case " "it fails to allocate, making it impossible for `operator new(size_t, nothrow_t)` to fulfill its " - "contract (since it should return nullptr upon failure)."); + "contract (since it should return nullptr upon failure). Please make sure you override " + "`operator new(size_t, nothrow_t)` as well."); # endif return operator_new_impl(size); @@ -106,7 +107,8 @@ _LIBCPP_WEAK void* operator new[](size_t size, const std::nothrow_t&) noexcept { "but `operator new[](size_t, nothrow_t)` has not been overridden. This is problematic because " "`operator new[](size_t, nothrow_t)` must call `operator new[](size_t)`, which will terminate in case " "it fails to allocate, making it impossible for `operator new[](size_t, nothrow_t)` to fulfill its " - "contract (since it should return nullptr upon failure)."); + "contract (since it should return nullptr upon failure). Please make sure you override " + "`operator new[](size_t, nothrow_t)` as well."); # endif return operator_new_impl(size); @@ -171,7 +173,8 @@ _LIBCPP_WEAK void* operator new(size_t size, std::align_val_t alignment, const s "but `operator new(size_t, align_val_t, nothrow_t)` has not been overridden. This is problematic because " "`operator new(size_t, align_val_t, nothrow_t)` must call `operator new(size_t, align_val_t)`, which will " "terminate in case it fails to allocate, making it impossible for `operator new(size_t, align_val_t, nothrow_t)` " - "to fulfill its contract (since it should return nullptr upon failure)."); + "to fulfill its contract (since it should return nullptr upon failure). Please make sure you override " + "`operator new(size_t, align_val_t, nothrow_t)` as well."); # endif return operator_new_aligned_impl(size, alignment); @@ -199,8 +202,9 @@ _LIBCPP_WEAK void* operator new[](size_t size, std::align_val_t alignment, const "but `operator new[](size_t, align_val_t, nothrow_t)` has not been overridden. This is problematic because " "`operator new[](size_t, align_val_t, nothrow_t)` must call `operator new[](size_t, align_val_t)`, which will " "terminate in case it fails to allocate, making it impossible for `operator new[](size_t, align_val_t, " - "nothrow_t)` " - "to fulfill its contract (since it should return nullptr upon failure)."); + "nothrow_t)` to fulfill its contract (since it should return nullptr upon failure). Please make sure you " + "override " + "`operator new[](size_t, align_val_t, nothrow_t)` as well."); # endif return operator_new_aligned_impl(size, alignment); >From 0f2b77ffa9074b750189e99f8f9cbfd5159a9be1 Mon Sep 17 00:00:00 2001 From: Louis Dionne <ldionn...@gmail.com> Date: Tue, 14 Nov 2023 23:10:35 -0500 Subject: [PATCH 3/8] Make sure to override operator new(nothrow) count_new tests --- libcxx/test/support/count_new.h | 105 +++++++++++++++++++++----------- 1 file changed, 68 insertions(+), 37 deletions(-) diff --git a/libcxx/test/support/count_new.h b/libcxx/test/support/count_new.h index b6424850101625b..f89cdc0c01487d9 100644 --- a/libcxx/test/support/count_new.h +++ b/libcxx/test/support/count_new.h @@ -374,78 +374,109 @@ TEST_DIAGNOSTIC_POP MemCounter &globalMemCounter = *getGlobalMemCounter(); #ifndef DISABLE_NEW_COUNT -void* operator new(std::size_t s) TEST_THROW_SPEC(std::bad_alloc) -{ - getGlobalMemCounter()->newCalled(s); - void* ret = std::malloc(s); - if (ret == nullptr) - detail::throw_bad_alloc_helper(); - return ret; +// operator new(size_t[, nothrow_t]) and operator delete(size_t[, nothrow_t]) +void* operator new(std::size_t s) TEST_THROW_SPEC(std::bad_alloc) { + void* ret = operator new(s, std::nothrow); + if (ret == nullptr) + detail::throw_bad_alloc_helper(); + return ret; } -void operator delete(void* p) TEST_NOEXCEPT -{ - getGlobalMemCounter()->deleteCalled(p); - std::free(p); +void* operator new(std::size_t s, std::nothrow_t const&) TEST_NOEXCEPT { + getGlobalMemCounter()->newCalled(s); + void* ret = std::malloc(s); + return ret; } -void* operator new[](std::size_t s) TEST_THROW_SPEC(std::bad_alloc) -{ - getGlobalMemCounter()->newArrayCalled(s); - return operator new(s); +void operator delete(void* p) TEST_NOEXCEPT { operator delete(p, std::nothrow); } + +void operator delete(void* p, std::nothrow_t const&) TEST_NOEXCEPT { + getGlobalMemCounter()->deleteCalled(p); + std::free(p); } -void operator delete[](void* p) TEST_NOEXCEPT -{ - getGlobalMemCounter()->deleteArrayCalled(p); - operator delete(p); +// operator new[](size_t[, nothrow_t]) and operator delete[](size_t[, nothrow_t]) +void* operator new[](std::size_t s) TEST_THROW_SPEC(std::bad_alloc) { + void* ret = operator new[](s, std::nothrow); + if (ret == nullptr) + detail::throw_bad_alloc_helper(); + return ret; } -#ifndef TEST_HAS_NO_ALIGNED_ALLOCATION -#if defined(_LIBCPP_MSVCRT_LIKE) || \ - (!defined(_LIBCPP_VERSION) && defined(_WIN32)) -#define USE_ALIGNED_ALLOC -#endif +void* operator new[](std::size_t s, std::nothrow_t const&) TEST_NOEXCEPT { + getGlobalMemCounter()->newArrayCalled(s); + return operator new(s); +} + +void operator delete[](void* p) TEST_NOEXCEPT { operator delete[](p, std::nothrow); } + +void operator delete[](void* p, std::nothrow_t const&) TEST_NOEXCEPT { + getGlobalMemCounter()->deleteArrayCalled(p); + operator delete(p); +} +# ifndef TEST_HAS_NO_ALIGNED_ALLOCATION +# if defined(_LIBCPP_MSVCRT_LIKE) || (!defined(_LIBCPP_VERSION) && defined(_WIN32)) +# define USE_ALIGNED_ALLOC +# endif + +// operator new(size_t, align_val_t[, nothrow_t]) and operator delete(size_t, align_val_t[, nothrow_t]) void* operator new(std::size_t s, std::align_val_t av) TEST_THROW_SPEC(std::bad_alloc) { + void* p = operator new(s, av, std::nothrow); + if (p == nullptr) + detail::throw_bad_alloc_helper(); + return p; +} + +void* operator new(std::size_t s, std::align_val_t av, std::nothrow_t const&) TEST_NOEXCEPT { const std::size_t a = static_cast<std::size_t>(av); getGlobalMemCounter()->alignedNewCalled(s, a); - void *ret = nullptr; -#ifdef USE_ALIGNED_ALLOC + void* ret = nullptr; +# ifdef USE_ALIGNED_ALLOC ret = _aligned_malloc(s, a); -#else +# else assert(posix_memalign(&ret, std::max(a, sizeof(void*)), s) != EINVAL); -#endif - if (ret == nullptr) - detail::throw_bad_alloc_helper(); +# endif return ret; } -void operator delete(void *p, std::align_val_t av) TEST_NOEXCEPT { +void operator delete(void* p, std::align_val_t av) TEST_NOEXCEPT { operator delete(p, av, std::nothrow); } + +void operator delete(void* p, std::align_val_t av, std::nothrow_t const&) TEST_NOEXCEPT { const std::size_t a = static_cast<std::size_t>(av); getGlobalMemCounter()->alignedDeleteCalled(p, a); if (p) { -#ifdef USE_ALIGNED_ALLOC +# ifdef USE_ALIGNED_ALLOC ::_aligned_free(p); -#else +# else ::free(p); -#endif +# endif } } +// operator new[](size_t, align_val_t[, nothrow_t]) and operator delete[](size_t, align_val_t[, nothrow_t]) void* operator new[](std::size_t s, std::align_val_t av) TEST_THROW_SPEC(std::bad_alloc) { + void* ret = operator new[](s, av, std::nothrow); + if (ret == nullptr) + detail::throw_bad_alloc_helper(); + return ret; +} + +void* operator new[](std::size_t s, std::align_val_t av, std::nothrow_t const&) TEST_NOEXCEPT { const std::size_t a = static_cast<std::size_t>(av); getGlobalMemCounter()->alignedNewArrayCalled(s, a); - return operator new(s, av); + return operator new(s, av, std::nothrow); } -void operator delete[](void *p, std::align_val_t av) TEST_NOEXCEPT { +void operator delete[](void* p, std::align_val_t av) TEST_NOEXCEPT { operator delete[](p, av, std::nothrow); } + +void operator delete[](void* p, std::align_val_t av, std::nothrow_t const&) TEST_NOEXCEPT { const std::size_t a = static_cast<std::size_t>(av); getGlobalMemCounter()->alignedDeleteArrayCalled(p, a); return operator delete(p, av); } -#endif // TEST_HAS_NO_ALIGNED_ALLOCATION +# endif // TEST_HAS_NO_ALIGNED_ALLOCATION #endif // DISABLE_NEW_COUNT >From 82d3e73b83ef0273ce6067ff803ae08cd5e3e406 Mon Sep 17 00:00:00 2001 From: Louis Dionne <ldionn...@gmail.com> Date: Mon, 18 Dec 2023 14:25:18 -0500 Subject: [PATCH 4/8] Add __overridable_function to modulemap --- libcxx/include/module.modulemap.in | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/libcxx/include/module.modulemap.in b/libcxx/include/module.modulemap.in index 5f57a8a2b1bf71e..4422598d15dcbf1 100644 --- a/libcxx/include/module.modulemap.in +++ b/libcxx/include/module.modulemap.in @@ -608,6 +608,10 @@ module std_private_node_handle [system] { header "__node_handle" export * } +module std_private_overridable_function [system] { + header "__overridable_function" + export * +} module std_private_split_buffer [system] { header "__split_buffer" export * >From 398091447a8377f243fa2f6c960089af8301eb54 Mon Sep 17 00:00:00 2001 From: Louis Dionne <ldionn...@gmail.com> Date: Mon, 18 Dec 2023 15:11:48 -0500 Subject: [PATCH 5/8] Fix implementation of count_new.h operators --- libcxx/test/support/count_new.h | 132 ++++++++++++++++++++++---------- 1 file changed, 92 insertions(+), 40 deletions(-) diff --git a/libcxx/test/support/count_new.h b/libcxx/test/support/count_new.h index f89cdc0c01487d9..93142fc9ad4dede 100644 --- a/libcxx/test/support/count_new.h +++ b/libcxx/test/support/count_new.h @@ -376,19 +376,30 @@ MemCounter &globalMemCounter = *getGlobalMemCounter(); #ifndef DISABLE_NEW_COUNT // operator new(size_t[, nothrow_t]) and operator delete(size_t[, nothrow_t]) void* operator new(std::size_t s) TEST_THROW_SPEC(std::bad_alloc) { - void* ret = operator new(s, std::nothrow); - if (ret == nullptr) + getGlobalMemCounter()->newCalled(s); + void* p = std::malloc(s); + if (p == nullptr) detail::throw_bad_alloc_helper(); - return ret; + return p; } void* operator new(std::size_t s, std::nothrow_t const&) TEST_NOEXCEPT { +# ifdef TEST_HAS_NO_EXCEPTIONS getGlobalMemCounter()->newCalled(s); - void* ret = std::malloc(s); - return ret; +# else + try { + getGlobalMemCounter()->newCalled(s); + } catch (std::bad_alloc const&) { + return nullptr; + } +# endif + return std::malloc(s); } -void operator delete(void* p) TEST_NOEXCEPT { operator delete(p, std::nothrow); } +void operator delete(void* p) TEST_NOEXCEPT { + getGlobalMemCounter()->deleteCalled(p); + std::free(p); +} void operator delete(void* p, std::nothrow_t const&) TEST_NOEXCEPT { getGlobalMemCounter()->deleteCalled(p); @@ -397,22 +408,34 @@ void operator delete(void* p, std::nothrow_t const&) TEST_NOEXCEPT { // operator new[](size_t[, nothrow_t]) and operator delete[](size_t[, nothrow_t]) void* operator new[](std::size_t s) TEST_THROW_SPEC(std::bad_alloc) { - void* ret = operator new[](s, std::nothrow); - if (ret == nullptr) + getGlobalMemCounter()->newArrayCalled(s); + void* p = std::malloc(s); + if (p == nullptr) detail::throw_bad_alloc_helper(); - return ret; + return p; } void* operator new[](std::size_t s, std::nothrow_t const&) TEST_NOEXCEPT { +# ifdef TEST_HAS_NO_EXCEPTIONS getGlobalMemCounter()->newArrayCalled(s); - return operator new(s); +# else + try { + getGlobalMemCounter()->newArrayCalled(s); + } catch (std::bad_alloc const&) { + return nullptr; + } +# endif + return std::malloc(s); } -void operator delete[](void* p) TEST_NOEXCEPT { operator delete[](p, std::nothrow); } +void operator delete[](void* p) TEST_NOEXCEPT { + getGlobalMemCounter()->deleteArrayCalled(p); + std::free(p); +} void operator delete[](void* p, std::nothrow_t const&) TEST_NOEXCEPT { getGlobalMemCounter()->deleteArrayCalled(p); - operator delete(p); + std::free(p); } # ifndef TEST_HAS_NO_ALIGNED_ALLOCATION @@ -420,60 +443,89 @@ void operator delete[](void* p, std::nothrow_t const&) TEST_NOEXCEPT { # define USE_ALIGNED_ALLOC # endif +inline void* alocate_aligned_impl(std::size_t size, std::align_val_t align) { + const std::size_t alignment = static_cast<std::size_t>(align); + void* ret = nullptr; +# ifdef USE_ALIGNED_ALLOC + ret = _aligned_malloc(size, alignment); +# else + assert(posix_memalign(&ret, std::max(alignment, sizeof(void*)), size) != EINVAL); +# endif + return ret; +} + +inline void free_aligned_impl(void* ptr, std::align_val_t) { + if (ptr) { +# ifdef USE_ALIGNED_ALLOC + ::_aligned_free(ptr); +# else + ::free(ptr); +# endif + } +} + // operator new(size_t, align_val_t[, nothrow_t]) and operator delete(size_t, align_val_t[, nothrow_t]) void* operator new(std::size_t s, std::align_val_t av) TEST_THROW_SPEC(std::bad_alloc) { - void* p = operator new(s, av, std::nothrow); + getGlobalMemCounter()->alignedNewCalled(s, static_cast<std::size_t>(av)); + void* p = alocate_aligned_impl(s, av); if (p == nullptr) detail::throw_bad_alloc_helper(); return p; } void* operator new(std::size_t s, std::align_val_t av, std::nothrow_t const&) TEST_NOEXCEPT { - const std::size_t a = static_cast<std::size_t>(av); - getGlobalMemCounter()->alignedNewCalled(s, a); - void* ret = nullptr; -# ifdef USE_ALIGNED_ALLOC - ret = _aligned_malloc(s, a); +# ifdef TEST_HAS_NO_EXCEPTIONS + getGlobalMemCounter()->alignedNewCalled(s, static_cast<std::size_t>(av)); # else - assert(posix_memalign(&ret, std::max(a, sizeof(void*)), s) != EINVAL); + try { + getGlobalMemCounter()->alignedNewCalled(s, static_cast<std::size_t>(av)); + } catch (std::bad_alloc const&) { + return nullptr; + } # endif - return ret; + return alocate_aligned_impl(s, av); } -void operator delete(void* p, std::align_val_t av) TEST_NOEXCEPT { operator delete(p, av, std::nothrow); } +void operator delete(void* p, std::align_val_t av) TEST_NOEXCEPT { + getGlobalMemCounter()->alignedDeleteCalled(p, static_cast<std::size_t>(av)); + free_aligned_impl(p, av); +} void operator delete(void* p, std::align_val_t av, std::nothrow_t const&) TEST_NOEXCEPT { - const std::size_t a = static_cast<std::size_t>(av); - getGlobalMemCounter()->alignedDeleteCalled(p, a); - if (p) { -# ifdef USE_ALIGNED_ALLOC - ::_aligned_free(p); -# else - ::free(p); -# endif - } + getGlobalMemCounter()->alignedDeleteCalled(p, static_cast<std::size_t>(av)); + free_aligned_impl(p, av); } // operator new[](size_t, align_val_t[, nothrow_t]) and operator delete[](size_t, align_val_t[, nothrow_t]) void* operator new[](std::size_t s, std::align_val_t av) TEST_THROW_SPEC(std::bad_alloc) { - void* ret = operator new[](s, av, std::nothrow); - if (ret == nullptr) + getGlobalMemCounter()->alignedNewArrayCalled(s, static_cast<std::size_t>(av)); + void* p = alocate_aligned_impl(s, av); + if (p == nullptr) detail::throw_bad_alloc_helper(); - return ret; + return p; } void* operator new[](std::size_t s, std::align_val_t av, std::nothrow_t const&) TEST_NOEXCEPT { - const std::size_t a = static_cast<std::size_t>(av); - getGlobalMemCounter()->alignedNewArrayCalled(s, a); - return operator new(s, av, std::nothrow); +# ifdef TEST_HAS_NO_EXCEPTIONS + getGlobalMemCounter()->alignedNewArrayCalled(s, static_cast<std::size_t>(av)); +# else + try { + getGlobalMemCounter()->alignedNewArrayCalled(s, static_cast<std::size_t>(av)); + } catch (std::bad_alloc const&) { + return nullptr; + } +# endif + return alocate_aligned_impl(s, av); } -void operator delete[](void* p, std::align_val_t av) TEST_NOEXCEPT { operator delete[](p, av, std::nothrow); } +void operator delete[](void* p, std::align_val_t av) TEST_NOEXCEPT { + getGlobalMemCounter()->alignedDeleteArrayCalled(p, static_cast<std::size_t>(av)); + free_aligned_impl(p, av); +} void operator delete[](void* p, std::align_val_t av, std::nothrow_t const&) TEST_NOEXCEPT { - const std::size_t a = static_cast<std::size_t>(av); - getGlobalMemCounter()->alignedDeleteArrayCalled(p, a); - return operator delete(p, av); + getGlobalMemCounter()->alignedDeleteArrayCalled(p, static_cast<std::size_t>(av)); + free_aligned_impl(p, av); } # endif // TEST_HAS_NO_ALIGNED_ALLOCATION >From e9863c390c900aff3f18b4c24e0aa4a8d8a01e08 Mon Sep 17 00:00:00 2001 From: Louis Dionne <ldionn...@gmail.com> Date: Mon, 18 Dec 2023 15:23:00 -0500 Subject: [PATCH 6/8] Fix formatting --- libcxx/test/support/check_assertion.h | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/libcxx/test/support/check_assertion.h b/libcxx/test/support/check_assertion.h index 34e41e8f0d8eaf8..9a092d75267db80 100644 --- a/libcxx/test/support/check_assertion.h +++ b/libcxx/test/support/check_assertion.h @@ -258,9 +258,7 @@ void std::__libcpp_verbose_abort(char const* format, ...) { std::exit(DeathTest::RK_Terminate); } -[[noreturn]] inline void abort_handler(int) { - std::exit(DeathTest::RK_Terminate); -} +[[noreturn]] inline void abort_handler(int) { std::exit(DeathTest::RK_Terminate); } template <class Func> inline bool ExpectDeath(const char* stmt, Func&& func, AssertionInfoMatcher Matcher) { >From 12e32398362272562cdc8377f7cfb52416773600 Mon Sep 17 00:00:00 2001 From: Louis Dionne <ldionn...@gmail.com> Date: Tue, 16 Jan 2024 15:48:47 -0500 Subject: [PATCH 7/8] Add XFAILs for a few tests that can't work anymore --- .../new.size_align_nothrow.replace.indirect.pass.cpp | 4 ++++ .../new.size_nothrow.replace.indirect.pass.cpp | 4 ++++ .../new.size_align_nothrow.replace.indirect.pass.cpp | 4 ++++ .../new.size_nothrow.replace.indirect.pass.cpp | 4 ++++ 4 files changed, 16 insertions(+) diff --git a/libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.array/new.size_align_nothrow.replace.indirect.pass.cpp b/libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.array/new.size_align_nothrow.replace.indirect.pass.cpp index df8a651932cef1b..f6959172ea24ef9 100644 --- a/libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.array/new.size_align_nothrow.replace.indirect.pass.cpp +++ b/libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.array/new.size_align_nothrow.replace.indirect.pass.cpp @@ -11,6 +11,10 @@ // Test that we can replace the operator by replacing `operator new[](std::size_t, std::align_val_t)` // (the throwing version). +// This doesn't work when the shared library was built with exceptions disabled, because +// we can't implement the non-throwing new from the throwing new in that case. +// XFAIL: no-exceptions + // UNSUPPORTED: c++03, c++11, c++14 // UNSUPPORTED: sanitizer-new-delete diff --git a/libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.array/new.size_nothrow.replace.indirect.pass.cpp b/libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.array/new.size_nothrow.replace.indirect.pass.cpp index 70d891b2a82c0a3..84bfe8205b6006c 100644 --- a/libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.array/new.size_nothrow.replace.indirect.pass.cpp +++ b/libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.array/new.size_nothrow.replace.indirect.pass.cpp @@ -10,6 +10,10 @@ // Test that we can replace the operator by replacing `operator new[](std::size_t)` (the throwing version). +// This doesn't work when the shared library was built with exceptions disabled, because +// we can't implement the non-throwing new from the throwing new in that case. +// XFAIL: no-exceptions + // UNSUPPORTED: sanitizer-new-delete // XFAIL: libcpp-no-vcruntime // XFAIL: LIBCXX-AIX-FIXME diff --git a/libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.single/new.size_align_nothrow.replace.indirect.pass.cpp b/libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.single/new.size_align_nothrow.replace.indirect.pass.cpp index a68cdab54528c27..2e7fa132890b808 100644 --- a/libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.single/new.size_align_nothrow.replace.indirect.pass.cpp +++ b/libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.single/new.size_align_nothrow.replace.indirect.pass.cpp @@ -10,6 +10,10 @@ // Test that we can replace the operator by replacing `operator new(std::size_t, std::align_val_t)` (the throwing version). +// This doesn't work when the shared library was built with exceptions disabled, because +// we can't implement the non-throwing new from the throwing new in that case. +// XFAIL: no-exceptions + // UNSUPPORTED: c++03, c++11, c++14 // UNSUPPORTED: sanitizer-new-delete diff --git a/libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.single/new.size_nothrow.replace.indirect.pass.cpp b/libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.single/new.size_nothrow.replace.indirect.pass.cpp index 64edbfd7e9af993..8b5019cf7eb63ea 100644 --- a/libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.single/new.size_nothrow.replace.indirect.pass.cpp +++ b/libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.single/new.size_nothrow.replace.indirect.pass.cpp @@ -10,6 +10,10 @@ // Test that we can replace the operator by replacing `operator new(std::size_t)` (the throwing version). +// This doesn't work when the shared library was built with exceptions disabled, because +// we can't implement the non-throwing new from the throwing new in that case. +// XFAIL: no-exceptions + // UNSUPPORTED: sanitizer-new-delete // XFAIL: libcpp-no-vcruntime // XFAIL: LIBCXX-AIX-FIXME >From cf6209af59f750f066a25cd2a0aed0ef395a6847 Mon Sep 17 00:00:00 2001 From: Louis Dionne <ldionn...@gmail.com> Date: Mon, 22 Jan 2024 13:52:52 -0500 Subject: [PATCH 8/8] Move to src/ --- libcxx/include/CMakeLists.txt | 1 - libcxx/include/module.modulemap.in | 4 ---- .../include/overridable_function.h} | 6 +++--- libcxx/src/new.cpp | 2 +- libcxxabi/src/stdlib_new_delete.cpp | 2 +- 5 files changed, 5 insertions(+), 10 deletions(-) rename libcxx/{include/__overridable_function => src/include/overridable_function.h} (97%) diff --git a/libcxx/include/CMakeLists.txt b/libcxx/include/CMakeLists.txt index 777ef9f950be34f..ed721d467e94f4c 100644 --- a/libcxx/include/CMakeLists.txt +++ b/libcxx/include/CMakeLists.txt @@ -573,7 +573,6 @@ set(files __numeric/transform_exclusive_scan.h __numeric/transform_inclusive_scan.h __numeric/transform_reduce.h - __overridable_function __random/bernoulli_distribution.h __random/binomial_distribution.h __random/cauchy_distribution.h diff --git a/libcxx/include/module.modulemap.in b/libcxx/include/module.modulemap.in index 071aadebca932db..194a74a1e07b145 100644 --- a/libcxx/include/module.modulemap.in +++ b/libcxx/include/module.modulemap.in @@ -606,10 +606,6 @@ module std_private_node_handle [system] { header "__node_handle" export * } -module std_private_overridable_function [system] { - header "__overridable_function" - export * -} module std_private_split_buffer [system] { header "__split_buffer" export * diff --git a/libcxx/include/__overridable_function b/libcxx/src/include/overridable_function.h similarity index 97% rename from libcxx/include/__overridable_function rename to libcxx/src/include/overridable_function.h index 81dd81bdc0ae8a2..7b0fba10f47d4aa 100644 --- a/libcxx/include/__overridable_function +++ b/libcxx/src/include/overridable_function.h @@ -7,8 +7,8 @@ // //===----------------------------------------------------------------------===// -#ifndef _LIBCPP___OVERRIDABLE_FUNCTION -#define _LIBCPP___OVERRIDABLE_FUNCTION +#ifndef _LIBCPP_SRC_INCLUDE_OVERRIDABLE_FUNCTION_H +#define _LIBCPP_SRC_INCLUDE_OVERRIDABLE_FUNCTION_H #include <__config> #include <cstdint> @@ -116,4 +116,4 @@ _LIBCPP_END_NAMESPACE_STD #endif -#endif // _LIBCPP___OVERRIDABLE_FUNCTION +#endif // _LIBCPP_SRC_INCLUDE_OVERRIDABLE_FUNCTION_H diff --git a/libcxx/src/new.cpp b/libcxx/src/new.cpp index 5ca9d6b43da443f..71a3e297e2cc351 100644 --- a/libcxx/src/new.cpp +++ b/libcxx/src/new.cpp @@ -6,8 +6,8 @@ // //===----------------------------------------------------------------------===// +#include "include/overridable_function.h" #include <__memory/aligned_alloc.h> -#include <__overridable_function> #include <cstddef> #include <cstdlib> #include <new> diff --git a/libcxxabi/src/stdlib_new_delete.cpp b/libcxxabi/src/stdlib_new_delete.cpp index cba6c3968775ab2..1ad69e7891503f9 100644 --- a/libcxxabi/src/stdlib_new_delete.cpp +++ b/libcxxabi/src/stdlib_new_delete.cpp @@ -8,8 +8,8 @@ #include "__cxxabi_config.h" #include "abort_message.h" +#include "include/overridable_function.h" // from libc++ #include <__memory/aligned_alloc.h> -#include <__overridable_function> #include <cstddef> #include <cstdlib> #include <new> _______________________________________________ lldb-commits mailing list lldb-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
