Hi Jakob,
Sorry, I should have looked at the code more carefully before sending my reply.
The lambda capture-by-copy of the variadic template arguments being buggy would
be one explanation, but it looks like it's something else.
I tried following workaround:
CodeRange* CodeRange::EnsureProcessWideCodeRange(
v8::PageAllocator* page_allocator, size_t requested_size) {
base::CallOnce(&init_code_range_once, [&]() {
InitProcessWideCodeRange(page_allocator, requested_size);
});
return process_wide_code_range_;
}
This uses the overload taking the `std::function<void()>` directly. It crashes
in `CallOnceImpl` at call of `init_func()`.
I also tried to call `CallOnceImpl` directly, same crash.
It looks like the `std::function` is corrupted in `CallOnceImpl`. It works when
supplying a static function with no arguments, but any lambda crashes. Using
simpler test code, it seems that it's linked to pass a `std::function` across
dll boundary. It works when calling an template/inline function, but calling a
function exported by the dll doesn't work (see [1]).
I can't see why passing a `std::function` to a dll call is problematic. I
tested the same test code on another project that uses MSVC compiler and it
works fine there. It's not impossible it's a clang issue. Unfortunately, this
looks too complicated for me to look further into (I'm not familiar with
assembler and such low level stuff).
I tried this ugly workaround since I can pass static functions, which allows
the compile to pass. d8.exe launches, but tests fail.
v8::PageAllocator* g_page_allocator;
size_t g_requested_size;
static void InitProcessWideCodeRangeNoArgs() {
InitProcessWideCodeRange(g_page_allocator, g_requested_size);
}
// static
CodeRange* CodeRange::EnsureProcessWideCodeRange(
v8::PageAllocator* page_allocator, size_t requested_size) {
static base::Mutex mx;
base::MutexGuard guard(&mx);
g_page_allocator = page_allocator;
g_requested_size = requested_size;
base::CallOnce(&init_code_range_once, InitProcessWideCodeRangeNoArgs);
return process_wide_code_range_;
}
Regards,
Jean-Claude
-------------------
[1] Test to reproduce the issue of passing a std::function across dll
boundaries:
src/base/once.h
inline void Test1(std::function<void()> init_func) {
printf("Test1 before init_func\n");
init_func();
printf("Test1 after init_func\n");
}
V8_BASE_EXPORT void Test2(std::function<void()> init_func);
src/base/once.cc
void Test2(std::function<void()> init_func) {
printf("Test2 before init_func\n");
init_func();
printf("Test2 after init_func\n");
}
mksnapshot.cc
void TestFunction() {
printf(" called TestFunction\n");
}
int main(int argc, char** argv) {
v8::base::Test1([]() {
printf(" called CallOnceFake1 lambda\n");
});
v8::base::Test2(TestFunction);
v8::base::Test2([]() {
printf(" called CallOnceFake2 lambda\n");
});
The output is
> Test1 before init_func
> called CallOnceFake1 lambda
> Test1 after init_func
> Test2 before init_func
> called TestFunction
> Test2 after init_func
> Test2 before init_func
The call to `Test2` with a lambda crashes
On Wed, Aug 9, 2023, at 10:32 AM, Jakob Gruber wrote:
>
>
> On Wed, Aug 9, 2023 at 10:05 AM Jean-Claude Monnin <jc_mon...@emailplus.org>
> wrote:
>> __
>> I've tried to figure out a bit more what is going on by adding prints along
>> the call stack. It looks like it's the `base::CallOnce` in `code-range.cc`
>> introduced in commit 26bc8bb4 that is the problem. Here the code with the
>> added prints:
>> V8_DECLARE_ONCE(init_code_range_once);
>> void InitProcessWideCodeRange(v8::PageAllocator* page_allocator,
>> size_t requested_size) {
>> i::PrintF(stdout, "InitProcessWideCodeRange %u\n", requested_size);
>> CodeRange* code_range = new CodeRange();
>> if (!code_range->InitReservation(page_allocator, requested_size)) {
>> V8::FatalProcessOutOfMemory(
>> nullptr, "Failed to reserve virtual memory for CodeRange");
>> }
>> process_wide_code_range_ = code_range;
>> #ifdef V8_EXTERNAL_CODE_SPACE
>> #ifdef V8_COMPRESS_POINTERS_IN_SHARED_CAGE
>> ExternalCodeCompressionScheme::InitBase(
>> ExternalCodeCompressionScheme::PrepareCageBaseAddress(
>> code_range->base()));
>> #endif // V8_COMPRESS_POINTERS_IN_SHARED_CAGE
>> #endif // V8_EXTERNAL_CODE_SPACE
>> }
>> } // namespace
>>
>> // static
>> CodeRange* CodeRange::EnsureProcessWideCodeRange(
>> v8::PageAllocator* page_allocator, size_t requested_size) {
>> i::PrintF(stdout, "CodeRange::EnsureProcessWideCodeRange %u\n",
>> requested_size);
>> base::CallOnce(&init_code_range_once, InitProcessWideCodeRange,
>> page_allocator, requested_size);
>> return process_wide_code_range_;
>> }
>>
>> It outputs:
>> CodeRange::EnsureProcessWideCodeRange 536870912
>> InitProcessWideCodeRange 2034756544 <tel:(203)%20475-6544>
>>
>> It looks like the `requested_size` isn't forwarded correctly in
>> `base::CallOnce`.
>> I'm not sure to understand the CallOnce implementation, but I wonder if
>> calling `std::function<void()>` with `init_func(args...)` isn't undefined
>> behavior. Not sure how to fix/work around.
>
> Which part would be undefined behavior? From a quick glance, the CallOnce
> implementation
> <https://source.chromium.org/chromium/chromium/src/+/refs/heads/main:v8/src/base/once.h;l=101;drc=a9163a67bbe920dfcfa6286bf0d172a1af377dba>
> looks reasonable to me. It sounds like something around the lambda
> capture-by-copy is buggy (accessing the wrong stack slot?). Perhaps you can
> verify with disassembly of InitProcessWideCodeRange, or step through and find
> out what it's actually copying.
>
>>
>>
>> On Wed, Aug 9, 2023, at 8:03 AM, Jakob Gruber wrote:
>>>
>>>
>>> On Tue, Aug 8, 2023 at 3:20 PM Jean-Claude Monnin <jc_mon...@emailplus.org>
>>> wrote:
>>>> __
>>>> Hi Jakob,
>>>>
>>>> Thanks for your reply.
>>>> It looks like using Microsoft's C++ library instead of libc++ is somewhat
>>>> exotic for v8. Unfortunately there are cases where it's almost impossible
>>>> to switch to libc++.
>>>>
>>>> Since I have a chance to get some feedback here of how to address this
>>>> issue, I'm going to try to give you as much info as possible.
>>>>
>>>> `mksnapshot.exe` aborts at `VirtualMemoryCage::InitReservation` at
>>>> following check:
>>>> CHECK(IsAligned(params.reservation_size, allocate_page_size));
>>>>
>>>> When adding following print on the line before
>>>> i::PrintF(stdout, "VirtualMemoryCage::InitReservation %u %u\n",
>>>> params.reservation_size, allocate_page_size);
>>>> it prints
>>>> VirtualMemoryCage::InitReservation 3356617664 65536
>>>> It looks like the supplied `params.reservation_size` is not aligned.
>>>
>>> Thanks for the investigation, very helpful. I wonder where that
>>> reservation_size comes from. It doesn't look like any value we'd set in V8.
>>> Corrupted? Uninitialized?
>>>
>>> I'd expect it to be set by mksnapshot here
>>> <https://source.chromium.org/chromium/chromium/src/+/refs/heads/main:v8/src/snapshot/mksnapshot.cc;l=282;drc=a9163a67bbe920dfcfa6286bf0d172a1af377dba>
>>> and picked up by isolate initialization here
>>> <https://source.chromium.org/chromium/chromium/src/+/refs/heads/main:v8/src/heap/heap.cc;l=5420;drc=a9163a67bbe920dfcfa6286bf0d172a1af377dba>.
>>> There it should either be some reasonable aligned value, or 0 and we'd
>>> fall back to kMaximalCodeRangeSize.
>>>
>>>>
>>>>
>>>> Full call stack is included in screenshot below (sorry for the screenshot,
>>>> I couldn't find a way to copy text from WinDbg)
>>>>
>>>> I'm happy to investigate further, but wanted to send this out in case
>>>> there is anything specific that would be helpful.
>>>>
>>>> Jean-Claude
>>>>
>>>> On Tue, Aug 8, 2023, at 12:13 PM, Jakob Gruber wrote:
>>>>> Hi Jean-Claude,
>>>>>
>>>>> no, we don't have a lot of test coverage for `use_custom_libcxx=false`,
>>>>> this mode is only supported on a best-effort basis.
>>>>>
>>>>> For debugging: a backtrace and symbols would be useful. Does running
>>>>> `mksnapshot` in a debugger give more infos? Also, a bisect to find the
>>>>> culprit change would be very helpful.
>>>>>
>>>>> On Thu, Aug 3, 2023 at 6:54 PM Jean-Claude Monnin
>>>>> <jc_mon...@emailplus.org> wrote:
>>>>>> Hi,
>>>>>>
>>>>>> On windows, the v8 version 11.5 build fails when generating the snapshot
>>>>>> with following error:
>>>>>>
>>>>>> C:/Users/jean-claude/Documents/src/google/depot_tools/bootstrap-2@3_8_10_chromium_26_bin/python3/bin/python3.exe
>>>>>> ../../tools/run.py ./mksnapshot --turbo_instruction_scheduling
>>>>>> --target_os=win --target_arch=x64 --embedded_src gen/embedded.S
>>>>>> --embedded_variant Default --random-seed 314159265 --startup_blob
>>>>>> snapshot_blob.bin --no-native-code-counters
>>>>>> Return code is 2147483651 <tel:(214)%20748-3651>
>>>>>>
>>>>>> These are the options used (args.gn):
>>>>>> is_official_build = true
>>>>>> target_cpu = "x64"
>>>>>> is_component_build = true
>>>>>> use_custom_libcxx = false
>>>>>> chrome_pgo_phase = false
>>>>>> treat_warnings_as_errors = false
>>>>>> fatal_linker_warnings = false
>>>>>> symbol_level = 0
>>>>>>
>>>>>> When using `is_debug=false` instead of `is_official_build = true` it
>>>>>> builds fine, but it comes with performance regressions compared to older
>>>>>> version 9.3 build with `is_official_build = true`.
>>>>>>
>>>>>> If using either `is_component_build = false` or `use_custom_libcxx =
>>>>>> true`, it builds fine too, however it's not really an option as I need a
>>>>>> dll build and I need to use Microsoft's C++ standard library because
>>>>>> third party dependencies prevents us to use libc++.
>>>>>>
>>>>>> I also tried version 11.4 and 11.6 and they give the same error.
>>>>>>
>>>>>> Any hints in how to diagnose/fix that would be appreciated.
>>>>>>
>>>>>> Auxiliary question: Is any big project using `use_custom_libcxx = false`
>>>>>> (eg. Microsoft's C++ standard library), or is this untested?
>>>>>> Chrome/node/deno all use libc++?
>>>>>>
>>>>>> Best regards,
>>>>>> Jean-Claude
>>>>>>
>>>>>> --
>>>>>> --
>>>>>> v8-users mailing list
>>>>>> v8-users@googlegroups.com
>>>>>> http://groups.google.com/group/v8-users
>>>>>> ---
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "v8-users" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>>> an email to v8-users+unsubscr...@googlegroups.com
>>>>>> <mailto:v8-users%2bunsubscr...@googlegroups.com>.
>>>>>> To view this discussion on the web visit
>>>>>> https://groups.google.com/d/msgid/v8-users/984f3518-4b8e-4403-b794-923be66ccf08%40app.fastmail.com.
>>>>>
>>>>>
>>>>> --
>>>>> --
>>>>> v8-users mailing list
>>>>> v8-users@googlegroups.com
>>>>> http://groups.google.com/group/v8-users
>>>>> ---
>>>>> You received this message because you are subscribed to the Google Groups
>>>>> "v8-users" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send an
>>>>> email to v8-users+unsubscr...@googlegroups.com.
>>>>> To view this discussion on the web visit
>>>>> https://groups.google.com/d/msgid/v8-users/CAH3p7oO%2BEkjhuGaa9nXVWHd9Da3W4h0mr3ychCWu4J3gqRut8Q%40mail.gmail.com
>>>>>
>>>>> <https://groups.google.com/d/msgid/v8-users/CAH3p7oO%2BEkjhuGaa9nXVWHd9Da3W4h0mr3ychCWu4J3gqRut8Q%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>>>>
>>>>
>>>>
>>>> --
>>>> --
>>>> v8-users mailing list
>>>> v8-users@googlegroups.com
>>>> http://groups.google.com/group/v8-users
>>>> ---
>>>> You received this message because you are subscribed to the Google Groups
>>>> "v8-users" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send an
>>>> email to v8-users+unsubscr...@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/v8-users/7d4484d9-13bd-4f86-be64-79b12abc0ec9%40app.fastmail.com
>>>>
>>>> <https://groups.google.com/d/msgid/v8-users/7d4484d9-13bd-4f86-be64-79b12abc0ec9%40app.fastmail.com?utm_medium=email&utm_source=footer>.
>>>
>>>
>>> --
>>> --
>>> v8-users mailing list
>>> v8-users@googlegroups.com
>>> http://groups.google.com/group/v8-users
>>> ---
>>> You received this message because you are subscribed to the Google Groups
>>> "v8-users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to v8-users+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/v8-users/CAH3p7oOHbcZqCXLDL9m2Eyw-%3DGJLQJCac22W%3Dnf0R2xaX7z-GA%40mail.gmail.com
>>>
>>> <https://groups.google.com/d/msgid/v8-users/CAH3p7oOHbcZqCXLDL9m2Eyw-%3DGJLQJCac22W%3Dnf0R2xaX7z-GA%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>>
>>
>>
>> --
>> --
>> v8-users mailing list
>> v8-users@googlegroups.com
>> http://groups.google.com/group/v8-users
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "v8-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to v8-users+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/v8-users/1ac31ebe-76de-4872-9ad7-32e0c6baf81a%40app.fastmail.com
>>
>> <https://groups.google.com/d/msgid/v8-users/1ac31ebe-76de-4872-9ad7-32e0c6baf81a%40app.fastmail.com?utm_medium=email&utm_source=footer>.
>
>
> --
> --
> v8-users mailing list
> v8-users@googlegroups.com
> http://groups.google.com/group/v8-users
> ---
> You received this message because you are subscribed to the Google Groups
> "v8-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to v8-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/v8-users/CAH3p7oPGApgXG-Qf6u8q56tD%3D7ZFThDQfjsPSW2n%3DiTgiOBH2g%40mail.gmail.com
>
> <https://groups.google.com/d/msgid/v8-users/CAH3p7oPGApgXG-Qf6u8q56tD%3D7ZFThDQfjsPSW2n%3DiTgiOBH2g%40mail.gmail.com?utm_medium=email&utm_source=footer>.
--
--
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
---
You received this message because you are subscribed to the Google Groups
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to v8-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/v8-users/6e962428-87a3-4408-b491-16029338c13a%40app.fastmail.com.
