Re: Resolving LTO symbols for static library boundary
On Dienstag, 6. Februar 2018 01:01:29 CET Jan Hubicka wrote: > Dne 2018-02-05 18:44, Richard Biener napsal: > > On February 5, 2018 12:26:58 PM GMT+01:00, Allan Sandfeld Jensen > > > > wrote: > >> Hello GCC > >> > >> In trying to make it possible to use LTO for distro-builds of Qt, I > >> have again > >> hit the problem of static libraries. In Qt in general we for LTO rely > >> on a > >> library boundary, where LTO gets resolved when generating the library > >> but no > >> LTO-symbols are exported in the shared library. This ensure the > >> library > >> has a > >> well defined binary compatible interface and gets LTO optimizations > >> within > >> each library. For some private libraries we use static libraries > >> however, > >> because we don't need binary compatibility, but though we don't need > >> BC > >> > >> between Qt versions, the libraries should still be linkable with > >> different gcc > >> versions (and with different compilers). However when LTO is enabled > >> the > >> static libraries will contain definitions that depend on a single gcc > >> version > >> making it unsuitable for distribution. > >> > >> One solution is to enable fat-lto object files for static libraries > >> but > >> that > >> is both a waste of space and compile time, and disables any LTO > >> optimization > >> within the library. Ideally I would like to have the static library do > >> LTO > >> optimizations internally just like a shared library, but then exported > >> as > >> static library. > >> > >> I suspect this is more of gcc task than ar/ld task, as it basically > >> boils down > >> to gcc doing for a static library what it does for shared library, but > >> maybe > >> export the result as a single combined .o file, that can then be ar'ed > >> into a > >> compatible static library. > >> > >> Is this possible? > > > > Hmm. I think you could partially link the static archive contents into > > a single relocatable object. Or we could add a mode where you do a > > 1to1 LTO link of the objects and stop at the ltrans object files. You > > could stuff those into an archive again. > > > > I'm not sure how far Honza got partial LTO linking to work? > > Parital linking of lto .o files into single non-lto .o file should work > and it will get you cross-module optimization done. The problem is that > without resolution info compiler needs to assume that all symbols > exported by object files are possibly referneced by the later > incremental link and thus the code quality will definitly not be > comparable with what you get for LTO on final binary or DSO. Still > should be better than non-lto build. > I would be curious if it is useful for you in practice. > How would I do that partial link, and what are the requirements? Best regards 'Allan
Re: gcc 7.3: Replacing global operator new/delete in shared libraries
On Tue, 6 Feb 2018, Paul Smith wrote: My environment has been using GCC 6.2 (locally compiled) on GNU/Linux systems. We use a separate heap management library (jemalloc) rather than the libc allocator. The way we did this in the past was to declare operator new/delete (all forms) as inline functions in a header Are you sure you still have all forms? The aligned versions were added in gcc-7 IIRC. and ensure that this header was always the very first thing in every source file, before even any standard header files. I know that inline operator new/delete isn't OK in the C++ standard, but in fact it has worked for us on the systems we care about. Inline usually works, but violating the ODR is harder... I would at least use the always_inline attribute to improve chances (I assume that static (or anonymous namespace) versions wouldn't work), since the optimizer may decide not to inline otherwise. Something based on visibility should be somewhat safer. But it still seems dangerous, some global libstdc++ object might be initialized using one allocator then used with another one... I'm attempting a toolchain upgrade which is switching to GCC 7.3 / binutils 2.30 (along with many other updates). Now when I run our code, I get a core on exit. It appears an STL container delete is invoking libc free() with a pointer to memory allocated by jemalloc. An example would help the discussion. My question is, what do I need to do to ensure this behavior persists if I create a global operator new/delete? Is it sufficient to ensure that the symbol for our shared library global new/delete symbols are hidden and not global, using a linker map or -fvisibility=hidden? I think so (hidden implies not-interposable, so locally bound), but I don't have much experience there. -- Marc Glisse
Re: GCC 8.0.0 status on x86_64-w64-mingw32, some issues
Am 06.02.2018 um 22:50 schrieb Eric Botcazou: >> Here's a short status report for trunk on x86_64-w64-mingw32 host. >> >> I know this is only a secondary platform, but there are some serius issues. >> >> Especially the ada part is in a bad shape compared to 7.3.0, see >> https://gcc.gnu.org/ml/gcc-testresults/2018-02/msg00439.html >> and >> https://gcc.gnu.org/ml/gcc-testresults/2018-02/msg00308.html >> >> At least 20 of the acats tests catch all memory until the host memory is >> exhausted. Same holds for the two libstdc++ tests >> 23_containers/unordered_set/requirements/exception/basic.cc >> and >> 23_containers/unordered_set/requirements/exception/propagation_consistent.cc > > Does passing -fno-reorder-blocks-and-partition change anything? > Indeed, this solves most of the new failures. Here is the acats test summary: === acats Summary === # of expected passes2298 # of unexpected failures22 *** FAILURES: c23003b c23003g c23003i c250002 c380004 cd2b11a cd2b15c ce2102l ce2102m ce2103a ce2103b ce3102d ce3107a ce3115a cxa4005 cxa4008 cxa4016 cxa4019 cxac003 cxb3012 cxf3a01 cxf3a02 /opt/devel/gnu/src/gcc-mingw-w64/gcc-8.0.0/gcc/testsuite/ada/acats/run_all.sh completed at Wed Feb 7 12:28:36 CET 2018 I may mail the log, if it's of any help. signature.asc Description: OpenPGP digital signature
Re: Copyright Assignment
On 05/02/18 22:01 +0100, Tim van Deurzen wrote: Hi, I've written to this list previously to mention I'm working on implementing p0515 (the spaceship operator) for C++. Although I'm still far from finished I'd like to make sure that when I am, I will be able to contribute my changes to GCC. Please tell me what I should do to take care of the copyright assignment. Thank you! Tim. Sent offlist.
Re: Copyright Assignment
On 07/02/18 12:16 +, Jonathan Wakely wrote: On 05/02/18 22:01 +0100, Tim van Deurzen wrote: Hi, I've written to this list previously to mention I'm working on implementing p0515 (the spaceship operator) for C++. Although I'm still far from finished I'd like to make sure that when I am, I will be able to contribute my changes to GCC. Please tell me what I should do to take care of the copyright assignment. Thank you! Tim. Sent offlist. Although I'm getting SPF failures from mail.kompiler.org so I don't know if the mail got through.
Re: gdb 8.x - g++ 7.x compatibility
On 2018-02-07 02:21, Daniel Berlin wrote: As the person who, eons ago, wrote a bunch of the the GDB code for this C++ ABI support, and as someone who helped with DWARF support in both GDB and GCC, let me try to propose a useful path forward (in the hopes that someone will say "that's horrible, do it this instead") Here are the constraints i believe we are working with. 1. GDB should work with multiple DWARF producers and multiple C++ compilers implementing the C++ ABI 2. There is no canonical demangled format for the C++ ABI 3. There is no canoncial target demangler you can say everyone should use (and even if there was, you don't want to avoid debugging working because someone chose not to) 4. You don't want to slow down GDB if you can avoid it 5. Despite them all implementation the same ABI, it's still possible to distinguish the producers by the producer/compiler in the dwarf info. Given all that: GDB has ABI hooks that tell it what to do for various C++ ABIs. This is how it knows to call the right demangler for gcc v3's abi vs gcc v2's abi. and handle various differences between them. See gdb/cp-abi.h The IMHO, obvious thing to do here is: Handle the resulting demangler differences with 1 or more new C++ ABI hooks. Or, introduce C++ debuginfo producer hooks that the C++ ABI hooks use if folks want it to be separate. Once the producer is detected, fill in the hooks with a set of functions that does the right thing. I imagine this would also clean up a bundle of hacks in various parts of gdb trying to handle these differences anyway (which is where a lot of the multiple symbol lookups/etc that are often slow come from. If we just detected and said "this is gcc 6, it behaves like this", we wouldn't need to do that) In case you are worried, you will discover this is how a bunch of stuff is done and already contains a ball of hacks. Using hooks would be, IMHO, a significant improvement. Hi Daniel, Thanks for chiming in. This addresses the issue of how to do good software design in GDB to support different producers cleanly, but I think we have some issues even before that, like how to support g++ 7.3 and up. I'll try to summarize the issue quickly. It's now possible to end up with two templated classes with the same name that differ only by the signedness of their non-type template parameter. One is Foo and the other is Foo (the 10 is unsigned). Until 7.3, g++ would generate names like Foo<10> for the former and names like Foo<10u> for the later (in the DW_AT_name attribute of the classes' DIEs). Since 7.3, it produces Foo<10> for both. When GDB wants to know the run time type of an object, it fetches the pointer to its vtable, does a symbol lookup to get the linkage name and demangles it, which gives a string like "vtable for Foo<10>" or "vtable for Foo<10u>". It strips the "vtable for " and uses the remainder to do a type lookup. Since g++ 7.3, you can see that doing a type lookup for Foo<10> may find the wrong type, and doing a lookup for Foo<10u> won't find anything. So the problem here is how to uniquely identify those two classes when we are doing this run-time type finding operation (and probably in other cases too). Simon
Re: gcc 7.3: Replacing global operator new/delete in shared libraries
On Wed, 2018-02-07 at 11:32 +0100, Marc Glisse wrote: > On Tue, 6 Feb 2018, Paul Smith wrote: > > > My environment has been using GCC 6.2 (locally compiled) on > > GNU/Linux systems. We use a separate heap management library > > (jemalloc) rather than the libc allocator. The way we did this in > > the past was to declare operator new/delete (all forms) as inline > > functions in a header > > Are you sure you still have all forms? The aligned versions were > added in gcc-7 IIRC. Hm. I didn't realize there were new forms in C++17; I had the throw and no-throw versions and also sized delete. No, I didn't create the new ones. I'll do that. > > and ensure that this header was always the very first thing in > > every source file, before even any standard header files. I know > > that inline operator new/delete isn't OK in the C++ standard, but > > in fact it has worked for us on the systems we care about. > > Inline usually works, but violating the ODR is harder... I would at > least use the always_inline attribute to improve chances (I assume > that static (or anonymous namespace) versions wouldn't work) I definitely do that already; for example: inline __attribute__((__always_inline__)) void* operator new(size_t size) ... > > Now when I run our code, I get a core on exit. It appears an STL > > container delete is invoking libc free() with a pointer to memory > > allocated by jemalloc. > > An example would help the discussion. Good point. Here's an example (from a unit test crash, in GTest): *** Error in `EncodedStreamTests.gtest': free(): invalid pointer: 0x7481d0a0 *** Program received signal SIGABRT, Aborted. 0x74c62c37 in raise () from /lib/x86_64-linux-gnu/libc.so.6 (gdb) bt #0 0x74c62c37 in raise () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x74c66028 in abort () from /lib/x86_64-linux-gnu/libc.so.6 #2 0x74c9f2a4 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #3 0x74cab82e in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #4 0x00539b89 in __gnu_cxx::new_allocator::deallocate (this=, __p=) at /work/src/build/x86_64- linux/cc/generic/x86_64-generic-linux- gnu/include/c++/7.3.0/ext/new_allocator.h:125 #5 std::allocator_traits >::deallocate (__a=..., __n=, __p=) at /work/src/build/x86_64- linux/cc/generic/x86_64-generic-linux- gnu/include/c++/7.3.0/bits/alloc_traits.h:462 #6 std::__cxx11::basic_string, std::allocator >::_M_destroy (__size=, this=0x7fffe4e0) at /work/src/build/x86_64-linux/cc/generic/x86_64- generic-linux-gnu/include/c++/7.3.0/bits/basic_string.h:226 #7 std::__cxx11::basic_string, std::allocator >::_M_dispose (this=0x7fffe4e0) at /work/src/build/x86_64-linux/cc/generic/x86_64-generic-linux- gnu/include/c++/7.3.0/bits/basic_string.h:221 #8 std::__cxx11::basic_string, std::allocator >::~basic_string (this=0x7fffe4e0, __in_chrg=) at /work/src/build/x86_64- linux/cc/generic/x86_64-generic-linux- gnu/include/c++/7.3.0/bits/basic_string.h:647 #9 testing::internal::CodeLocation::~CodeLocation (this=0x7fffe4e0, __in_chrg=) at Tests/GTest/include/gtest/internal/gtest-internal.h:504 #10 testing::internal::MakeAndRegisterTestInfo (test_case_name=test_cas e_name@entry=0x5d8ed7 "ESTests", name=name@entry=0x5d8ec9 "bytesForCount", type_param=type_param@entry=0x0, value_param=value_par am@entry=0x0, code_location=..., fixture_class_id=fixture_class_id@entr y=0x65dd5c ::dummy_>, set_up_tc=0x48e010 , tear_down_tc=0x48e020 , factory=0x7481c000) at Tests/GTest/src/gtest.cc:2580 #11 0x00475286 in __static_initialization_and_destruction_0 (__priority=65535, __initialize_p=1) at EncodedStreamTests.cpp:351 #12 0x005d8dcd in __libc_csu_init () #13 0x74c4ded5 in __libc_start_main () from /lib/x86_64-linux- gnu/libc.so.6 #14 0x00477375 in _start () > > My question is, what do I need to do to ensure this behavior > > persists if I create a global operator new/delete? > > > > Is it sufficient to ensure that the symbol for our shared library > > global new/delete symbols are hidden and not global, using a linker > > map or -fvisibility=hidden? > > I think so (hidden implies not-interposable, so locally bound), but > I don't have much experience there. OK I'll pursue this for now. Thanks!
Re: gdb 8.x - g++ 7.x compatibility
On 02/07/2018 02:44 PM, Simon Marchi wrote: On 2018-02-07 02:21, Daniel Berlin wrote: As the person who, eons ago, wrote a bunch of the the GDB code for this C++ ABI support, and as someone who helped with DWARF support in both GDB and GCC, let me try to propose a useful path forward (in the hopes that someone will say "that's horrible, do it this instead") Here are the constraints i believe we are working with. 1. GDB should work with multiple DWARF producers and multiple C++ compilers implementing the C++ ABI 2. There is no canonical demangled format for the C++ ABI 3. There is no canoncial target demangler you can say everyone should use (and even if there was, you don't want to avoid debugging working because someone chose not to) 4. You don't want to slow down GDB if you can avoid it 5. Despite them all implementation the same ABI, it's still possible to distinguish the producers by the producer/compiler in the dwarf info. Given all that: GDB has ABI hooks that tell it what to do for various C++ ABIs. This is how it knows to call the right demangler for gcc v3's abi vs gcc v2's abi. and handle various differences between them. See gdb/cp-abi.h The IMHO, obvious thing to do here is: Handle the resulting demangler differences with 1 or more new C++ ABI hooks. Or, introduce C++ debuginfo producer hooks that the C++ ABI hooks use if folks want it to be separate. Once the producer is detected, fill in the hooks with a set of functions that does the right thing. I imagine this would also clean up a bundle of hacks in various parts of gdb trying to handle these differences anyway (which is where a lot of the multiple symbol lookups/etc that are often slow come from. If we just detected and said "this is gcc 6, it behaves like this", we wouldn't need to do that) In case you are worried, you will discover this is how a bunch of stuff is done and already contains a ball of hacks. Using hooks would be, IMHO, a significant improvement. Hi Daniel, Thanks for chiming in. This addresses the issue of how to do good software design in GDB to support different producers cleanly, but I think we have some issues even before that, like how to support g++ 7.3 and up. I'll try to summarize the issue quickly. It's now possible to end up with two templated classes with the same name that differ only by the signedness of their non-type template parameter. One is Foo and the other is Foo (the 10 is unsigned). Until 7.3, g++ would generate names like Foo<10> for the former and names like Foo<10u> for the later (in the DW_AT_name attribute of the classes' DIEs). Since 7.3, it produces Foo<10> for both. When GDB wants to know the run time type of an object, it fetches the pointer to its vtable, does a symbol lookup to get the linkage name and demangles it, which gives a string like "vtable for Foo<10>" or "vtable for Foo<10u>". It strips the "vtable for " and uses the remainder to do a type lookup. Since g++ 7.3, you can see that doing a type lookup for Foo<10> may find the wrong type, and doing a lookup for Foo<10u> won't find anything. So the problem here is how to uniquely identify those two classes when we are doing this run-time type finding operation (and probably in other cases too). Simon Hi all, In the perspective of "type identity", the way I see it the issue has a few parts: 1) How GCC compiles such templates 2) How GCC emits debugging information via -g 3) How such information is interpreted (and merged with the compiled code) by GDB Regarding 1) and 2), IMHO I think that there should be a one-to-one relationship between the compiled code output and debug info: This means that if GCC compiles such templates into two different classes[1], it should generate two different type identifiers. Conversely, if it compiles the templates into the same class, then a single identifier should be emitted for the single class compiled. (This goes besides the point of what the standard dictates[2]) If I understand it right, currently the issue is that gcc emits two types with the same debug identifier. Regarding 3), I think that after 1) and 2) are set up, GDB should be able to find the correct type definition (using the most appropriate design choice). Hope this helps, Manfred [1] According to the findings of Simon, this appears to be the case with clang, older GCC, and current GCC master. Do I understand this right? [2] About handling both templates instantiation as a single class, I think that if GCC wants to emit a single class, then its argument type instantiation should be well-definined,i.e. independent of the order of declaration - see the findings from Simon earlier in this thread where you could get the program output either -10 or 4294967286 depending on which declaration would come first.
Re: gdb 8.x - g++ 7.x compatibility
On 7 February 2018 at 15:07, Manfred wrote: > > > On 02/07/2018 02:44 PM, Simon Marchi wrote: >> >> On 2018-02-07 02:21, Daniel Berlin wrote: >>> >>> As the person who, eons ago, wrote a bunch of the the GDB code for this >>> C++ >>> ABI support, and as someone who helped with DWARF support in both GDB and >>> GCC, let me try to propose a useful path forward (in the hopes that >>> someone >>> will say "that's horrible, do it this instead") >>> >>> Here are the constraints i believe we are working with. >>> >>> 1. GDB should work with multiple DWARF producers and multiple C++ >>> compilers >>> implementing the C++ ABI >>> 2. There is no canonical demangled format for the C++ ABI >>> 3. There is no canoncial target demangler you can say everyone should use >>> (and even if there was, you don't want to avoid debugging working because >>> someone chose not to) >>> 4. You don't want to slow down GDB if you can avoid it >>> 5. Despite them all implementation the same ABI, it's still possible to >>> distinguish the producers by the producer/compiler in the dwarf info. >>> >>> Given all that: >>> >>> GDB has ABI hooks that tell it what to do for various C++ ABIs. This is >>> how >>> it knows to call the right demangler for gcc v3's abi vs gcc v2's abi. >>> and >>> handle various differences between them. >>> >>> See gdb/cp-abi.h >>> >>> The IMHO, obvious thing to do here is: Handle the resulting demangler >>> differences with 1 or more new C++ ABI hooks. >>> Or, introduce C++ debuginfo producer hooks that the C++ ABI hooks use if >>> folks want it to be separate. >>> >>> Once the producer is detected, fill in the hooks with a set of functions >>> that does the right thing. >>> >>> I imagine this would also clean up a bundle of hacks in various parts of >>> gdb trying to handle these differences anyway (which is where a lot of >>> the >>> multiple symbol lookups/etc that are often slow come from. >>> If we just detected and said "this is gcc 6, it behaves like this", we >>> wouldn't need to do that) >>> >>> In case you are worried, you will discover this is how a bunch of stuff >>> is >>> done and already contains a ball of hacks. >>> >>> Using hooks would be, IMHO, a significant improvement. >> >> >> Hi Daniel, >> >> Thanks for chiming in. >> >> This addresses the issue of how to do good software design in GDB to >> support different producers cleanly, but I think we have some issues even >> before that, like how to support g++ 7.3 and up. I'll try to summarize the >> issue quickly. It's now possible to end up with two templated classes with >> the same name that differ only by the signedness of their non-type template >> parameter. One is Foo and the other is Foo (the 10 >> is unsigned). Until 7.3, g++ would generate names like Foo<10> for the >> former and names like Foo<10u> for the later (in the DW_AT_name attribute of >> the classes' DIEs). Since 7.3, it produces Foo<10> for both. >> >> When GDB wants to know the run time type of an object, it fetches the >> pointer to its vtable, does a symbol lookup to get the linkage name and >> demangles it, which gives a string like "vtable for Foo<10>" or "vtable for >> Foo<10u>". It strips the "vtable for " and uses the remainder to do a type >> lookup. Since g++ 7.3, you can see that doing a type lookup for Foo<10> may >> find the wrong type, and doing a lookup for Foo<10u> won't find anything. >> >> So the problem here is how to uniquely identify those two classes when we >> are doing this run-time type finding operation (and probably in other cases >> too). >> >> Simon > > > Hi all, > > In the perspective of "type identity", the way I see it the issue has a few > parts: > > 1) How GCC compiles such templates > 2) How GCC emits debugging information via -g > 3) How such information is interpreted (and merged with the compiled code) > by GDB > > Regarding 1) and 2), IMHO I think that there should be a one-to-one > relationship between the compiled code output and debug info: > > This means that if GCC compiles such templates into two different > classes[1], it should generate two different type identifiers. What do you mean by "such templates"? There have been several different examples in the thread, which should be handled differently. > Conversely, if it compiles the templates into the same class, then a single > identifier should be emitted for the single class compiled. > (This goes besides the point of what the standard dictates[2]) > > If I understand it right, currently the issue is that gcc emits two types > with the same debug identifier. > > Regarding 3), I think that after 1) and 2) are set up, GDB should be able to > find the correct type definition (using the most appropriate design choice). > > Hope this helps, Not really :-) You're basically just saying "GCC and GDB should do the right thing" which is a statement of the obvious. > [1] According to the findings of Simon, this appears to be the case with > clang, older GCC, and current GCC
Re: gdb 8.x - g++ 7.x compatibility
On 4 February 2018 at 05:01, Simon Marchi wrote: > On 2018-02-03 13:35, Manfred wrote: >> n4659 17.4 (Type equivalence) p1.3: >> >> Two template-ids refer to the same class, function, or variable if >> ... >> their corresponding non-type template arguments of integral or >> enumeration type have identical values >> ... >> >> It looks that for non-type template arguments the template type >> equivalence is based on argument /value/ not /type/ (and value), so >> IMHO gcc is correct where it considers foo<10u> and foo<10> to be the >> same type, i.e. "refer to the same class" >> >> FWIW, type_info reports the same class name for both templates, which >> appears to be correct as per the above. >> >> I would think someone from gcc might be more specific on why both >> templates print 4294967286, and what debug info is actually stored by >> -g in this case. > > I think that Roman's example clearly shows that they are not equivalent in > all cases. > > Building Roman's example with g++ 7.3 results in a single instantiated type. > You > can see that both "new foo<10>()" and "new foo<10u>()" end up calling the same > constructor. It seems like which type is instantiated depends on which > template > parameter (the signed or unsigned one) you use first. So with this: > > base * fi = new foo<10>(); > base * fu = new foo<10u>(); > > the output is -10 for both, and with > > base * fu = new foo<10u>(); > base * fi = new foo<10>(); > > the output is 4294967286 for both. But it's probably a bogus behavior. I > tested > with clangd, it instantiates two different types, so you get 4294967286 for > the > <10u> case and -10 for the <10> case. I also just built gcc from master, and > it > also instantiates two types, so it seems like that was fixed recently. That was https://gcc.gnu.org/bugzilla/show_bug.cgi?id=79092 FWIW
Re:Re: [gcc plugin] get member offset in struct just like offsetof
Hi Eric, Thank you very much. Your advise is exact right. :) The function bit_position and byte_position is what I want to implement offsetof. Regards At 2018-02-07 00:58:17, "Eric Botcazou" wrote: >> I am writing a gcc plugin for parsing the structure fields. But I have the >> problem how to get the offset of each field in the struct? Just like the >> offsetof macro in gcc. >> >> I see DECL_FIELD_OFFSET in tree.h, but the result seems not right. > >The offset in bits/bytes is given by tree.c:bit_position/byte_position. > >-- >Eric Botcazou
Re: gdb 8.x - g++ 7.x compatibility
On 2/7/2018 4:15 PM, Jonathan Wakely wrote:
On 7 February 2018 at 15:07, Manfred wrote:
On 02/07/2018 02:44 PM, Simon Marchi wrote:
[...]
This addresses the issue of how to do good software design in GDB to
support different producers cleanly, but I think we have some issues even
before that, like how to support g++ 7.3 and up. I'll try to summarize the
issue quickly. It's now possible to end up with two templated classes with
the same name that differ only by the signedness of their non-type template
parameter. One is Foo and the other is Foo (the 10
is unsigned). Until 7.3, g++ would generate names like Foo<10> for the
former and names like Foo<10u> for the later (in the DW_AT_name attribute of
the classes' DIEs). Since 7.3, it produces Foo<10> for both.
When GDB wants to know the run time type of an object, it fetches the
pointer to its vtable, does a symbol lookup to get the linkage name and
demangles it, which gives a string like "vtable for Foo<10>" or "vtable for
Foo<10u>". It strips the "vtable for " and uses the remainder to do a type
lookup. Since g++ 7.3, you can see that doing a type lookup for Foo<10> may
find the wrong type, and doing a lookup for Foo<10u> won't find anything.
So the problem here is how to uniquely identify those two classes when we
are doing this run-time type finding operation (and probably in other cases
too).
Simon
Hi all,
In the perspective of "type identity", the way I see it the issue has a few
parts:
1) How GCC compiles such templates
2) How GCC emits debugging information via -g
3) How such information is interpreted (and merged with the compiled code)
by GDB
Regarding 1) and 2), IMHO I think that there should be a one-to-one
relationship between the compiled code output and debug info:
This means that if GCC compiles such templates into two different
classes[1], it should generate two different type identifiers.
What do you mean by "such templates"? There have been several
different examples in the thread, which should be handled differently.
From Roman 2/3/2018
#include
struct base {
virtual void print() = 0;
};
template< auto IVAL>
struct foo : base {
decltype(IVAL) x = -IVAL;
void print() override { std::cout << x << std::endl; };
};
From Simon 2/4/2018
base * fi = new foo<10>();
base * fu = new foo<10u>();
You are right that the original thread was started by Roman with:
struct base { virtual ~base(){} };
template< int IVAL, unsigned UVAL, unsigned long long ULLVAL>
struct derived : base {
int x = IVAL + + UVAL + ULLVAL;
};
Conversely, if it compiles the templates into the same class, then a single
identifier should be emitted for the single class compiled.
(This goes besides the point of what the standard dictates[2])
If I understand it right, currently the issue is that gcc emits two types
with the same debug identifier.
Regarding 3), I think that after 1) and 2) are set up, GDB should be able to
find the correct type definition (using the most appropriate design choice).
Hope this helps,
Not really :-)
Sorry for that :-)
You're basically just saying "GCC and GDB should do the right thing"
which is a statement of the obvious.
Besides the obvious, the main point was:
"IMHO I think that there should be a one-to-one relationship between the
compiled code output and debug info"
and:
"If I understand it right, currently the issue is that gcc emits two
types with the same debug identifier."
Which was an attempt to help by making obvious what I understood was
going wrong.
[1] According to the findings of Simon, this appears to be the case with
clang, older GCC, and current GCC master. Do I understand this right?
As I said above, it's not clear what you're referring to.
I had in mind foo<10> and foo<10u>
After your remark, I realize I should have left out "older GCC" because
'auto' does not apply to it - older GCC dealt with the initial example:
template< int IVAL, unsigned UVAL, unsigned long long ULLVAL>
[2] About handling both templates instantiation as a single class, I think
that if GCC wants to emit a single class, then its argument type
instantiation should be well-definined,i.e. independent of the order of
declaration - see the findings from Simon earlier in this thread where you
could get the program output either -10 or 4294967286 depending on which
declaration would come first.
That's just a GCC 7 bug in the handling of auto template parameters,
see https://gcc.gnu.org/bugzilla/show_bug.cgi?id=79092
It's not really relevant here, and is already fixed on trunk.
Thanks for pointing this out.
If I understand it correctly, the solution of the bug is that foo<10>
and foo<10u> result in two different classes
(according to your comment #1 in the bug, which by the way I am not sure
how it plays with the wording of the standard, but that's beyond gdb
compatibility)
Has -g type identification been differentiated too?
Getting back to auto/non-auto templat
Re: gdb 8.x - g++ 7.x compatibility
Hi, On Wed, 7 Feb 2018, Simon Marchi wrote: > This addresses the issue of how to do good software design in GDB to > support different producers cleanly, but I think we have some issues > even before that, like how to support g++ 7.3 and up. I'll try to > summarize the issue quickly. It's now possible to end up with two > templated classes with the same name that differ only by the signedness > of their non-type template parameter. One is Foo and the other > is Foo (the 10 is unsigned). Until 7.3, g++ would > generate names like Foo<10> for the former and names like Foo<10u> for > the later (in the DW_AT_name attribute of the classes' DIEs). Since 7.3, > it produces Foo<10> for both. Yeah, gdb needs a way to lookup types by name, and since the change DW_AT_name can't be used for this anymore. Either that needs to be fixed/reverted, or we do the more obvious thing: since types in C++ have linkage it makes sense to add the linkage (i.e. mangled) name to the types DIE using the traditional DW_AT_MIPS_linkage_name. That latter solution would have the advantage that you don't need to demangle anything anymore. From vtable you get to typeinfo, from there for typeinfo name, and that contains the mangled type name (without _Z prefix). Ciao, Michael.
Re: gdb 8.x - g++ 7.x compatibility
On 2018-02-07 11:26, Michael Matz wrote: Hi, On Wed, 7 Feb 2018, Simon Marchi wrote: This addresses the issue of how to do good software design in GDB to support different producers cleanly, but I think we have some issues even before that, like how to support g++ 7.3 and up. I'll try to summarize the issue quickly. It's now possible to end up with two templated classes with the same name that differ only by the signedness of their non-type template parameter. One is Foo and the other is Foo (the 10 is unsigned). Until 7.3, g++ would generate names like Foo<10> for the former and names like Foo<10u> for the later (in the DW_AT_name attribute of the classes' DIEs). Since 7.3, it produces Foo<10> for both. Yeah, gdb needs a way to lookup types by name, and since the change DW_AT_name can't be used for this anymore. Either that needs to be fixed/reverted, or we do the more obvious thing: since types in C++ have linkage it makes sense to add the linkage (i.e. mangled) name to the types DIE using the traditional DW_AT_MIPS_linkage_name. That latter solution would have the advantage that you don't need to demangle anything anymore. From vtable you get to typeinfo, from there for typeinfo name, and that contains the mangled type name (without _Z prefix). But do struct/classes have mangled names? Simon
Re: gdb 8.x - g++ 7.x compatibility
On 7 February 2018 at 16:36, Simon Marchi wrote: > On 2018-02-07 11:26, Michael Matz wrote: >> >> Hi, >> >> On Wed, 7 Feb 2018, Simon Marchi wrote: >> >>> This addresses the issue of how to do good software design in GDB to >>> support different producers cleanly, but I think we have some issues >>> even before that, like how to support g++ 7.3 and up. I'll try to >>> summarize the issue quickly. It's now possible to end up with two >>> templated classes with the same name that differ only by the signedness >>> of their non-type template parameter. One is Foo and the other >>> is Foo (the 10 is unsigned). Until 7.3, g++ would >>> generate names like Foo<10> for the former and names like Foo<10u> for >>> the later (in the DW_AT_name attribute of the classes' DIEs). Since 7.3, >>> it produces Foo<10> for both. >> >> >> Yeah, gdb needs a way to lookup types by name, and since the change >> DW_AT_name can't be used for this anymore. Either that needs to be >> fixed/reverted, or we do the more obvious thing: since types in C++ have >> linkage it makes sense to add the linkage (i.e. mangled) name to the types >> DIE using the traditional DW_AT_MIPS_linkage_name. >> >> That latter solution would have the advantage that you don't need to >> demangle anything anymore. From vtable you get to typeinfo, from there >> for typeinfo name, and that contains the mangled type name (without _Z >> prefix). > > > But do struct/classes have mangled names? Yes.
Re: gdb 8.x - g++ 7.x compatibility
On Wed, Feb 7, 2018 at 5:44 AM, Simon Marchi wrote: > On 2018-02-07 02:21, Daniel Berlin wrote: > >> As the person who, eons ago, wrote a bunch of the the GDB code for this >> C++ >> ABI support, and as someone who helped with DWARF support in both GDB and >> GCC, let me try to propose a useful path forward (in the hopes that >> someone >> will say "that's horrible, do it this instead") >> >> Here are the constraints i believe we are working with. >> >> 1. GDB should work with multiple DWARF producers and multiple C++ >> compilers >> implementing the C++ ABI >> 2. There is no canonical demangled format for the C++ ABI >> 3. There is no canoncial target demangler you can say everyone should use >> (and even if there was, you don't want to avoid debugging working because >> someone chose not to) >> 4. You don't want to slow down GDB if you can avoid it >> 5. Despite them all implementation the same ABI, it's still possible to >> distinguish the producers by the producer/compiler in the dwarf info. >> >> Given all that: >> >> GDB has ABI hooks that tell it what to do for various C++ ABIs. This is >> how >> it knows to call the right demangler for gcc v3's abi vs gcc v2's abi. and >> handle various differences between them. >> >> See gdb/cp-abi.h >> >> The IMHO, obvious thing to do here is: Handle the resulting demangler >> differences with 1 or more new C++ ABI hooks. >> Or, introduce C++ debuginfo producer hooks that the C++ ABI hooks use if >> folks want it to be separate. >> >> Once the producer is detected, fill in the hooks with a set of functions >> that does the right thing. >> >> I imagine this would also clean up a bundle of hacks in various parts of >> gdb trying to handle these differences anyway (which is where a lot of the >> multiple symbol lookups/etc that are often slow come from. >> If we just detected and said "this is gcc 6, it behaves like this", we >> wouldn't need to do that) >> >> In case you are worried, you will discover this is how a bunch of stuff is >> done and already contains a ball of hacks. >> >> Using hooks would be, IMHO, a significant improvement. >> > > Hi Daniel, > > Thanks for chiming in. > > This addresses the issue of how to do good software design in GDB to > support different producers cleanly, but I think we have some issues even > before that, like how to support g++ 7.3 and up. They are, IMHO, the same. > I'll try to summarize the issue quickly. It's now possible to end up with > two templated classes with the same name that differ only by the signedness > of their non-type template parameter. Yup. > One is Foo and the other is Foo (the 10 is > unsigned). Until 7.3, g++ would generate names like Foo<10> for the former > and names like Foo<10u> for the later (in the DW_AT_name attribute of the > classes' DIEs). Since 7.3, it produces Foo<10> for both. > > When GDB wants to know the run time type of an object, it fetches the > pointer to its vtable, does a symbol lookup to get the linkage name and > demangles it, Yes, this is code i wrote :) > which gives a string like "vtable for Foo<10>" or "vtable for Foo<10u>". > It strips the "vtable for " and uses the remainder to do a type lookup. > Since g++ 7.3, you can see that doing a type lookup for Foo<10> may find > the wrong type, Certainly if you can't distinguish the types you are screwed, but this is not the only way to find this type. This was in fact, the first hack i thought up to make it work because the ABI was not entirely fully formed at the time, and the debug info did not have fully qualified names. Here is a different way that should produce more consistent results. Find the linker symbol look up the symbol in the dwarf info by address. This will give you the vtable type debug info. Look at the name attribute of the debug info, which should already be demangled. Strip the "vtable for" from that. Look that up. This avoids the problem of the demangler gdb is using getting a different name than the producer used. It also should always give you the right one. If the producer calls the type "vtable fo Foo<2u>" here and "Foo<2>" elsewhere, yes, that's a bug. It should be consistent. If there are multiple types named Foo<2u>, DWARF needs to be extended to allow a pointer from the vtable debug info to the class type debug info (unless they already added one). Then you would do *no* symbol lookups, you'd follow that pointer (gdb would add it to the symbol_info structure) > and doing a lookup for Foo<10u> won't find anything. > Correct. This stripping is a hook that does the stripping and lookup in gnuv3_rtti_type. That is not the only way yo to it. > > So the problem here is how to uniquely identify those two classes when we > are doing this run-time type finding operation (and probably in other cases > too). > > > Simon >
Re: gdb 8.x - g++ 7.x compatibility
On 7 February 2018 at 17:03, Simon Marchi wrote:
> On 2018-02-07 11:50, Jonathan Wakely wrote:
>>
>> On 7 February 2018 at 16:36, Simon Marchi wrote:
>>>
>>> On 2018-02-07 11:26, Michael Matz wrote:
Hi,
On Wed, 7 Feb 2018, Simon Marchi wrote:
> This addresses the issue of how to do good software design in GDB to
> support different producers cleanly, but I think we have some issues
> even before that, like how to support g++ 7.3 and up. I'll try to
> summarize the issue quickly. It's now possible to end up with two
> templated classes with the same name that differ only by the signedness
> of their non-type template parameter. One is Foo and the other
> is Foo (the 10 is unsigned). Until 7.3, g++ would
> generate names like Foo<10> for the former and names like Foo<10u> for
> the later (in the DW_AT_name attribute of the classes' DIEs). Since
> 7.3,
> it produces Foo<10> for both.
Yeah, gdb needs a way to lookup types by name, and since the change
DW_AT_name can't be used for this anymore. Either that needs to be
fixed/reverted, or we do the more obvious thing: since types in C++ have
linkage it makes sense to add the linkage (i.e. mangled) name to the
types
DIE using the traditional DW_AT_MIPS_linkage_name.
That latter solution would have the advantage that you don't need to
demangle anything anymore. From vtable you get to typeinfo, from there
for typeinfo name, and that contains the mangled type name (without _Z
prefix).
>>>
>>>
>>>
>>> But do struct/classes have mangled names?
>>
>>
>> Yes.
Why would they not have a mangled name?
> Interesting. What do they look like, and in what context do they appear?
Anywhere you need a name for linkage purposes, such as in a function
signature, or as a template argument of another type, or in the
std::type_info::name() for the type etc. etc.
$ g++ -o test.o -c -x c++ - <<< 'struct X {}; void f(X) {}
template struct Y { }; void g(Y) {}' && nm
--defined-only test.o
T _Z1f1X
0007 T _Z1g1YI1XE
The mangled name for X is "X" and the mangled name for Y is "YI1XE"
which includes the name "X".
This isn't really on-topic for solving the GDB type lookup problem though.
Re: gdb 8.x - g++ 7.x compatibility
> > > This avoids the problem of the demangler gdb is using getting a different > name than the producer used. It also should always give you the right one. > If the producer calls the type "vtable fo Foo<2u>" here and "Foo<2>" > elsewhere, yes, that's a bug. It should be consistent. > > This should be Foo<2u> vs Foo<2> > If there are multiple types named Foo<2u>, DWARF needs to be extended to > allow a pointer from the vtable debug info to the class type debug info > (unless they already added one). > Then you would do *no* symbol lookups, you'd follow that pointer (gdb > would add it to the symbol_info structure) > Note that the ABI is explicitly designed so that type identity can be done by address comparison. Also note that adding alternative names for symbols is probably a "not great" idea, though it would work. The *vast* majority of debug info is in those names, and adding long names will often triple or quadruple the size of debug info. Google has binaries where 90% of the size is in gigabytes of linkage names. People have worked hard to need the names *less*. So you want to get *away* from going by name, especially when the compiler knows "this is the vtable that goes with this type". It should just tell you. Right now, that is what you are missing "given a vtable for a type, how do i get the type". Trying to do that by name is a hack. A hack that has lasted 15+ years mind you, but still a hack. I would just kill that hack.
Announce: GNU MPFR 4.0.1 is released
GNU MPFR 4.0.1 ("dinde aux marrons", patch level 1), a C library for
multiple-precision floating-point computations with correct rounding,
is now available for download from the MPFR web site:
http://www.mpfr.org/mpfr-4.0.1/
from InriaForge:
https://gforge.inria.fr/projects/mpfr/
and from the GNU FTP site:
https://ftp.gnu.org/gnu/mpfr/
Thanks very much to those who sent us bug reports and/or tested the
release candidate.
The SHA1 digests:
fcbbafb37c683898e585b926608d540ed037609e mpfr-4.0.1.tar.bz2
655e3cf416a0cc9530d9cb3c38dc8839504f0e98 mpfr-4.0.1.tar.gz
ae555c56a6fccd21a0ffe3dd3bdc5eb5cc1a5fce mpfr-4.0.1.tar.xz
8194e79f639c8b7c4a2bf63dc5e9945780497eb8 mpfr-4.0.1.zip
The SHA256 digests:
a4d97610ba8579d380b384b225187c250ef88cfe1d5e7226b89519374209b86b
mpfr-4.0.1.tar.bz2
e650f8723bfc6eca4f222c021db3d5d4cebe2e21c82498329bb9e6815b99c88c
mpfr-4.0.1.tar.gz
67874a60826303ee2fb6affc6dc0ddd3e749e9bfcb4c8655e3953d0458a6e16e
mpfr-4.0.1.tar.xz
a9f955d2e4ff59e001c4bf2283a0a77bd7a9f3cbdfce77a05a8249847453cfea mpfr-4.0.1.zip
The signatures:
http://www.mpfr.org/mpfr-4.0.1/mpfr-4.0.1.tar.xz.asc
http://www.mpfr.org/mpfr-4.0.1/mpfr-4.0.1.tar.bz2.asc
http://www.mpfr.org/mpfr-4.0.1/mpfr-4.0.1.tar.gz.asc
http://www.mpfr.org/mpfr-4.0.1/mpfr-4.0.1.zip.asc
Each tarball is signed by Vincent Lefèvre. This can be verified using
the DSA key ID 980C197698C3739D; this key can be retrieved with:
gpg --recv-keys 980C197698C3739D
or by downloading it from .
The key fingerprint is:
07F3 DBBE CC1A 3960 5078 094D 980C 1976 98C3 739D
The signatures can be verified with: gpg --verify
You should check that the key fingerprint matches.
Changes from version 4.0.0 to version 4.0.1:
- Improved MPFR manual.
- Improved __GMP_CC and __GMP_CFLAGS retrieval (in particular for MS Windows).
- Fixed a build failure on some platforms when --with-gmp-build is used.
- Bug fixes (see ChangeLog file), in particular in mpfr_div_ui, which
could yield an incorrectly rounded result to nearest when using
different precisions; this bug had been present since the introduction
of mpfr_div_ui, and in MPFR 4.0.0, it was affecting mpfr_div too.
- New: optional "make check-exported-symbols", mainly for the MPFR developers
and binary distributions, to check that MPFR does not define symbols with a
GMP reserved prefix (experimental).
You can send success and failure reports to , and give
us the canonical system name (by running the "./config.guess" script),
the processor and the compiler version, in order to complete the
"Platforms Known to Support MPFR" section of the MPFR 4.0.1 web page.
Regards,
--
Vincent Lefèvre - Web:
Re: gdb 8.x - g++ 7.x compatibility
On 2018-02-07 11:50, Jonathan Wakely wrote: On 7 February 2018 at 16:36, Simon Marchi wrote: On 2018-02-07 11:26, Michael Matz wrote: Hi, On Wed, 7 Feb 2018, Simon Marchi wrote: This addresses the issue of how to do good software design in GDB to support different producers cleanly, but I think we have some issues even before that, like how to support g++ 7.3 and up. I'll try to summarize the issue quickly. It's now possible to end up with two templated classes with the same name that differ only by the signedness of their non-type template parameter. One is Foo and the other is Foo (the 10 is unsigned). Until 7.3, g++ would generate names like Foo<10> for the former and names like Foo<10u> for the later (in the DW_AT_name attribute of the classes' DIEs). Since 7.3, it produces Foo<10> for both. Yeah, gdb needs a way to lookup types by name, and since the change DW_AT_name can't be used for this anymore. Either that needs to be fixed/reverted, or we do the more obvious thing: since types in C++ have linkage it makes sense to add the linkage (i.e. mangled) name to the types DIE using the traditional DW_AT_MIPS_linkage_name. That latter solution would have the advantage that you don't need to demangle anything anymore. From vtable you get to typeinfo, from there for typeinfo name, and that contains the mangled type name (without _Z prefix). But do struct/classes have mangled names? Yes. Interesting. What do they look like, and in what context do they appear? Simon
Re: gdb 8.x - g++ 7.x compatibility
On 2018-02-07 12:08, Jonathan Wakely wrote:
Why would they not have a mangled name?
Interesting. What do they look like, and in what context do they
appear?
Anywhere you need a name for linkage purposes, such as in a function
signature, or as a template argument of another type, or in the
std::type_info::name() for the type etc. etc.
$ g++ -o test.o -c -x c++ - <<< 'struct X {}; void f(X) {}
template struct Y { }; void g(Y) {}' && nm
--defined-only test.o
T _Z1f1X
0007 T _Z1g1YI1XE
The mangled name for X is "X" and the mangled name for Y is "YI1XE"
which includes the name "X".
This isn't really on-topic for solving the GDB type lookup problem
though.
Ah ok, the class name appears mangled in other entities' mangled name.
But from what I understand there's no mangled name for the class such
that
echo | c++filt
outputs the class name (e.g. "Foo<10>"). That wouldn't make sense,
since there's no symbol for the class itself.
Simon
Re: gdb 8.x - g++ 7.x compatibility
On 7 February 2018 at 17:20, Simon Marchi wrote:
> On 2018-02-07 12:08, Jonathan Wakely wrote:
>>
>> Why would they not have a mangled name?
>>
>>> Interesting. What do they look like, and in what context do they appear?
>>
>>
>> Anywhere you need a name for linkage purposes, such as in a function
>> signature, or as a template argument of another type, or in the
>> std::type_info::name() for the type etc. etc.
>>
>> $ g++ -o test.o -c -x c++ - <<< 'struct X {}; void f(X) {}
>> template struct Y { }; void g(Y) {}' && nm
>> --defined-only test.o
>> T _Z1f1X
>> 0007 T _Z1g1YI1XE
>>
>> The mangled name for X is "X" and the mangled name for Y is "YI1XE"
>> which includes the name "X".
>>
>> This isn't really on-topic for solving the GDB type lookup problem though.
>
>
> Ah ok, the class name appears mangled in other entities' mangled name. But
> from what I understand there's no mangled name for the class such that
>
> echo | c++filt
>
> outputs the class name (e.g. "Foo<10>"). That wouldn't make sense, since
> there's no symbol for the class itself.
echo _Z3FooILi10EE | c++filt
Re: gdb 8.x - g++ 7.x compatibility
On Wed, 7 Feb 2018, Simon Marchi wrote:
On 2018-02-07 12:08, Jonathan Wakely wrote:
Why would they not have a mangled name?
Interesting. What do they look like, and in what context do they appear?
Anywhere you need a name for linkage purposes, such as in a function
signature, or as a template argument of another type, or in the
std::type_info::name() for the type etc. etc.
$ g++ -o test.o -c -x c++ - <<< 'struct X {}; void f(X) {}
template struct Y { }; void g(Y) {}' && nm
--defined-only test.o
T _Z1f1X
0007 T _Z1g1YI1XE
The mangled name for X is "X" and the mangled name for Y is "YI1XE"
which includes the name "X".
This isn't really on-topic for solving the GDB type lookup problem though.
Ah ok, the class name appears mangled in other entities' mangled name. But
from what I understand there's no mangled name for the class such that
echo | c++filt
outputs the class name (e.g. "Foo<10>"). That wouldn't make sense, since
there's no symbol for the class itself.
$ echo _Z1YI1XE | c++filt
Y
--
Marc Glisse
Re: gdb 8.x - g++ 7.x compatibility
On 2018-02-07 12:30, Jonathan Wakely wrote: Ah ok, the class name appears mangled in other entities' mangled name. But from what I understand there's no mangled name for the class such that echo | c++filt outputs the class name (e.g. "Foo<10>"). That wouldn't make sense, since there's no symbol for the class itself. echo _Z3FooILi10EE | c++filt Ok, thanks for the precision!
Re: gdb 8.x - g++ 7.x compatibility
> "Dan" == Daniel Berlin writes: Dan> If there are multiple types named Foo<2u>, DWARF needs to be extended to Dan> allow a pointer from the vtable debug info to the class type debug info Dan> (unless they already added one). This is what we did for Rust. Rust doesn't have a stable ABI yet, so using gdb's current approach -- having the debugger use details of the ABI in addition to the debug info -- wasn't an option. So, instead, the Rust compiler emits DWARF for the vtable and associates the vtable with the concrete type for which it was emitted. This required a minor DWARF extension. I think C++ could probably do something along these lines as well. The current gdb approach hasn't been really solid since function-local classes were added to C++. IIRC there are bugs in gdb bugzilla about this. These kinds of problems are, I think, completely avoided by a DWARF-based approach. Tom
Re: gdb 8.x - g++ 7.x compatibility
On 02/07/2018 12:11 PM, Daniel Berlin wrote: Note that the ABI is explicitly designed so that type identity can be done by address comparison. correct, but be aware that lots of dynamic objects seem to step outside the ABI by building shared objects with -Bsymbolic[1], or the equivalent visibility=hidden, or similar. So now the typeinfo comparison operator is something like return this->name == other->name || (this->name[0] != '*' && other->name[0] != '*' && !strcmp (this->name, other->name)) nathan [1] I see -Bsymbolic-functions is now a thing, which would be better -- Nathan Sidwell
gcc-6-20180207 is now available
Snapshot gcc-6-20180207 is now available on ftp://gcc.gnu.org/pub/gcc/snapshots/6-20180207/ and on various mirrors, see http://gcc.gnu.org/mirrors.html for details. This snapshot has been generated from the GCC 6 SVN branch with the following options: svn://gcc.gnu.org/svn/gcc/branches/gcc-6-branch revision 257466 You'll find: gcc-6-20180207.tar.xzComplete GCC SHA256=3c580e191f8ecf088f9ff5cc7db2716017b9622bfd4e2f8b61b2df7b84240745 SHA1=27197d50e9c89ba52933d5b9b754284ea1a6818c Diffs from 6-20180131 are available in the diffs/ subdirectory. When a particular snapshot is ready for public consumption the LATEST-6 link is updated and a message is sent to the gcc list. Please do not use a snapshot before it has been announced that way.
Re: GCC 8.0.0 status on x86_64-w64-mingw32, some issues
> Indeed, this solves most of the new failures. Here is the acats test > summary: > === acats Summary === > # of expected passes 2298 > # of unexpected failures 22 > *** FAILURES: c23003b c23003g c23003i c250002 c380004 cd2b11a cd2b15c > ce2102l ce2102m ce2103a ce2103b ce3102d ce3107a ce3115a cxa4005 cxa4008 > cxa4016 cxa4019 cxac003 cxb3012 cxf3a01 cxf3a02 > /opt/devel/gnu/src/gcc-mingw-w64/gcc-8.0.0/gcc/testsuite/ada/acats/run_all.s > h completed at Wed Feb 7 12:28:36 CET 2018 Please open a PR for the ACATS regressions on mainline. -- Eric Botcazou
Re: gcc 7.3: Replacing global operator new/delete in shared libraries
On 02/06/2018 03:56 PM, Paul Smith wrote: Hi all. Hopefully this isn't too annoying a question :). My environment has been using GCC 6.2 (locally compiled) on GNU/Linux systems. We use a separate heap management library (jemalloc) rather than the libc allocator. The way we did this in the past was to declare operator new/delete (all forms) as inline functions in a header and ensure that this header was always the very first thing in every source file, before even any standard header files. I know that inline operator new/delete isn't OK in the C++ standard, but in fact it has worked for us on the systems we care about. I don't know if something has changed that would expose this problem but... I'm not sure I see how defining operator new inline can work unless you recompile the world (i.e., all the DSOs used by the program, including libstdc++). As Marc already hinted, if libstdc++ dynamically allocates an object using the default operator new and returns that object to the program to destroy, it will end up causing a mismatch. The same can happen in the opposite direction. To avoid such mismatches the entire program needs to use a single operator new (each of the required forms), and the only safe way to do that is to define each out-of-line. Martin PS This question was the subject of C++ CWG issue 412: http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_closed.html#412 I'm attempting a toolchain upgrade which is switching to GCC 7.3 / binutils 2.30 (along with many other updates). Now when I run our code, I get a core on exit. It appears an STL container delete is invoking libc free() with a pointer to memory allocated by jemalloc. I suspect that between 6.2 and 7.3 something in the STL has been modified to call new in a header file, so it's using our inline operator new, but call the matching delete from inside libstdc++.a (we link with static libstdc++ for portability), so it doesn't use our inline operator delete. While it's unfortunate for us, obviously that's a perfectly legal implementation choice. I don't know whether this is something the GCC folks care about. If so I can do more to track down the specifics. If I create a real global operator new/delete, even keeping the inlined versions, then the problem goes away (lending more weight to my guess above). I should point out that we don't use much STL memory so having some compiled (not header-based) STL use the libc allocator is not a big deal to us... it's just the mismatch which is a problem. This leads to my question: One of the things we provide is a shared library including much of our code, and also jemalloc. Users link this shared library with their code and we do not want them to use our allocator. By having all our operator new/delete inlined we are sure that all our requests go to our allocator and their requests do not. It's a bit grungy, perhaps, but it's worked well until now. My question is, what do I need to do to ensure this behavior persists if I create a global operator new/delete? Is it sufficient to ensure that the symbol for our shared library global new/delete symbols are hidden and not global, using a linker map or -fvisibility=hidden?
Re: gcc 7.3: Replacing global operator new/delete in shared libraries
On 7 February 2018 at 23:38, Martin Sebor wrote:
> On 02/06/2018 03:56 PM, Paul Smith wrote:
>>
>> Hi all.
>>
>> Hopefully this isn't too annoying a question :).
>>
>> My environment has been using GCC 6.2 (locally compiled) on GNU/Linux
>> systems. We use a separate heap management library (jemalloc) rather
>> than the libc allocator. The way we did this in the past was to
>> declare operator new/delete (all forms) as inline functions in a header
>> and ensure that this header was always the very first thing in every
>> source file, before even any standard header files. I know that inline
>> operator new/delete isn't OK in the C++ standard, but in fact it has
>> worked for us on the systems we care about.
>
>
> I don't know if something has changed that would expose this
> problem but...
>
> I'm not sure I see how defining operator new inline can work
> unless you recompile the world (i.e., all the DSOs used by
> the program, including libstdc++). As Marc already hinted,
> if libstdc++ dynamically allocates an object using the default
> operator new and returns that object to the program to destroy,
> it will end up causing a mismatch.
And without actually checking the code, I'm pretty sure that's what
happens in this case:
#include
int main()
{
std::string s;
std::cin >> s;
}
The code to read a string from an istream is instantiated in the
library, so the string gets resized (allocating using malloc) by code
inside libstdc++.so.6 and then the destructor is run (deallocating
using your inline operator delete) in main.
That would be my first guess at explaining the stack trace you showed.
Re: gcc 7.3: Replacing global operator new/delete in shared libraries
On Wed, 7 Feb 2018, Paul Smith wrote: My question is, what do I need to do to ensure this behavior persists if I create a global operator new/delete? Is it sufficient to ensure that the symbol for our shared library global new/delete symbols are hidden and not global, using a linker map or -fvisibility=hidden? I think so (hidden implies not-interposable, so locally bound), but I don't have much experience there. OK I'll pursue this for now. I answered too fast. It isn't just new/delete that need to be hidden. It is also anything that uses them and might be used in both contexts. For instance, std::allocator::allocate is an inline function that calls operator new. You get one version that calls new1, and one version that calls new2. If you don't do anything special, the linker keeps only one (more or less arbitrarily). So I believe you need -fvisibility=hidden to hide everything but a few carefully chosen interfaces. -- Marc Glisse
Re: gcc 7.3: Replacing global operator new/delete in shared libraries
On Wed, 2018-02-07 at 16:38 -0700, Martin Sebor wrote: > I'm not sure I see how defining operator new inline can work > unless you recompile the world (i.e., all the DSOs used by > the program, including libstdc++). As Marc already hinted, > if libstdc++ dynamically allocates an object using the default > operator new and returns that object to the program to destroy, > it will end up causing a mismatch. The same can happen in > the opposite direction. To avoid such mismatches the entire > program needs to use a single operator new (each of > the required forms), and the only safe way to do that > is to define each out-of-line. I'm aware of these issues, and I know it's a dangerous game. As I mentioned I wasn't surprised, really, that eventually it caught us out. However, it did work. We don't use huge amounts of the STL but we use basics like vector, string, a bit of unordered_map, etc., and those worked fine. All that's required for it to work is that either both the new and delete were both performed inside libstdc++, or both the new and delete were performed in STL header file implementations. In the former case that memory is coming from the system alloc, not jemalloc, but the amount is small enough that it's not worrisome. In the latter case it will use jemalloc via the inline. The problem comes in where you do the new in an STL header and the delete in the compiled libstdc++, or vice versa... that's what I ran into in GCC 7.3. On GNU/Linux you can just replace malloc/free using non-global symbols in the shared library, to ensure even libstdc++.a implementations use jemalloc. Unfortunately this is not possible in Windows. On Wed, 7 Feb 2018 Jonathan Wakely writes: > The code to read a string from an istream is instantiated in the > library, so the string gets resized (allocating using malloc) by code > inside libstdc++.so.6 and then the destructor is run (deallocating > using your inline operator delete) in main. We don't use C++ iostream in our code. Although, I'm sure GTest does use it in THEIR code. Of course, for unit tests we don't really care which allocator is used as long as there's no mismatch. I haven't investigated exactly how it all worked in 6.2, I can only tell you that for our use, it did :). Thanks for the conversation. I'm moving forward with a real global operator new/delete and working out the magic needed to ensure those symbols are not global in our shared library.
Re: gcc 7.3: Replacing global operator new/delete in shared libraries
On Thu, 2018-02-08 at 01:17 +0100, Marc Glisse wrote:
> On Wed, 7 Feb 2018, Paul Smith wrote:
> > > > My question is, what do I need to do to ensure this behavior
> > > > persists if I create a global operator new/delete?
> > > >
> > > > Is it sufficient to ensure that the symbol for our shared
> > > > library global new/delete symbols are hidden and not global,
> > > > using a linker map or -fvisibility=hidden?
> > >
> > > I think so (hidden implies not-interposable, so locally bound),
> > > but I don't have much experience there.
> >
> > OK I'll pursue this for now.
>
> I answered too fast. It isn't just new/delete that need to be hidden.
> It is also anything that uses them and might be used in both
> contexts. For instance, std::allocator::allocate is an inline
> function that calls operator new. You get one version that calls
> new1, and one version that calls new2. If you don't do anything
> special, the linker keeps only one (more or less arbitrarily). So I
> believe you need -fvisibility=hidden to hide everything but a few
> carefully chosen interfaces.
At the moment I'm compiling all my code with -fvisibility=hidden, and
marking out specific symbols to be public in the source code, with
__attribute__((visibility("default"))).
This is nice andeasy, but I've grown frustrated with it. In order to
run our unit tests we must statically link them with the code; we can't
create a shared library because all the internal symbols that the unit
tests want to refer to are hidden. Since we have about 150 individual
unit test programs and each one statically links all the code it uses
(although we're using GTest framework and I call these "unit tests",
they're not really unit tests in the classical sense that they mock out
a single class for test), that uses a lot of disk space and takes a lot
of link time during the builds... it's already aggravating and we're
only adding more unit tests over time.
So in the near future I intend to reset this and compile all the code
without -fvisibility=hidden, then when I create our shared library I'll
generate a linker map to make only specific symbols visible and hide
everything else.
What would be ideal is if I could continue to use the visibility
attributes in the source code to mark out symbols I wanted to publish.
If that information were preserved in the ELF output in a way I could
extract with a script running objdump or readelf on the object files,
for example, then I could automate the generation of the proper linker
map for my shared library.
However the last time I checked this, visibility("default") didn't
leave a trace in the object file unless -fvisibility=hidden was used to
make the default visibility hidden.
Too bad.
