[cfe-users] order of object files at link affects exception catching
First of all a preface - This problem was spotted while trying to build a large C++ project which links a close to 100 of object file together, plus libraries. I can't replicate this behavior in a simple isolated test. Just want to understand if potentially this may be caused by clang's compiler or linker behavior (missed flag, or optimization effect/bug). The project builds and runs correctly with GCC. Compiler: clang-10 on OSX 10.13 The project builds fully without errors and the final binary executable is produced. The binary starts up ok and presents a prompt. However any exception-based processing (like input errors are expected to show a message and continue, or catching Ctrl+C and processing into a message and continue) result in uncaught exception and ends in abort(). Basically, the libc++ calls std::terminate(), as if the proper catch statement is missing, which clearly is in the code. Somehow the exception unwind stack gets broken. The code links a large number of objects and a few .a libraries, so I tried to put the individual objects into another .a lib to try to eliminate the order effects. Still, the resulting binary has the exception catching issues. Then I tried to craft a simple test (which does not use any of the actual project's code) that has throw/catch and then linked it in the same way. The results: * when the test code is linked from the .a library (with all objects as above), the exceptions are processed ok. * when the test code is linked with all the objects above specified on the command line, the exception issues are back. Obviously, the simple test code does not need any of the code from the other objects, yet the resulting code appears somehow broken. Granted, the linker will have to resolve all dependecies it finds on the command line and tie it into the binary, still none of those functions should be executed by the sample test code. Finally, I tried to change the order of the project's object files at line and put the object file which does the actual throw, right next to the main's object file. To my surprise, the exceptions were caught ok... But too soon to celebrate, exceptions tripped in other parts of the code still were not caught properly. So the bottom line, some how the exceptions table gets messed up in the process on linking. I can't think of any other way to diagnose this. By the way, the very same code is properly linked and functioning when using GCC default compile/link options. I tired without success -fno-lto to disable link-time-optimization, but that's default anyway. To reiterate the questions: 1. Why would order of the object files matter for correct exception processing? 2. Are there some clang's options specific for such cases? Any ideas are welcome! ___ cfe-users mailing list cfe-users@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-users
Re: [cfe-users] clang and C++: exporting member function template from library using attribute visibility("default")
On Mon, 30 Mar 2020 at 14:19, Alexis Murzeau via cfe-users <
cfe-users@lists.llvm.org> wrote:
> Hi,
>
> When using clang, I discovered that it errors out where other compilers
> doesn't (GCC and MSVC).
>
> I'm trying to do this:
> - Have a library compiled with -fvisibility=hidden and adding
> __attribute__((visibility("default")))
>only for stuff that must be exported from the library .so.
>
> - Have an exported class (with __attribute__((visibility("default"
> that
>declares a member function template
>
> - A cpp file in the library does does instantiation of that member
> function
>template for all applicable types that would ever be usable by that
> member function template.
>
If you want one source file to provide instantiations for use by a
different source file, it's not sufficient to merely trigger those
instantiations. That only generates a discardable definition of the
instantiation, and if (for example) that definition is inlined, no symbol
will be provided for other translation units to link against. Instead you
need to use an explicit instantiation. (See
https://en.cppreference.com/w/cpp/language/function_template#Explicit_instantiation
)
> When compiling that library, the member function template instantiations
> are
> not being exported from the library (they are hidden).
>
> The error comes then when an executable try to use that function, but
> compilation fails because of undefined reference to `void
> NetworkPacketLogger::logType(T1 const*)'.
>
>
> In a lib.h:
> ```
> // An enum defining the network packet
> enum class Opcode {
> T1,
> T2
> };
>
> // Base class
> struct NetworkPacket {
> NetworkPacket(enum Opcode t) : type(t) {}
>
> enum Opcode type;
> };
>
> // Sample of possible network packets
> struct T1 : public NetworkPacket {
> T1() : NetworkPacket(Opcode::T1) {}
> int value;
> };
>
> struct T2 : public NetworkPacket {
> T2() : NetworkPacket(Opcode::T2) {}
> float value;
> };
>
> // Class to log a network packet
> class __attribute__((visibility("default"))) NetworkPacketLogger {
> public:
> // "Slow" function that find the correct packet opcode and log
> it's content
> // vvv this function is exported just because of the attribute on
> the NetworkPacketLogger, OK
> static void logAbstractType(struct NetworkPacket* abstractData);
>
> // Fast function that doesn't have to find the packet opcode
> // It does just a log of data->value
> // T can only be either T1 or T2
>
> // vvv this is the hidden function that should be exported from
> the lib
> template static void logType(const T* data)
> __attribute__((visibility("default")));
> };
> ```
>
> In a lib.cpp:
> ```
> template
> void NetworkPacketLogger::logType(const T* data) {
> std::cout << data->value;
> }
>
> void NetworkPacketLogger::logAbstractType(struct NetworkPacket*
> abstractData) {
> // Possibly generated code is there are many possible network
> packets
> // This will implicitely instanciate all possible combination of
> NetworkPacketLogger::logType
> // I want these instanciations to be available by user of the
> library
>
To make these available to users, you should explicitly instantiate them as
follows:
template void NetworkPacketLogger::logType(const T1*);
template void NetworkPacketLogger::logType(const T2*);
switch(abstractData->type) {
> case Opcode::T1:
> logType(static_cast(abstractData));
> break;
> case Opcode::T2:
> logType(static_cast(abstractData));
> break;
> }
> }
> ```
>
>
> I found by tweaking the code that, if I add a
> __attribute__((visibility("default")))
> on struct T1 and struct T2, then logAbstractType and
> logAbstractType are exported.
>
> But why is this required for clang ?
> It seems to be like this old bug:
> https://bugs.llvm.org/show_bug.cgi?id=8457
>
> Is this expected ?
>
Yes. You're probably just getting lucky with the other compilers, and they
happen to not inline either of the 'logType' functions.
> Shouldn't a warning be emitted when a function that should be
> "visibility("default")"
> is not because of one of the arguments use a struct with
> visibility("hidden") ?
>
>
> I'm attaching a test case.
> Can be compiled with something like this:
> ```
> mkdir build && cd build
> CC=clang-9 CXX=clang++-9 cmake ..
> make
> nm -CD liblib.so | grep logType
> ```
>
> nm will print only this (no NetworkPacketLogger::logType):
> 11b0 T NetworkPacketLogger::logAbstractType(NetworkPacket*)
> 1260 W void NetworkPacketLogger::logType(T2 const*)
> 1290 W void NetworkPacketLogger::logType(T3 const*)
>
>
> Thanks for your hindsight :)
>
> --
> Alexis Murzeau
> PGP: B7E6 0EBB 9293 7B06 BDBC 2787 E7BD 1904 F480 937F
> _
Re: [cfe-users] order of object files at link affects exception catching
On Sun, 5 Apr 2020 at 15:31, krokus via cfe-users wrote: > First of all a preface - This problem was spotted while trying to > build a large C++ project which links a close to 100 of object file > together, plus libraries. I can't replicate this behavior in a simple > isolated test. Just want to understand if potentially this may be > caused by clang's compiler or linker behavior (missed flag, or > optimization effect/bug). The project builds and runs correctly with > GCC. > > Compiler: clang-10 on OSX 10.13 > > The project builds fully without errors and the final binary > executable is produced. The binary starts up ok and presents a prompt. > However any exception-based processing (like input errors are expected > to show a message and continue, or catching Ctrl+C and processing into > a message and continue) result in uncaught exception and ends in > abort(). Basically, the libc++ calls std::terminate(), as if the > proper catch statement is missing, which clearly is in the code. > Somehow the exception unwind stack gets broken. > > The code links a large number of objects and a few .a libraries, so I > tried to put the individual objects into another .a lib to try to > eliminate the order effects. Still, the resulting binary has the > exception catching issues. > > Then I tried to craft a simple test (which does not use any of the > actual project's code) that has throw/catch and then linked it in the > same way. The results: > * when the test code is linked from the .a library (with all objects > as above), the exceptions are processed ok. > * when the test code is linked with all the objects above specified > on the command line, the exception issues are back. > Only objects that contain referenced symbols get pulled in from archives, so using a .a library will tend to result in fewer objects being linked in than specifying the .o files on the command line. That might explain part of the difference you're seeing here. > Obviously, the simple test code does not need any of the code from the > other objects, yet the resulting code appears somehow broken. Granted, > the linker will have to resolve all dependecies it finds on the > command line and tie it into the binary, still none of those functions > should be executed by the sample test code. > > Finally, I tried to change the order of the project's object files at > line and put the object file which does the actual throw, right next > to the main's object file. To my surprise, the exceptions were caught > ok... But too soon to celebrate, exceptions tripped in other parts of > the code still were not caught properly. > > So the bottom line, some how the exceptions table gets messed up in > the process on linking. I can't think of any other way to diagnose > this. > > By the way, the very same code is properly linked and functioning when > using GCC default compile/link options. > > I tired without success -fno-lto to disable link-time-optimization, > but that's default anyway. > > To reiterate the questions: > 1. Why would order of the object files matter for correct exception > processing? > The most likely explanation is that your program contains a violation of C++'s "One Definition Rule" (ODR). Specifically, you probably have a function or class that's defined in different ways in two different source files, and the behavior of your program depends on which one gets picked at link time. (Worse, there are ways in which we can end up picking one version from one .o file and a different version from a different .o file.) Given the symptoms, it's possible that this is happening because part of your program is built with -fno-exceptions and part of your program is build without that flag, and an exception in question is propagating through a (perhaps inline) function that was built both ways. But that's just a guess. > 2. Are there some clang's options specific for such cases? > Do you still see the issue with -O0? Do you still see the issue if you explicitly add -fexceptions to every compilation? ___ cfe-users mailing list cfe-users@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-users
