On Thu, Sep 26, 2019 at 05:32:12PM +0200, Andrzej Ostruszka wrote: > On 9/24/19 2:59 PM, Neil Horman wrote: > > On Tue, Sep 24, 2019 at 12:25:35PM +0200, Bruce Richardson wrote: > >> On Tue, Sep 24, 2019 at 08:46:25AM +0200, Andrzej Ostruszka wrote: > >>> On 9/23/19 6:13 PM, Bruce Richardson wrote: > [...] > >> The real issue seems to be that the compat.h header has different > >> compilation paths for static and shared libraries, which means that any C > >> file including it can't have a .o file that can be used for both a .a and a > >> .so simultaneously. Having it default to the shared library path seems to > >> work fine thus far, but with LTO it seems broken as you say. Adding Neil as > >> the original file author of this in case he has any suggestions here. I'd > >> really rather not have to go back to building .a's and .so's separately. > >> > > The notion of using the same object file to link to a static archive and a > > dso > > seems somewhat suspect to me in general. > > I'd think so too ... but there might be something fishy with gcc here. > More on this below. > > [...] > > That said, if the goal is to just overcome this particular situation, it > > might > > (strong might), be sufficient to simply augment the MAP_STATIC_SYMBOL macro > > in > > the CONFIG_RTE_BUILD_SHARED_LIB=n case to append the 'used' attribute. > > Ostensibly, LTO would be smart enough then to not eliminate the symbol? > > Just a > > thought. > Just to clarify things here is the current status in a nutshell. > 1. The make based build work with LTO (both static and shared). It is > using CONFIG_*_SHARED_LIB option and thus different paths of > rte_compat.h are used. > > 2. The meson build is not using config and has "SHARED" fixed to "y" in > config/rte_config.h. This works and produces: > - shared library that is working when linking both w/ and w/o LTO > - static library that is only working when linking w/o LTO - when > linking with LTO then it complains about missing symbols for which > different symbol versions are defined. > > Augmenting MAP_STATIC_SYMBOL won't help since it is not used at all in > meson build. > > I've played around with couple ideas. Some of them might sound stupid > for you - but I'll report them anyway > > Modifying symbol tables > ----------------------- > I thought that since problems are only with versioned symbols then I'll > try to modify the symbols tables archives so they look like in "static > make" case e.g. (for just one symbol): > > $ objcopy --strip-symbol=rte_timer_subsystem_init@DPDK_2.0 \ > --redefine-sym \ > rte_timer_subsystem_init@@DPDK_19.05=rte_timer_subsystem_init \ > librte_timer.a > > After this the symbol table looks like in fully static case but this > doesn't work so I'm pretty sure that when using LTO linker does not > check the symbol table at all and just looks in "lto" sections. > > Adding static macro > ------------------- > I've added for the shared case macro: > #define MAP_STATIC_SYMBOL(f, p) f \ > __attribute__((alias(RTE_STR(p)),weak)) > with the idea that maybe during linking against *.a versioned symbols > are not taken into account and if I add weak non-versioned symbol then > it will be used when linking against *.a and when linking against *.so > the strong versioned one will be used. This thinking is in contrast > with the fact that meson build works w/o LTO where only versioned > symbols are present in those problematic libs :) - and it doesn't work. > Linker reports multiple definitions. > > So adding these two together it seems to me that: > a) gcc is using only internal "lto" sections when linking with LTO > b) gcc is storing those versioned symbols in those "lto" sections > c) when linking w/ LTO against archives is not capable to use those > versioned symbols in "lto" sections > d) when linking w/ LTO against shared libraries is capable to use > versioned symbols from "lto" sections > e) when linking w/o LTO against archive is capable to use versioned > symbols from global symbol table (in contrast with point 'c' above) > > I'd appreciate some input from those who know gcc internals. In the > meantime I'll try to come up with minimal example and follow up on gcc > related lists/groups. > Please have a look over the patchset I just posted, as one possible solution to this. http://patches.dpdk.org/project/dpdk/list/?series=6594
For the general case of compiling DPDK, a given block of C code is going to result in the same object file whether compiled for static or shared (so long as -fPIC is passed to the compile), so running two compiles for each and every C file would be wasted effort. However, for those files with function versioning, I think any proper solution has to involve compiling twice with different macros, so that's what the patchset seeks to achieve. /Bruce
