Hi Ludovic, thanks for your thoughts. You’re right, the LD_LIBRARY_PATH will not change the loading order, but using LD_PRELOAD will by the same mechanism we’re using, pre-filling the cache with a library at the same soname. As part of other explorations we’re doing around tweaking or wrapping the loader, it may be possible to get semantics like LD_LIBRARY_PATH other ways, but at the moment the goal is to make a program that will correctly load all the dependencies it would have loaded were it run in the same environment it was built in, despite LD_LIBRARY_PATH or RUNPATH in dependencies or similar. Making a little tool that would override the same way LD_LIBRARY_PATH would have would be relatively straightforward though, would that be worth exploring do you think?
-Tom On 8 Jan 2022, at 19:05, Farid Zakaria wrote: > I did forget to mention the point of LD_LIBRARY_PATH, that you can > still make use of LD_PRELOAD and I am also thinking about maybe using > something like dlopen-resolver[1] to further expand the NEEDED > section. > > [1] > https://urldefense.us/v3/__https://github.com/Mic92/dlopen-resolver__;!!G2kpM7uM-TzIFchu!gFf3bOVCBsw_Ld35XMHl8Y0nYb0k7ikmOrOuo5SGPLCLqrCqx5qaP3giJkQTBeRD1A$ > > On Sat, Jan 8, 2022 at 7:00 PM Farid Zakaria <fmzak...@ucsc.edu> wrote: >> >> Hi Ludovic, >> >> On Sat, Jan 8, 2022 at 1:22 PM Ludovic Courtès <l...@gnu.org> wrote: >>> >>> Hi Farid, >>> >>> Farid Zakaria <fmzak...@ucsc.edu> skribis: >>> >>>> I have written a tool _shrinkwrap_ [2] that takes all transitive >>>> dynamic shared object dependencies (only those listed in DT_NEEDED) >>>> and turns them into an absolute path. >>>> >>>> This has the same result as caching the entries and avoids the >>>> unnecessary failed attempts at trying each RUNPATH entry. >>>> >>>> Using the same demo application _emacs_ shows as much as well: >>> >>> Nice! I think that’s another interesting way to address the problem. >>> >>> I guess the advantage is that you don’t need the ld.so patch. The >>> downside is that PatchELF needs to be able to write longer NEEDED >>> strings in the dynamic section, which it may not always be successful at >>> (I think?). >> >> I can't claim to be a ELF specification guru but I have not >> encountered that longer NEEDED strings to be a cause for failure. >> The emacs example is a pretty good test case because the transitive >> closure of all NEEDED libraries is quite large, which all seem to be >> added successfully to the ELF header. >> >> The benefit to me seems: >> 1 - does not need a glibc patch for functionality (although for other >> libc such as musl it might in this case >> https://urldefense.us/v3/__https://www.openwall.com/lists/musl/2021/12/21/1__;!!G2kpM7uM-TzIFchu!gFf3bOVCBsw_Ld35XMHl8Y0nYb0k7ikmOrOuo5SGPLCLqrCqx5qaP3giJkSfjFvBcw$ >> ) >> 2 - understanding the dependencies of an application become simpler >> 3 - there are esoteric cases where in fact libraries might link to the >> wrong libraries (although they are correct at build time) given a >> RUNPATH/RPATH since there are subtleties with the inheritance model. >> >> I'm actually researching ways to improve (3) as well through >> mentorship with Tom Scogland by researching alternative ways to do >> linking: >> - RUNPATH per NEEDED >> - the ability to specify whether a RUNPATH should be inherited or not >> to downstream dependencies >> >>> Also, I wonder if the absolute file names in NEEDED interfere with uses >>> of $LD_LIBRARY_PATH (making it impossible to force use of another >>> libxyz.so than the one that would be found in RUNPATH.) >> >> Correct. For a system with reproducibility in mind this can perhaps be >> a desired feature. >> It is the current limitation of the proposal. >> >> In fact, Carlos brought up a great philosophical question: >> "Is linking to libraries through a content-addressable value allowed >> for LGPL software?" >> What if the linked address also forced the content-address by having >> it resolve to something on IPFS ? >> >>> Thoughts? >>> >>> Thanks for sharing! >>> >>> Ludo’.
