Dear George, as we discussed this issue back in 2016, 'cutting edge distribution' may be a relative term. This discussion, https://einsteinathome.org/content/vsyscall-now-disabled-latest-linux-distros, refers to 'ancient versions of glibc' where the problem occurs. It seems to be solved with libc-2.15, which was released 2012. I suspect that there are so few reports w.r.t. shelx? because most distributions enable 'vsyscall=emulate' by default to avoid complaints. Debian just does not do this.
It may be time to balance the number of SHELX users that still run a 'ancient versions of' linux vs. those which upgraded their operating system at least once since 2011... Best, Tim On Tuesday, October 8, 2019 3:53:02 PM CEST Peter Keller wrote: > Dear George > > On 08/10/2019 11:21, George Sheldrick wrote: > > As explained in the attached email from Peter Keller, I was > > deliberately preparing the binary Linux SHELX distribution using older > > (2011) system libraries so that the programs would run on older > > systems that many users are still using. Unfortunately this means that > > they do not run on some recent cutting edge distributions including > > Debian 10. This can be fixed with 'vsyscall=emulate' when installing > > the OS but not all users may be allowed to do this. Dynamic binaries > > would be smaller but would require the user to provide the right > > libraries. > > > > If I prepare statically linked binaries (using the latest ifort and > > ubuntu) they appear to run on current systems but may have problems on > > older systems. I may have to offer (e.g. in CCP4 and on the SHELX > > server) two sets of Linux binaries in the future, one for 'vintage' > > systems and one for current systems. > > This is treating the two possibilities as different architectures. This > is a viable approach. Having said that, I doubt that anyone is using a > system now that only supports the vsyscall method for time() etc. I > suspect that linking with a suitable version of glibc would produce a > portable, static executable. > > Then, there is also the package approach that David mentioned, although > that requires learning new tools. > > > Alternatively I could provide both statically and dynamically linked > > versions. > > There are two approaches for rolling your own dynamically-linked > executables that avoid the need for users to supply the required > libraries themselves: > > (1) Link most libraries statically, using link options like > '-Wl,-Bstatic,-lsomelib1,-lsomelib2,-Bdynamic'. This is the kind of > approach that Clemens Vonrhein is using for a future release of Global > Phasing software. The final executables only have minimal run-time > dependencies: > > % ldd `which process` > linux-vdso.so.1 (0x00007ffffb54f000) > libc.so.6 => /lib64/libc.so.6 (0x00007f144842a000) > /lib64/ld-linux-x86-64.so.2 (0x00007f14487e4000) > > If a system cannot provide these dependencies, the sysadmin will have > bigger problems to solve than SHELXD not working ;-) > > (2) Ship any required libraries that are not provided by the system > > along with your binaries. This is the approach that CCP4 is taking: > > % ldd `which mtzdump` > > linux-vdso.so.1 (0x00007ffe039e1000) > > libccp4f.so.0 => > > /public/xtal/ccp4/7.0/official/Linux-x86_64/ccp4-7.0/bin/../lib/libccp4f.s > > o.0 (0x00007fbdd53f5000) > > libccp4c.so.0 => > > /public/xtal/ccp4/7.0/official/Linux-x86_64/ccp4-7.0/bin/../lib/libccp4c.s > > o.0 (0x00007fbdd51bd000) > > libgfortran.so.3 => > > /public/xtal/ccp4/7.0/official/Linux-x86_64/ccp4-7.0/bin/../lib/libgfortra > > n.so.3 (0x00007fbdd4ecb000) > > libm.so.6 => /lib64/libm.so.6 (0x00007fbdd4b93000) > > libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fbdd497b000) > > libc.so.6 => /lib64/libc.so.6 (0x00007fbdd45c1000) > > libmmdb2.so.0 => > > /public/xtal/ccp4/7.0/official/Linux-x86_64/ccp4-7.0/bin/../lib/../lib/lib > > mmdb2.so.0 (0x00007fbdd42af000) > > /lib64/ld-linux-x86-64.so.2 (0x00007fbdd7f5e000) > > libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007fbdd3f25000) > > The path to the directory containing CCP4-supplied shared libraries is > > encoded in the executable itself: > > % readelf -d `which mtzdump` > > > > Dynamic section at offset 0x7650 contains 31 entries: > > Tag Type Name/Value > > 0x0000000000000001 (NEEDED) Shared library: [libccp4f.so.0] > > 0x0000000000000001 (NEEDED) Shared library: [libccp4c.so.0] > > 0x0000000000000001 (NEEDED) Shared library: > > [libgfortran.so.3] > > 0x0000000000000001 (NEEDED) Shared library: [libm.so.6] > > 0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1] > > 0x0000000000000001 (NEEDED) Shared library: [libc.so.6] > > 0x000000000000000f (RPATH) Library rpath: [$ORIGIN/../lib] > > Note the special token "$ORIGIN", which means the directory containing > the executable (this is documented in the 'man' page for ld.so). The > RPATH item is set to the above value with an option like > '-Wl,-rpath,\$ORIGIN/../lib' (where the '$' character needs to be > escaped from the shell, obviously). > > Note that -rpath is not the same as -L: -L is used at link-time to find > the libraries needed to check that all the symbols are resolvable. > -rpath is used to locate shared libraries at run-time: its argument > doesn't even have to exist at link-time. If you go down this route, you > will need to specify both -L and -rpath. > > > What do users think? > > As for which is the best approach, I'll let others make their opinions > known, but I hope that this explanation has helped. > > Regards, > > Peter. > > > Best wishes, George > > > > On 08.10.19 11:04, Peter Keller wrote: > >> HI Tim, > >> > >> On Mon, 7 Oct 2019, Tim Gruene wrote: > >>> Date: Mon, 07 Oct 2019 23:04:28 +0200 > >>> From: Tim Gruene <tim.gru...@univie.ac.at> > >>> To: Peter Keller <pkel...@globalphasing.com> > >>> Cc: CCP4BB@jiscmail.ac.uk > >>> Subject: Re: [ccp4bb] Shelx and debian 10 > >>> > >>> > >>> > >>> @Peter: are you sure that without 'vsyscall=emulate' linux binaries > >>> need to be > >>> dynamically linked? I would be very surprised if the linux kernel would > >>> disable statically linked binaries. I rather think that the vanilla > >>> versions > >>> of shelx c/d/e (from shelx.uni-goettingen.de) are compiled with an > >>> obsolete > >>> compiler / obsolete compiler options. > >> > >> You're right, I wrote my reply to Bernhard too rapidly, and conflated > >> two separate issues. Debian 10 still supports static binaries of > >> course, but in its default configuration (without vsyscall=emulate), > >> those static binaries must be linked with a version of glibc that > >> doesn't require vsyscalls. OTOH, dynamic binaries don't suffer from > >> this problem, because they can use vDSO provided by the running > >> (rather than the build) system. > >> > >> I think that part of the problem is that traditionally when we want > >> to build portable linux binaries, we tend to build on the oldest > >> distribution that we want to support, relying on backwards > >> compatibility to provide the portability that we are after. We often > >> build statically, because it is a robust way of including all the > >> required libraries and is less fiddly and error-prone than providing > >> them as dynamic libraries. There is also no danger of breakage caused > >> by rogue values of LD_LIBRARY_PATH (which users shouldn't be setting > >> of course, but we have no way of stopping them). The drawback of this > >> approach is that when backwards compatibility is broken, there is no > >> application-level fix. > >> > >> These kinds of problems are rare, but when they do happen the onus is > >> on those of us who distribute binary applications to find solutions. > >> Some sysadmins may have good reasons for being reluctant to change > >> kernel parameters to get third-party applications to work. > >> > >> Regards, > >> Peter. > >> > >>> On Monday, October 7, 2019 5:53:44 PM CEST Peter Keller wrote: > >>>> Dear Bernhard, > >>>> > >>>> We had this issue drawn to our attention last year by an early adopter > >>>> of Debian 10 while it was still in testing. I thought that it was a > >>>> bug, > >>>> and submitted a report accordingly here: > >>>> <https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=889965>. I was told > >>>> that it is not a bug, but a feature ;-) > >>>> > >>>> If you are able, you could try setting the kernel parameter > >>>> vsyscall=emulate. In the longer term, SHELXC/D/E will have to be > >>>> rebuilt > >>>> to support systems where the vsyscall has been disabled. This means > >>>> they > >>>> have to be dynamic executables that include the following in the > >>>> output > >>>> of 'ldd': > >>>> > >>>> % ldd /bin/bash > >>>> linux-vdso.so.1 (0x00007fff50952000) > >>>> .... > >>>> > >>>> All current distros use vDSO, so this shouldn't cause portability > >>>> problems by itself, but handling dynamic executables can be trickier > >>>> than static ones. > >>>> > >>>> For a little more background, see <https://lwn.net/Articles/446528/> > >>>> > >>>> Finally, you have my commiserations: although this change has been a > >>>> long time coming, it hasn't attracted a lot of attention. It was bound > >>>> to catch users of static executables by surprise. > >>>> > >>>> Regards, > >>>> > >>>> Peter. > >>>> > >>>> On 07/10/2019 16:05, Bernhard Rupp wrote: > >>>>> Hi Fellows, > >>>>> > >>>>> we updated to Debian 10 on the local workshop computers, and > >>>>> reinstalled > >>>>> > >>>>> Coot and ccp4. All fine. > >>>>> > >>>>> Problem: Shelxc/d/e/ does not run, and > >>>>> > >>>>> the call exits immediately sans any message. > >>>>> > >>>>> This holds for the binaries included in ccp4 as well as for those > >>>>> from > >>>>> the SHELX site. > >>>>> > >>>>> The executables from CCP4 and SHELX site – same file size, probably > >>>>> same - run fine under Debian 9. > >>>>> > >>>>> I suspect a library problem. > >>>>> > >>>>> Does some kind soul have CDE binaries for Debian 10 to share? > >>>>> > >>>>> Many thx in advance, BR > >>>>> > >>>>> ---------------------------------------------------------------------- > >>>>> ---- > >>>>> > >>>>> -------------- > >>>>> > >>>>> Bernhard Rupp > >>>>> > >>>>> Department of Genetics and Pharmacology > >>>>> > >>>>> Institute of Genetic Epidemiology > >>>>> > >>>>> Medical University Innsbruck > >>>>> > >>>>> Schöpfstrasse 41 > >>>>> > >>>>> A 6020 Innsbruck – Austria > >>>>> > >>>>> +43 (676) 571-0536 > >>>>> > >>>>> bernhard.r...@i-med.ac.at > >>>>> > >>>>> ---------------------------------------------------------------------- > >>>>> ---- > >>>>> > >>>>> -------------- > >>>>> > >>>>> k.k. Hofkristallamt > >>>>> > >>>>> San Diego, CA 92084 > >>>>> > >>>>> 001 (925) 209-7429 > >>>>> > >>>>> b...@ruppweb.org > >>>>> > >>>>> b...@hofkristallamt.org > >>>>> > >>>>> http://www.ruppweb.org/ > >>>>> > >>>>> ---------------------------------------------------------------------- > >>>>> - > >>>>> > >>>>> > >>>>> > >>>>> ---------------------------------------------------------------------- > >>>>> -- > >>>>> > >>>>> > >>>>> To unsubscribe from the CCP4BB list, click the following link: > >>>>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 -- -- Tim Gruene Head of the Centre for X-ray Structure Analysis Faculty of Chemistry University of Vienna Phone: +43-1-4277-70202 GPG Key ID = A46BEE1A ######################################################################## To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
signature.asc
Description: This is a digitally signed message part.
