Re: [PATCH] Don't install .la files when --no-la-files is used

[Roumen Petrov]

Russ Allbery wrote:

Roumen Petrov <[EMAIL PROTECTED]> writes:

Russ Allbery wrote:



Debian's experience to date is that --as-needed is buggy and breaks a
lot of software, and overall is not a particularly stable solution.
Removing *.la files so that the unneeded shared libraries aren't linked
in the first place works considerably better at the moment.



Could you point by example how libtool add "pile of unnecessary
dependencies" ?  Whit this example could you confirm that libtool add
more(extra) libraries then specified by project authors ?


When you create a libtool library, libtool records every library against
which that library was linked into the *.la file.  If you then link
another shared library against that shared library using libtool, libtool
reads that list of libraries from the *.la file and links the new library
against them as well.


But problem is not in the libtool.


This is not necessary.  For distribution packages, it's actually harmful.


This depend from platform.



To see why, consider some library libfoo which uses readline.  Suppose
that libreadline is linked and installed with libtool, so it has a *.la
file saying that it depends on libncurses.  When you link libfoo against
libreadline using libtool, libtool will also link it against libncurses,
so it now acquires a dependency on libncurses as well.


This is one of the good samples.
Libtool don't add dependency if is not specified by project makefiles, 
i.e. LDFLAGS/LDADD.


Readline has to be build without dependent libraries to allow 
application to link
with readline but with different compatible libraries: ncursesw, 
ncurses, curses, termcap, etc.


But if readline project make rule specify that library has to linked 
with a particular curses library, libtool has to use this rule, when 
link an application with readline.


So I think that projects are responsible how to list library dependencies.


Also this mail thread show that --as-needed flag is not universal 
solution for platforms that allow shared libraries with unresolved 
externals. I quick internet search show problems with ld from binutils 
series < 2.17 and some version of 2.17.x.
Another "--as-needed" problem is that some projects use xxx_LDFLAGS to 
add libraries to applications instead xxx_LDADD in Makefile.am.

Other projects use LDFLAGS instead LIBS in configure scripts.
This is most common problem for package build rules and I could not 
understand why libtool is responsible to fix incorrect project build rules.



Also removing la-files won't help. Another tool that list all package 
libs is pkg-tool. It is so common projects to append all libraries 
returned from pkg-tool :) . And of course the whole list go in la-files.



Libtool is for multi-platform use and list with complete dependent 
libraries help me to build binaries on platform that don't support 
unresolved symbols, as example windows. On this platform libtool don't 
create shared library if flags -no-undefined is not specified. Many 
projects don't set this libtool flags and patches for their build system 
stay open couple of years.
In this case command 'make LDFLAGS="-no-undefined ..."' help a lot to 
build without to touch project files.



I guess that a new flag that order libtool do not link a shared library 
with dependent libraries is more useful then removing la-files.

This flag may overcome --as-needed problems.

In this case is vendor decision how to build a library



[SNIP]


Roumen



___
http://lists.gnu.org/mailman/listinfo/libtool

Re: [PATCH] Don't install .la files when --no-la-files is used

[Dan Nicholson]

On Sat, Nov 8, 2008 at 7:00 AM, Roumen Petrov
<[EMAIL PROTECTED]> wrote:
> Russ Allbery wrote:
>>
>> Roumen Petrov <[EMAIL PROTECTED]> writes:
>>>
>>> Russ Allbery wrote:
>>
 Debian's experience to date is that --as-needed is buggy and breaks a
 lot of software, and overall is not a particularly stable solution.
 Removing *.la files so that the unneeded shared libraries aren't linked
 in the first place works considerably better at the moment.
>>
>>> Could you point by example how libtool add "pile of unnecessary
>>> dependencies" ?  Whit this example could you confirm that libtool add
>>> more(extra) libraries then specified by project authors ?
>>
>> When you create a libtool library, libtool records every library against
>> which that library was linked into the *.la file.  If you then link
>> another shared library against that shared library using libtool, libtool
>> reads that list of libraries from the *.la file and links the new library
>> against them as well.
>
> But problem is not in the libtool.

Yes it is. If you're linking to libfoo, libtool reads libfoo.la and
adds direct links to everything in dependency_libs. Let's say libfoo
depends on libbar and libbaz. You're application ends up directly
linking to libfoo, libbar and libbaz instead of just libfoo.

--
Dan


___
http://lists.gnu.org/mailman/listinfo/libtool

Re: [PATCH] Don't install .la files when --no-la-files is used

[Roumen Petrov]

[SNIP]

But problem is not in the libtool.


Yes it is. If you're linking to libfoo, libtool reads libfoo.la and
adds direct links to everything in dependency_libs. Let's say libfoo
depends on libbar and libbaz. You're application ends up directly
linking to libfoo, libbar and libbaz instead of just libfoo.


No. If project when build a library don't add dependent libraries to 
xxx_LIBADD libtool left dependency_libs empty in la-file.


Roumen


___
http://lists.gnu.org/mailman/listinfo/libtool

Re: [PATCH] Don't install .la files when --no-la-files is used

[Dan Nicholson]

On Sat, Nov 8, 2008 at 9:22 AM, Roumen Petrov
<[EMAIL PROTECTED]> wrote:
> [SNIP]
>>>
>>> But problem is not in the libtool.
>>
>> Yes it is. If you're linking to libfoo, libtool reads libfoo.la and
>> adds direct links to everything in dependency_libs. Let's say libfoo
>> depends on libbar and libbaz. You're application ends up directly
>> linking to libfoo, libbar and libbaz instead of just libfoo.
>
> No. If project when build a library don't add dependent libraries to
> xxx_LIBADD libtool left dependency_libs empty in la-file.

Um, but what if the libraries actually needs to link to other
libraries? Just emptying LIBADD means you're building a broken
library.

--
Dan


___
http://lists.gnu.org/mailman/listinfo/libtool

Re: [PATCH] Don't install .la files when --no-la-files is used

[Russ Allbery]

Roumen Petrov <[EMAIL PROTECTED]> writes:
> Russ Allbery wrote:

>> When you create a libtool library, libtool records every library
>> against which that library was linked into the *.la file.  If you then
>> link another shared library against that shared library using libtool,
>> libtool reads that list of libraries from the *.la file and links the
>> new library against them as well.

> But problem is not in the libtool.

Yes.  It is.

>> This is not necessary.  For distribution packages, it's actually
>> harmful.

> This depend from platform.

Yes.  I said that.  But it's harmful for the platforms that are in most
common use (anything that uses ELF, for example).

>> To see why, consider some library libfoo which uses readline.  Suppose
>> that libreadline is linked and installed with libtool, so it has a *.la
>> file saying that it depends on libncurses.  When you link libfoo
>> against libreadline using libtool, libtool will also link it against
>> libncurses, so it now acquires a dependency on libncurses as well.

> This is one of the good samples.  Libtool don't add dependency if is not
> specified by project makefiles, i.e. LDFLAGS/LDADD.

> But if readline project make rule specify that library has to linked
> with a particular curses library, libtool has to use this rule, when
> link an application with readline.

Only on non-ELF platforms.

> Also removing la-files won't help.

My practical experience disagrees with your theory.

> Libtool is for multi-platform use and list with complete dependent
> libraries help me to build binaries on platform that don't support
> unresolved symbols, as example windows.

Yes, I understand why libtool does this in general.  It's more portable.
However, it causes practical problems for distribution packages, which is
why distributors delete the *.la files to fix those problems in the
absence of a better solution.  Which is where this thread started.

-- 
Russ Allbery ([EMAIL PROTECTED]) 


___
http://lists.gnu.org/mailman/listinfo/libtool

Re: [PATCH] Don't install .la files when --no-la-files is used

[Roumen Petrov]

Russ Allbery wrote:

Roumen Petrov <[EMAIL PROTECTED]> writes:

Russ Allbery wrote:



When you create a libtool library, libtool records every library
against which that library was linked into the *.la file.  If you then
link another shared library against that shared library using libtool,
libtool reads that list of libraries from the *.la file and links the
new library against them as well.



But problem is not in the libtool.


Yes.  It is.


This is not necessary.  For distribution packages, it's actually
harmful.



This depend from platform.


Yes.  I said that.  But it's harmful for the platforms that are in most
common use (anything that uses ELF, for example).


To see why, consider some library libfoo which uses readline.  Suppose
that libreadline is linked and installed with libtool, so it has a *.la
file saying that it depends on libncurses.  When you link libfoo
against libreadline using libtool, libtool will also link it against
libncurses, so it now acquires a dependency on libncurses as well.



This is one of the good samples.  Libtool don't add dependency if is not
specified by project makefiles, i.e. LDFLAGS/LDADD.



But if readline project make rule specify that library has to linked
with a particular curses library, libtool has to use this rule, when
link an application with readline.


Only on non-ELF platforms.


It was old build bug when building readline library on some linux-es. In 
my memory is suse 7.1 but I'm sure that only this particular version was 
affected.
Many other linux verdors build readline without dependent libraries and 
this allow application to be linked against different curses compatible 
libraries.


Roumen





___
http://lists.gnu.org/mailman/listinfo/libtool

Re: [PATCH] Don't install .la files when --no-la-files is used

[Roumen Petrov]

Dan Nicholson wrote:

On Sat, Nov 8, 2008 at 9:22 AM, Roumen Petrov
<[EMAIL PROTECTED]> wrote:

[SNIP]

But problem is not in the libtool.

Yes it is. If you're linking to libfoo, libtool reads libfoo.la and
adds direct links to everything in dependency_libs. Let's say libfoo
depends on libbar and libbaz. You're application ends up directly
linking to libfoo, libbar and libbaz instead of just libfoo.

No. If project when build a library don't add dependent libraries to
xxx_LIBADD libtool left dependency_libs empty in la-file.


Um, but what if the libraries actually needs to link to other
libraries? Just emptying LIBADD means you're building a broken
library.


"building a broken library" please explain.

I just answer to another mail in this mail thread. You state that all 
linux vendor build broken readline library !


Roumen


___
http://lists.gnu.org/mailman/listinfo/libtool

Re: [PATCH] Don't install .la files when --no-la-files is used

[Russ Allbery]

Roumen Petrov <[EMAIL PROTECTED]> writes:

> It was old build bug when building readline library on some linux-es. In
> my memory is suse 7.1 but I'm sure that only this particular version was
> affected.

> Many other linux verdors build readline without dependent libraries and
> this allow application to be linked against different curses compatible
> libraries.

libreadline is linked against libncurses on Debian.

But surely it's obvious that this isn't an interesting argument and has
nothing to do with my point?  It may be that my specific example doesn't
apply on the system that you're looking at right now, but I'm sure that
you can find dozens or hundreds of others without even trying.  Any shared
library that is linked with other shared libraries and is built with
libtool can present this problem.

The best practice for distribution-packaged shared libraries and binaries
is that they should only be linked against shared libraries whose ABIs
they use directly.  They should never be linked against shared libraries
that they use only indirectly, since doing so adds unnecessary
dependencies and unnecessary rebuild work when the SONAMEs of those
additional shared libraries change.  The same issue applies to any large
local software installation.

libtool does not follow this best practice unless you delete the installed
*.la files or use --as-needed (which as a linker flag doesn't seem to be
reliable or robust as yet -- I do apologize if --as-needed referred to
some libtool-specific feature I didn't know about instead of the GNU ld
flag).  One of the problems with the GNU ld --as-needed flag is that it
applies indiscriminately to all linked libraries, even ones that the
application maintainer added explicitly (rather than being added
implicitly by libtool), and sometimes does the wrong thing with libraries
that are actually needed.

The desired behavior of libtool from a distribution perspective would be
to not include dependency libraries from the *.la file in the link on
platforms known to have proper transitive dependency support unless a
static link was requested.  (There would need to be a flag to override
this for the unusual cases where this is required; there are some edge
cases where it's needed, usually involving things like weak symbols or
other corner cases.)

-- 
Russ Allbery ([EMAIL PROTECTED]) 


___
http://lists.gnu.org/mailman/listinfo/libtool

Re: [PATCH] Don't install .la files when --no-la-files is used

[Roumen Petrov]

Russ Allbery wrote:

Roumen Petrov <[EMAIL PROTECTED]> writes:


It was old build bug when building readline library on some linux-es. In
my memory is suse 7.1 but I'm sure that only this particular version was
affected.



Many other linux verdors build readline without dependent libraries and
this allow application to be linked against different curses compatible
libraries.


libreadline is linked against libncurses on Debian.

Which version ?
This is an 7(5?) years old linux bug.



But surely it's obvious that this isn't an interesting argument and has
nothing to do with my point?  It may be that my specific example doesn't
apply on the system that you're looking at right now, but I'm sure that
you can find dozens or hundreds of others without even trying.  Any shared
library that is linked with other shared libraries and is built with
libtool can present this problem.


[SNIP]

The sample is attached.
save attached files:
- pkg1_bootstrap.sh as pkg1/bootstrap.sh
- pkg2_bootstrap.sh as pkg2/bootstrap.sh
- app_bootstrap.sh as app/bootstrap.sh

Adjust prefixes to libtool, automake, autoconf paths..

Details:
- library foo1 for pkg1 export function foo1
- library foo2 for pkg2 export function foo2 and use function foo1
- application use functions foo1 and foo2 and and to link with libraries 
that export them.


Where is libtool bug ?


Roumen
#! /bin/sh

set -e
testPREFIX=/tmp/test/lt/pkg-deplibs-minlibs

libtoolPREFIX=/usr/local/libtool/1.5.26
#libtoolPREFIX=/usr/local/libtool/2.2.2
automakePREFIX=/usr/local/automake/1.10.1
autoconfPREFIX=/usr/local/autoconf/2.61

PATH=${libtoolPREFIX}/bin:$PATH
PATH=${automakePREFIX}/bin:$PATH
PATH=${autoconfPREFIX}/bin:$PATH
export PATH


cat > foo.c <

extern void foo1(long n) {
  fprintf(stderr, "foo1(%ld)\n", n);
}
EOF
cat > Makefile.am < configure.ac <#! /bin/sh

set -e
testPREFIX=/tmp/test/lt/pkg-deplibs-minlibs

libtoolPREFIX=/usr/local/libtool/1.5.26
#libtoolPREFIX=/usr/local/libtool/2.2.2
automakePREFIX=/usr/local/automake/1.10.1
autoconfPREFIX=/usr/local/autoconf/2.61

PATH=${libtoolPREFIX}/bin:$PATH
PATH=${automakePREFIX}/bin:$PATH
PATH=${autoconfPREFIX}/bin:$PATH
export PATH


cat > foo.c <
extern void foo1(long n);

extern void foo2(long n) {
  foo1(n);
  fprintf(stderr, "foo2(%ld)\n", n);
}
EOF
cat > Makefile.am < configure.ac <#! /bin/sh

set -e
testPREFIX=/tmp/test/lt/pkg-deplibs-minlibs

libtoolPREFIX=/usr/local/libtool/1.5.26
#libtoolPREFIX=/usr/local/libtool/2.2.2
automakePREFIX=/usr/local/automake/1.10.1
autoconfPREFIX=/usr/local/autoconf/2.61

PATH=${libtoolPREFIX}/bin:$PATH
PATH=${automakePREFIX}/bin:$PATH
PATH=${autoconfPREFIX}/bin:$PATH
export PATH


test -d build-aux || mkdir build-aux
cat > foo.c <
extern void foo1(long n);
extern void foo2(long n);

int main() {
  foo1(11);
  foo2(22);
  fprintf(stderr, "main()\n");
  return(0);
}
EOF
cat > Makefile.am < configure.ac <___
http://lists.gnu.org/mailman/listinfo/libtool

Re: [PATCH] Don't install .la files when --no-la-files is used

[Russ Allbery]

Roumen Petrov <[EMAIL PROTECTED]> writes:
> Russ Allbery wrote:

>> libreadline is linked against libncurses on Debian.

> Which version ?

readline 5.2-3, ncurses 5.7-2.

> This is an 7(5?) years old linux bug.

I'm very dubious of that assertion.  Applications which use readline but
do not directly use any curses functions should not need to link with any
curses library.

> Details:
> - library foo1 for pkg1 export function foo1
> - library foo2 for pkg2 export function foo2 and use function foo1
> - application use functions foo1 and foo2 and and to link with libraries
> that export them.
>
> Where is libtool bug ?

Your example as described above has nothing to do with the case that we're
talking about since it contains no implicit dependencies.  I'm afraid I
can only conclude that I've not been sufficiently clear and you're not
following my point at all.  I've explained this as clearly as I can,
though.  Maybe someone else can do a better job.

-- 
Russ Allbery ([EMAIL PROTECTED]) 


___
http://lists.gnu.org/mailman/listinfo/libtool

Re: [PATCH] Don't install .la files when --no-la-files is used

[Roumen Petrov]

Russ Allbery wrote:

Roumen Petrov <[EMAIL PROTECTED]> writes:

Russ Allbery wrote:



libreadline is linked against libncurses on Debian.



Which version ?


readline 5.2-3, ncurses 5.7-2.


No,no debian version/release.



This is an 7(5?) years old linux bug.


I'm very dubious of that assertion.  Applications which use readline but
do not directly use any curses functions should not need to link with any
curses library.


Linking readline against ncurses prevent application to link against 
readline against ncursesw and to offer wide characters support.





Details:
- library foo1 for pkg1 export function foo1
- library foo2 for pkg2 export function foo2 and use function foo1
- application use functions foo1 and foo2 and and to link with libraries
that export them.




Where is libtool bug ?


Your example as described above has nothing to do with the case that we're
talking about since it contains no implicit dependencies.  I'm afraid I
can only conclude that I've not been sufficiently clear and you're not
following my point at all.  I've explained this as clearly as I can,
though.  Maybe someone else can do a better job.



But is a package author uncomment line:
#libfoo2_la_LIBADD = -L${testPREFIX}/lib -lfoo1
from pkg2/bootstrap.sh you will see dependency added in la-file.

Roumen








___
http://lists.gnu.org/mailman/listinfo/libtool

Re: [PATCH] Don't install .la files when --no-la-files is used

[Bob Friesenhahn]

I feel like today I just watched the movie "Groundhog Day" another 20 
times.  This topic re-emerges just as often as the one about whether 
list replies should default to the original sender, or to the list.


It seems that there is an issue for Linux distribution maintainers. 
What needs to be done about it so that this topic does not come up so 
often?


Bob
==
Bob Friesenhahn
[EMAIL PROTECTED], http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,http://www.GraphicsMagick.org/



___
http://lists.gnu.org/mailman/listinfo/libtool

Re: [PATCH] Don't install .la files when --no-la-files is used

[Bob Friesenhahn]

On Sat, 8 Nov 2008, Russ Allbery wrote:


pkg-config supports having separate dependency lists for static linking
and shared linking, which seems to adequately address this problem (if
that feature is actually used; the documentation last I looked was a bit
lacking and a lot of pkg-config *.pc providers aren't aware of it).
libtool could do something similar, but there would have to be some way to
tell libtool which libraries are required for shared linking and which are
required only for static linking.


Quite a lot can known from .la files but it is apparent that .la files 
are now spontaneously deleted.


It is really quite a burden on the developer to try to understand what 
is needed for static and shared linking.  Many Linux 
application/library authors are not even aware of all the libraries 
they are using and it might not be possible to fully know without 
using diagnostic tools, special environment variables like 
LD_BIND_NOW, or a particular execution mode of the software. 
Dependencies are not always obvious and is is possible for "pass 
through" dependencies to be introduced (stealthy hard dependencies) 
that the developer is not aware of.


Bob
==
Bob Friesenhahn
[EMAIL PROTECTED], http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,http://www.GraphicsMagick.org/



___
http://lists.gnu.org/mailman/listinfo/libtool

Re: [PATCH] Don't install .la files when --no-la-files is used

[Russ Allbery]

Bob Friesenhahn <[EMAIL PROTECTED]> writes:

> Quite a lot can known from .la files but it is apparent that .la files
> are now spontaneously deleted.

Hm, I must admit that I generally find them useless compared to reading
readelf -a output, but I'm not the normal user.  :)

> It is really quite a burden on the developer to try to understand what
> is needed for static and shared linking.  Many Linux application/library
> authors are not even aware of all the libraries they are using and it
> might not be possible to fully know without using diagnostic tools,
> special environment variables like LD_BIND_NOW, or a particular
> execution mode of the software. Dependencies are not always obvious and
> is is possible for "pass through" dependencies to be introduced
> (stealthy hard dependencies) that the developer is not aware of.

This is all true in the general case, but most of the cases where clients
of a library need to link against that library's dependencies are weird
edge cases.  My guess is that only linking with the libraries whose ABIs
you call directly works 95% of the time on modern ELF platforms, and
linking with -pthread where appropriate is the only additional bit
required in another 4% or more of the cases.

-- 
Russ Allbery ([EMAIL PROTECTED]) 


___
http://lists.gnu.org/mailman/listinfo/libtool

Re: [PATCH] Don't install .la files when --no-la-files is used

[Russ Allbery]

Bob Friesenhahn <[EMAIL PROTECTED]> writes:

> It seems that there is an issue for Linux distribution maintainers. What
> needs to be done about it so that this topic does not come up so often?

Well, my preference would be to implement the change to libtool described
in my previous message, but since that had been proposed many years ago
and was not done, I assume that there's some reason why it's a bad idea
that I'm not aware of.

pkg-config supports having separate dependency lists for static linking
and shared linking, which seems to adequately address this problem (if
that feature is actually used; the documentation last I looked was a bit
lacking and a lot of pkg-config *.pc providers aren't aware of it).
libtool could do something similar, but there would have to be some way to
tell libtool which libraries are required for shared linking and which are
required only for static linking.

-- 
Russ Allbery ([EMAIL PROTECTED]) 


___
http://lists.gnu.org/mailman/listinfo/libtool

Re: [PATCH] Don't install .la files when --no-la-files is used

[Charles Wilson]

Roumen Petrov wrote:

> Linking readline against ncurses prevent application to link against
> readline against ncursesw and to offer wide characters support.

Note that this is only even possible on a system with lazy binding. For
windows, shared libraries cannot have any undefined symbols at link
time, which means that the libreadline DLL (for instance) SHALL be bound
to some particular instance of libcurses. There is no choice.

If you try to exploit this in your build system, to allow "floating"
readline...that will not be a portable construct.  libtool is about
portability, not necessarily about squeezing every possible flexibility
out of ELF lazy binding.

--
Chuck


___
http://lists.gnu.org/mailman/listinfo/libtool