BOLETIN IESALC-UNESCO INVITACION
Title: Sin título-1 ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: Speeding up libtool
Hi Robert, * Robert Ögren wrote on Mon, Mar 14, 2005 at 11:11:00PM CET: > > I looked through the TODO thread from 2004-11 and saw some talk about > speeding up libtool. I'm particularly interested in improved compilation > speed on Cygwin (Windows). Is there any progress on this? For the HEAD branch of Libtool, I have made some changes that cause it to fork less often. For the particular bad case I was looking at (linking against ~500 objects), I got it from about 11 sec to about 5 sec, both of which include 2.5 sec of time spent actually linking (all data cached) on a fast linux/x86_64 machine. The improvements also benefit the general case, but are most likely less pronounced, even if Cygwin should be the candidate to benefit most. If you have resources/time to test, it would be nice to see a comparison for your projects. For best results on Cygwin, you should probably configure (with libtool HEAD) like CONFIG_SHELL=/bin/ash lt_ECHO='printf %s' /bin/ash configure [...] (replace /bin/ash with the path to Cygwin's Almquist shell clone). I have wanted to get this to work right automatically, but have not had time to look into it (the shell/ECHO selection is mixed Libtool/Autoconf code and focuses on portability more than speed -- there have been old and quite buggy ash variants on some systems). The speedup between using bash and using ash should be quite noticeable on Cygwin! > A while ago I created a temporary solution for myself which caches the > commands executed by libtool 1.5 to speed up the compilation process. It > is a small Perl script that is run in place of libtool and that > maintains a simple database of libtool command line => resulting > commands by running the real libtool any time a command is not already > in the database and parsing its standard out and storing the commands > and .la files. It understands very little of what libtool actually does, > the speedup is achieved by invoking libtool less often. This is a very nice idea! The hairy stuff is in the details, however. > By replacing the name of the source file and the corresponding output > file with placeholders if they match it usually only requires a few > libtool invocations for compiling many source files. The link commands > are usually unique but at least the second time a source tree is rebuilt > they are already cached. Ah, ok. For use within the same $buildtree and compilation only this should be mostly fine (save locking, as mentioned in your README). Linking is a problem, though: shell wrappers break, as do generating symbol lists. Can't think of more right away, but I'm sure there are many more corner cases which break. > This is aimed at software developers like myself who build projects from > CVS and frequently modify some source files, compile and relink. > > The code was not written with portability in mind and certainly has bugs > and quirks, but works well for me and should be possible to port to > other platforms than Cygwin. The speedup might not be very large on > those platforms however. But if anyone tries, please tell me how it worked. How much is the actual speedup in numbers? How much with the HEAD branch of Libtool? Iff this gives a decent speedup, I think the idea could belong in Libtool proper. That is, if you and the other Libtool developers are so inclined. Thinking out loud more: if this were created as output of `configure' (possibly acting upon some configure and/or libtoolize switch), it would also be possible to make libtool-cache (more) portable and much more foolproof. For example, if configure decides locking is necessary, the cache could be bypassed. $objdir, PATH_SEPARATOR, etc. would be trivially available, the `libtoolcacheize' beast could be killed.. We could add a marker whenever libtool does non-cacheable things so that the cache would be bypassed then. Or just let libtool output "everything" it does so that even writing wrappers would work. Maybe into a separate file descriptor. Most likely the cache would have to be invalidated whenever config.status changes. And it would have to be $top_builddir-local, so you'd have several caches for several subpackages. But for all of this to work reasonably, I think libtool needs to be audited so it outputs the executed commands very closely. I'd also think it would be possible to rewrite libtool-cache in portable shell, but then remaining speedup would have to be evaluated (one `perl' exec is, after all, not cheap). I'm pretty sure this would be more useful as a separate script than being directly integrated into the `libtool' script, since many shells need a long time for the initial parsing of `libtool'. Minor nit for your current script: Why do you remove comments from the created .la files? I don't see a benefit here. What do you think? What do the other Libtool developers think? Note that it's probably not worth investing work unless there is a demonstrable speedup involved. Regards,
Re: Speeding up libtool
Ralf Wildenhues writes: > Linking is a problem, though: shell wrappers break, I have never liked (or understood...) libtool's shell wrappers or its relinking on Win32. I always use --disable-static, I always run a "make install", and make sure the bin directory of the installation location is in my PATH, before any "make check". And I always comment out the "need_relink=yes" in any ltmain.sh that I come across ;-) This means shell wrappers won't be produced. Or am I completely confused? > How much is the actual speedup in numbers? Very significant. I don't have my old machine (450 MHz PIII running Win2k) around any longer, but libtool-cache made it feel like a totally new machine. A rough guess would five times faster builds of stuff like GLib or GTK. --tml ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: Speeding up libtool
Hi Tor, * Tor Lillqvist wrote on Tue, Mar 15, 2005 at 10:32:41AM CET: > Ralf Wildenhues writes: > > Linking is a problem, though: shell wrappers break, > > I have never liked (or understood...) libtool's shell wrappers or its > relinking on Win32. I always use --disable-static, I always run a > "make install", and make sure the bin directory of the installation > location is in my PATH, before any "make check". And I always comment > out the "need_relink=yes" in any ltmain.sh that I come across ;-) This > means shell wrappers won't be produced. Or am I completely confused? Well, the feeling towards shell wrappers and their need in general are different matters. :) It might be the case that libtool actually creates wrappers in situations where they are not strictly necessary. This has been reported before, and I would be happy to take patches which improve on the situation while not compromising functionality and portability. But in general, if you link against uninstalled libraries, you will have to relink upon `make install'. In general, you then also have to have add some linker hackery upon execution of the uninstalled binary, so that it will actually pick up the uninstalled libraries versus older installed versions. If that is not the case on some architectures, we could optimize here. Honestly, I don't know off the head whether Cygwin falls into that category (I think win uses PATH for searching libraries). > > How much is the actual speedup in numbers? > > Very significant. I don't have my old machine (450 MHz PIII running > Win2k) around any longer, but libtool-cache made it feel like a > totally new machine. A rough guess would five times faster builds of > stuff like GLib or GTK. Now it would _really_ be interesting to see how much speedup is left with Libtool HEAD using ash, vs. libtool-cache with Libtool HEAD using ash. And if there are bottlenecks left that can /easily/ be improved. Regards, Ralf ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: PGI and C++ templates
On Mar 14, 2005, at 1:49 PM, Ralf Wildenhues wrote:
I've long since added my own m4 to my configure scripts to see if the
C++ compiler uses a template subdirectory, and if so, add it to
CLEANFILES.
Can you be bothered to share it with us? Has it hung enough to be
considered for Automake?
Sure -- it's attached.
I don't know if it's "good enough" to be considered for AM; it's really
a shotgun approach (I don't know anything about their coding
standards):
- make a temp dir, cd into it
- compile some C++ stuff that contains templates
- remove all known files
- see what files/dirs are left
There's probably other ways to do it; I have dim recollections of this
macro coming into existence around 3am one night... :-)
(looking at this code again -- it's been years since I've examined it
-- there's a *lot* that would need to be cleaned up and generalized; I
am realizing how much I didn't know about AC when I wrote that macro --
it's embarrassing! ;-) The shotgun approach idea can be extracted from
this code, if it's useful -- that's probably about it)
Has something changed that I no longer need to do this?
We might want to have something like it.
That would be fantastic.
[Template.dir stuff snipped]
I'll let the PGI people reply to this stuff...
Does anybody have some real-world code using C++/templates/libraries
that I (or, even better, you) can test with (and that is not too
complicated to set up)?
'fraid not. We only needed C libraries to build with LAM/MPI -- the
C++ templating issue came up after that was solved. Sorry! :-(
--
{+} Jeff Squyres
{+} [EMAIL PROTECTED]
{+} http://www.lam-mpi.org/
cxx_find_template_repository.m4
Description: Binary data
___
http://lists.gnu.org/mailman/listinfo/libtool
Re: PGI and C++ templates
Sorry for the self-reply,
* Ralf Wildenhues wrote on Mon, Mar 14, 2005 at 07:49:00PM CET:
>
> This is a status update on "libtool/pgi/C++ with templates".
Now here's an improved patch against branch-2-0. Should work for both
static and shared uninstalled libraries plus programs linked against
them, and seems to also work when relinking is required upon
installation. It works with the example Markus provided a while ago.
Turns out pgCC is smart enough to recreate all that is needed with the
correct PICness all by itself, so we don't even need to adjust
Template.dir. Nice compiler work!
What's missing is correct `make clean'ing, which will be addressed later
in reply to Jeff's macro.
Testing would be *great*, as would be reviewing.
Cheers,
Ralf
> ,-- ***experimental*** patch for PGI + templates ---
>
> * m4/libtool.m4 (_LT_COMPILER_C_O): The compiler may create
> subdirectories which we have to remove here.
>
> * m4/libtool.m4 (_LT_LINKER_SHLIBS): New TAGDECL $prelink_cmds
> intended for compilation of template instantiations for program
> outputs.
> * m4/libtool.m4 (_LT_LANG_CXX_CONFIG) [ linux ]: Enable template
> compilation for Portland pgCC in prelink_cmds and old_archive_cmds.
> * config/ltmain.m4sh (func_mode_link): execute $prelink_cmds if
> nonempty.
Index: config/ltmain.m4sh
===
RCS file: /cvsroot/libtool/libtool/config/ltmain.m4sh,v
retrieving revision 1.1.2.47
diff -u -r1.1.2.47 ltmain.m4sh
--- config/ltmain.m4sh 12 Mar 2005 08:50:45 - 1.1.2.47
+++ config/ltmain.m4sh 15 Mar 2005 16:55:37 -
@@ -5530,6 +5530,18 @@
func_generate_dlsyms "$outputname" "@PROGRAM@" "no"
+ # template prelinking step
+ if test -n "$prelink_cmds"; then
+ cmds=$prelink_cmds
+ save_ifs="$IFS"; IFS='~'
+ for cmd in $cmds; do
+ IFS="$save_ifs"
+ eval cmd=\"$cmd\"
+ $show "$cmd"
+ $run eval "$cmd"
+ test $? -eq 0 || exit $?
+ done
+ fi
wrappers_required=yes
case $host in
Index: m4/libtool.m4
===
RCS file: /cvsroot/libtool/libtool/m4/libtool.m4,v
retrieving revision 1.125.2.36
diff -u -r1.125.2.36 libtool.m4
--- m4/libtool.m4 14 Mar 2005 15:03:24 - 1.125.2.36
+++ m4/libtool.m4 15 Mar 2005 16:55:37 -
@@ -1482,7 +1482,7 @@
test -d out/ii_files && $RM out/ii_files/* && rmdir out/ii_files
$RM out/* && rmdir out
cd ..
- rmdir conftest
+ $RM -r conftest
$RM conftest*
])
_LT_TAGDECL([compiler_c_o], [lt_cv_prog_compiler_c_o], [1],
@@ -3112,7 +3112,7 @@
_LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC'
_LT_TAGVAR(lt_prog_compiler_static, $1)='-static'
;;
- pgCC)
+ pgCC*)
# Portland Group C++ compiler
_LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,'
_LT_TAGVAR(lt_prog_compiler_pic, $1)='-fpic'
@@ -4468,6 +4468,8 @@
[Symbols that should not be listed in the preloaded symbols])
_LT_TAGDECL([], [include_expsyms], [1],
[Symbols that must always be exported])
+_LT_TAGDECL([], [prelink_cmds], [2],
+[Commands necessary for linking programs (against libraries) with
templates])
dnl FIXME: Not yet implemented
dnl _LT_TAGDECL([], [thread_safe_flag_spec], [1],
dnl[Compiler flag to generate thread safe objects])
@@ -5191,10 +5193,25 @@
_LT_TAGVAR(export_dynamic_flag_spec, $1)='${wl}--export-dynamic'
_LT_TAGVAR(whole_archive_flag_spec,
$1)='${wl}--whole-archive$convenience ${wl}--no-whole-archive'
;;
- pgCC)
+ pgCC*)
# Portland Group C++ compiler
-_LT_TAGVAR(archive_cmds, $1)='$CC -shared $predep_objects $libobjs
$deplibs $postdep_objects $compiler_flags ${wl}-soname ${wl}$soname -o $lib'
-_LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $predep_objects
$libobjs $deplibs $postdep_objects $compiler_flags ${wl}-soname ${wl}$soname
${wl}-retain-symbols-file ${wl}$export_symbols -o $lib'
+ _LT_TAGVAR(prelink_cmds, $1)='tpldir=Template.dir~
+ rm -rf $tpldir~
+ $CC --prelink_objects --instantiation_dir $tpldir $objs $libobjs
$compile_deplibs~
+ compile_command="$compile_command `ls $tpldir/*.o 2>/dev/null |
tr '"'\n' ' '"'`"'
+_LT_TAGVAR(old_archive_cmds, $1)='tpldir=Template.dir~
+ rm -rf $tpldir~
+ $CC --prelink_objects --instantiation_dir $tpldir
$oldobjs$old_deplibs~
+ $AR $AR_FLAGS $oldlib$oldobjs$old_deplibs `ls $tpldir/*.o
2>/dev/null | tr '"'\n' ' '"'`~
+ $RANLIB $oldlib'
+_LT_TAGVAR(archive_cmds, $1)='tpldir=Template.dir~
+ rm -rf $tpldir~
+ $CC --prelink_objects --instantiation_dir $tpldir $predep_objects
$libobjs $deplibs $postdep_objects~
+
Re: Speeding up libtool
Ralf Wildenhues wrote:
Hi Robert,
Hi Ralf,
For the HEAD branch of Libtool, I have made some changes that cause it
to fork less often. For the particular bad case I was looking at
(linking against ~500 objects), I got it from about 11 sec to about 5
sec, both of which include 2.5 sec of time spent actually linking (all
data cached) on a fast linux/x86_64 machine.
The improvements also benefit the general case, but are most likely less
pronounced, even if Cygwin should be the candidate to benefit most.
If you have resources/time to test, it would be nice to see a comparison
for your projects. For best results on Cygwin, you should probably
configure (with libtool HEAD) like
CONFIG_SHELL=/bin/ash lt_ECHO='printf %s' /bin/ash configure [...]
Interesting, I'll have to find some time to try that soon.
By replacing the name of the source file and the corresponding output
file with placeholders if they match it usually only requires a few
libtool invocations for compiling many source files. The link commands
are usually unique but at least the second time a source tree is rebuilt
they are already cached.
Ah, ok. For use within the same $buildtree and compilation only this
should be mostly fine (save locking, as mentioned in your README).
Linking is a problem, though: shell wrappers break, as do generating
symbol lists. Can't think of more right away, but I'm sure there are
many more corner cases which break.
Shell wrappers? If you mean the shell scripts that are created to set up
PATH etc. before running an uninstalled EXE from .libs, they are also
saved by libtool-cache by using the same mechanism that is used to save
the .la{,i} files (serialize_ltfile) and it seems to work.
I just open the generated files, transform them into a set of echo
statements that recreate the files, and splice them into the command
sequence that Libtool outputs. The corresponding wrapper .exe file that
is generated by Libtool when running on Cygwin is just copied from a
precompiled version.
I don't handle the generation of symbol lists apart from ignoring the
line that Libtool prints when it's generating it, so that is probably
broken. I haven't needed it yet...
How much is the actual speedup in numbers?
I will have to get back to you after doing some proper benchmarks, but
you can find at least some numbers here:
http://mail.gnome.org/archives/gtk-devel-list/2005-March/msg00062.html
This is on a quite fast machine with lots of RAM for disk cache, however
I did leave the antivirus software running (although set to ignore the
source tree). I should turn that off and see if it makes any difference.
> How much with the HEAD branch of Libtool?
Sorry, I must admit I haven't actually tried it yet, although I did
check out the source from anoncvs recently. I'll try it soon.
Iff this gives a decent speedup, I think the idea
could belong in Libtool proper. That is, if you and the other Libtool
developers are so inclined.
I certainly wouldn't mind this functionality being included in Libtool
(or perhaps just becoming unnecessary if Libtool is rewritten in C or
bytecode). If anyone wants to they should feel free to improve my crude
code or just take the useful ideas and reimplement them properly. I
can't promise that I'll have time myself to do that implementation.
Thinking out loud more: if this were created as output of `configure'
(possibly acting upon some configure and/or libtoolize switch), it would
also be possible to make libtool-cache (more) portable and much more
foolproof. For example, if configure decides locking is necessary, the
cache could be bypassed. $objdir, PATH_SEPARATOR, etc. would be
trivially available, the `libtoolcacheize' beast could be killed..
That would be great.
We could add a marker whenever libtool does non-cacheable things so that
the cache would be bypassed then. Or just let libtool output
"everything" it does so that even writing wrappers would work.
Maybe into a separate file descriptor.
Yes, that would be nice. Right now I remove some messages that Libtool
writes, and try to detect warning messages and skip caching in such
cases. But it is not perfect.
Most likely the cache would have to be invalidated whenever
config.status changes. And it would have to be $top_builddir-local, so
you'd have several caches for several subpackages.
But for all of this to work reasonably, I think libtool needs to be
audited so it outputs the executed commands very closely.
Agreed.
I'd also think it would be possible to rewrite libtool-cache in portable
shell, but then remaining speedup would have to be evaluated (one `perl'
exec is, after all, not cheap). I'm pretty sure this would be more
useful as a separate script than being directly integrated into the
`libtool' script, since many shells need a long time for the initial
parsing of `libtool'.
Also agree.
Minor nit for your current script: Why do you remove comments from the
created .la files? I don't see a benefit here.
That is ju
Re: Speeding up libtool
Ralf Wildenhues wrote: for your projects. For best results on Cygwin, you should probably configure (with libtool HEAD) like CONFIG_SHELL=/bin/ash lt_ECHO='printf %s' /bin/ash configure [...] (replace /bin/ash with the path to Cygwin's Almquist shell clone). That path would be /bin/sh, so it's possible libtool is using it already. Max. ___ http://lists.gnu.org/mailman/listinfo/libtool
enlarge girth and length
LONGZ capsules feature an exclusive blend of natural supplements that have been proven to increase and improve circulation to the groin. During arousal, you will experience harder and stronger erections Forget about your partner faking her orgasm or not being able to please her. You will be able to penetrate deeper so your partner will experience more pleasure as well as multiple orgasms during sexual intercourse. 86% of women surveyed said that they would like their partner to be more 'full' sexually. Check out the only Male Enhancement formula with a free DVD I want to let u guys know that I have seen over 1 inch in length increase since I started taking ur system. The exercises are easy too. I use them both and this is awesome. Clancy, Spokane not for you, then use link above Then imagine your condition were you able fully to control this great element, replied the other, gravely.The weaknesses and privations of mankind would be converted into power and luxury That's true, Mr ___ http://lists.gnu.org/mailman/listinfo/libtool
