Automake AM_COND_IF problem: libopts/Makefile.in split personality
I've put together a stripped-down example of a problem currently beguiling Harlan Stenn and myself with Automake used in the NTP reference implementation: http://support.ntp.org/people/hart/am-libopts-subpkg.tar.gz The layout is: am-libopts-subpkg/ configure.ac Makefile.am top.c subproj/ configure.ac Makefile.am sub.c libopts/ m4/ ... Both top.c and sub.c are libopts clients, sharing the single subproj/libopts bundled copy. The problem is that sometimes subproj/libopts/Makefile.in is generated correctly by the subproj machinery, and other times it is generated incorrectly as if it were produced by the top-level configure. The incorrect resulting subproj/libopts/Makefile breaks "make distcheck" using GNU Make, though curiously FreeBSD's make is able to complete distcheck successfully. The incorrect subproj/libopts/Makefile contains "subdir = subproj/libopts" while the working one has "subdir = libopts". The distcheck break happens when subproj/libopts/Makefile attempts to regenerate itself using a missing automake invocation that contains an extra "subproj/" that makes the filename non-existent. In trying to straighten this out, I modified our copy of libopts/m4/libopts.m4, originally from Autogen 5.11.2, to add an optional second argument to LIBOPTS_CHECK which can be used to conditionalize the AC_CONFIG_FILES(libopts-dir/Makefile) under AM_COND_IF. The top-level configure.ac uses this facility, so that the top-level config.status does not generate subproj/libopts/Makefile as it would otherwise. Instead, subproj/config.status generates (from its perspective) libopts/Makefile. The current snag I'm having is with autoreconf -i from the top level. After autoreconf completes, subproj/libopts/Makefile.in is consistently incorrect. This can be worked around with, for example: autoreconf -i -v --no-recursive ; cd subproj ; autoreconf -i -v ; cd .. # in the srcdir After which subproj/libopts/Makefile.in is consistently correct. The workaround is only temporary, as invariably something triggers automake self-rebuilding magic and the miscreant libopts/Makefile.in stops progress again. It appears autoreconf (or automake invoked from autoreconf) is not respecting the conditionalization of AC_CONFIG_FILES(libopts-dir/Makefile) using AM_COND_IF in LIBOPTS_CHECK, so that a recursive autoreconf gives the wrong top_builddir and subdir in subproj/libopts/Makefile.in, while autoreconf from the subproj dir does the right thing. Your thoughts and wisdom are solicited. Can we avoid going back to a separate top-level copy of libopts required by the subpackage? How might I convince in-the-field Autoconf and Automake to stop fighting amongst the packages over which owns subproj/libopts/Makefile? Can you help me in my quest to stop grepping Makefiles before kicking off a "make distcheck"? :) Thanks for your time, Dave Hart
Re: default -g ??!?
On Sun, 21 Nov 2010 10:07:31 +0900 Miles Bader wrote: > MK writes: > > If you say so, then I guess I am imagining things ;) I have never > > given the issue much thought until now, I suppose I need to do a bit > > more research on the issue. > > Indeed, it's often a good idea to do the research _before_ posting > flames and rants... Yes. On the other hand, in my defence, GNU's online docs for make: http://www.gnu.org/software/make/manual/make.html which I sited earlier after searching for "debugging symbols", do not make a mature attempt to explain the issue at all and instead just use inflammatory phrases like "Users who don't mind being helpless can strip the executables later if they wish," and "Devil-may-care users can use the install-strip target to do that". No further explanation. Is this how I'm supposed to learn about Coding Standards? Via jokes and threats of some inexplicable bogey-man? If every piece of documentation related to this issue is just as flippant and patronizing, maybe it's no surprise I had to learn this in a mail list discussion. On the other hand, if there were even one sentence under "target all" justifying the default, such as, "Debugging symbols do not affect performance, and are not loaded into memory during normal use" instead of offhand references to the Devil, etc, this all could have been avoided. $0.02. Anyway, all apologies and thanks everyone for taking interest and a bit of time with me here. I have been a bit paranoid with the packaging because I want to see things done right, and understand the whys and wherefores. Excuse my previous extraneous paragraphs as "debugging symbols" ;) Lucky I left them in or I'd still be in the dark... -- "The angel of history[...]is turned toward the past." (Walter Benjamin)
reword documentation about symbol stripping (was: default -g ??!?)
[ adding bug-standards; this thread is from
http://lists.gnu.org/archive/html/automake/2010-11/msg00114.html ]
* MK wrote on Sun, Nov 21, 2010 at 04:47:48PM CET:
> On Sun, 21 Nov 2010 10:07:31 +0900 Miles Bader wrote:
> >
> > Indeed, it's often a good idea to do the research _before_ posting
> > flames and rants...
>
> Yes. On the other hand, in my defence, GNU's online docs for make:
>
> http://www.gnu.org/software/make/manual/make.html
>
> which I sited earlier after searching for "debugging symbols", do not
> make a mature attempt to explain the issue at all and instead just use
> inflammatory phrases like "Users who don't mind being helpless can
> strip the executables later if they wish," and "Devil-may-care users
> can use the install-strip target to do that". No further
> explanation. Is this how I'm supposed to learn about Coding
> Standards? Via jokes and threats of some inexplicable bogey-man?
Oh well. This thread has been so noisy and unproductive, maybe we
should seize the opportunity to take a bit of good away from it.
Karl, what do you think about this rewording (against the gnulib copy
of make-stds.texi) that makes the text slightly less subjective and
slightly less tongue-in-cheek?
May we have a real name please to credit in the ChangeLog entry?
Thanks,
Ralf
2010-11-21 Ralf Wildenhues
* doc/make-stds.texi (Standard Targets): Reword recommendations
about debug symbols and stripping executables.
Suggested by MK .
diff --git a/doc/make-stds.texi b/doc/make-stds.texi
index 6c83b5d..61e90b6 100644
--- a/doc/make-stds.texi
+++ b/doc/make-stds.texi
@@ -715,8 +715,8 @@ documentation format) files should be made only when
explicitly asked
for.
By default, the Make rules should compile and link with @samp{-g}, so
-that executable programs have debugging symbols. Users who don't mind
-being helpless can strip the executables later if they wish.
+that executable programs have debugging symbols. Executables can be
+stripped later if necessary.
@item install
Compile the program and copy the executables, libraries, and so on to
@@ -724,8 +724,11 @@ the file names where they should reside for actual use.
If there is a
simple test to verify that a program is properly installed, this target
should run that test.
-Do not strip executables when installing them. Devil-may-care users can
-use the @code{install-strip} target to do that.
+Do not strip executables when installing them. This helps eventual
+debugging that may be needed later, and nowadays disk space is cheap
+and dynamic loaders typically ensure debug sections are not loaded during
+normal execution. Users that need stripped binaries may invoke the
+...@code{install-strip} target to do that.
If possible, write the @code{install} target rule so that it does not
modify anything in the directory where the program was built, provided
Re: reword documentation about symbol stripping (was: default -g ??!?)
On Sun, 21 Nov 2010 17:44:10 +0100 Ralf Wildenhues wrote: > Oh well. This thread has been so noisy and unproductive, maybe we > should seize the opportunity to take a bit of good away from it. > > Karl, what do you think about this rewording (against the gnulib copy > of make-stds.texi) that makes the text slightly less subjective and > slightly less tongue-in-cheek? Wow-wee is that refreshing gang, thanks. I do recognize that I could have done more of my own homework here, but: as a neophyte programmer, that is endlessly true (of an endless array of topics -- I think otherwise known as an "infinite regress"), and it is always nice to find something spelled out in a clear, concise manner. Then I can move on quickly to the next conundrum, rather than having to investigate some vague insinuation at every step, potentially wasting other people's time in the process. > May we have a real name please to credit in the ChangeLog entry? I would be "Mark T. Eriksen". -- "The angel of history[...]is turned toward the past." (Walter Benjamin)
on Windows, BUILT_SOURCES does not append .exe
Hey, I use: noinst_PROGRAMS = cmapdump fontdump BUILT_SOURCES = cmapdump fontdump With MSYS, noinst_PROGRAMS is set to cmapdump$(EXEEXT) fontdump$(EXEEXT) but BUILT_SOURCES is set to cmapdump fontdump Is it normal that BUILT_SOURCES does not try to see if they are binaries in some variables ? Vincent Torri
Re: on Windows, BUILT_SOURCES does not append .exe
Hello Vincent, * Vincent Torri wrote on Sun, Nov 21, 2010 at 10:51:53PM CET: > noinst_PROGRAMS = cmapdump fontdump > BUILT_SOURCES = cmapdump fontdump > > With MSYS, noinst_PROGRAMS is set to > > cmapdump$(EXEEXT) fontdump$(EXEEXT) > > but BUILT_SOURCES is set to > > cmapdump fontdump > > Is it normal that BUILT_SOURCES does not try to see if they are > binaries in some variables ? I don't think we've come across an example where it was necessary to put compiled programs into BUILT_SOURCES. For one, that would mean that cross compilation will not work. For another: can you please explain what you are trying to achieve? I'm sure there is another way. Other than that, you're right: automatic $(EXEEXT) appending is not done for BUILT_SOURCES. Thanks, Ralf
Re: on Windows, BUILT_SOURCES does not append .exe
On Sun, 21 Nov 2010, Ralf Wildenhues wrote: Hello Vincent, * Vincent Torri wrote on Sun, Nov 21, 2010 at 10:51:53PM CET: noinst_PROGRAMS = cmapdump fontdump BUILT_SOURCES = cmapdump fontdump With MSYS, noinst_PROGRAMS is set to cmapdump$(EXEEXT) fontdump$(EXEEXT) but BUILT_SOURCES is set to cmapdump fontdump Is it normal that BUILT_SOURCES does not try to see if they are binaries in some variables ? I don't think we've come across an example where it was necessary to put compiled programs into BUILT_SOURCES. For one, that would mean that cross compilation will not work. For another: can you please explain what you are trying to achieve? I'm sure there is another way. I am building a library that is using mupdf. I have autotooled mupdf for my personal project. mupdf is splitted into several libraries: libfitz libdraw libcmaps libfonts libmupdf libcmaps source files are generated by a binary named cmapdump. So I have something like: noinst_PROGRAMS = cmapdump cmapdump_SOURCES = mupdf-0.7/mupdf/cmapdump.c noinst_LTLIBRARIES = libfitz.la libdraw.la libcmaps.la libfonts.la libmupdf.la ... cmap_tounicode_files = \ mupdf-0.7/cmaps/Adobe-CNS1-UCS2 \ mupdf-0.7/cmaps/Adobe-GB1-UCS2 \ mupdf-0.7/cmaps/Adobe-Japan1-UCS2 \ mupdf-0.7/cmaps/Adobe-Korea1-UCS2 libcmaps_la_SOURCES = cmap_tounicode.c cmap_tounicode.c: cmapdump $(cmap_tounicode_files) ./cmapdump $@ $(cmap_tounicode_files) If I don't use BUILT_SOURCES, cmapdump binary is not built before libcmaps, hence cmap_tounicode.c is not created, and compilation of libcmaps fails. Is there another solution ? Other than that, you're right: automatic $(EXEEXT) appending is not done for BUILT_SOURCES. ok. thank you Vincent Torri
Re: default -g ??!?
The reason why users are helpless without debugging symbols is if a program crashes, all they can look at are the machine registers at the state of the crash. This is completely useless for figuring out why the program crashed, or getting help from another hacker to figure out why it crashed.
Re: on Windows, BUILT_SOURCES does not append .exe
* Vincent Torri wrote on Sun, Nov 21, 2010 at 11:14:23PM CET: > noinst_PROGRAMS = cmapdump > cmapdump_SOURCES = mupdf-0.7/mupdf/cmapdump.c > > noinst_LTLIBRARIES = libfitz.la libdraw.la libcmaps.la libfonts.la libmupdf.la > cmap_tounicode_files = \ > mupdf-0.7/cmaps/Adobe-CNS1-UCS2 \ [...] > libcmaps_la_SOURCES = cmap_tounicode.c > cmap_tounicode.c: cmapdump $(cmap_tounicode_files) > ./cmapdump $@ $(cmap_tounicode_files) > > If I don't use BUILT_SOURCES, cmapdump binary is not built before > libcmaps, hence cmap_tounicode.c is not created, and compilation of > libcmaps fails. > > Is there another solution ? Yes: just specify cmapdump$(EXEEXT) as prerequisite to cmap_tounicode.c. Cheers, Ralf
Re: on Windows, BUILT_SOURCES does not append .exe
On Sun, 21 Nov 2010, Ralf Wildenhues wrote: * Vincent Torri wrote on Sun, Nov 21, 2010 at 11:14:23PM CET: noinst_PROGRAMS = cmapdump cmapdump_SOURCES = mupdf-0.7/mupdf/cmapdump.c noinst_LTLIBRARIES = libfitz.la libdraw.la libcmaps.la libfonts.la libmupdf.la cmap_tounicode_files = \ mupdf-0.7/cmaps/Adobe-CNS1-UCS2 \ [...] libcmaps_la_SOURCES = cmap_tounicode.c cmap_tounicode.c: cmapdump $(cmap_tounicode_files) ./cmapdump $@ $(cmap_tounicode_files) If I don't use BUILT_SOURCES, cmapdump binary is not built before libcmaps, hence cmap_tounicode.c is not created, and compilation of libcmaps fails. Is there another solution ? Yes: just specify cmapdump$(EXEEXT) as prerequisite to cmap_tounicode.c. isn't what the line: cmap_tounicode.c: cmapdump $(cmap_tounicode_files) does ? Note that having that rule is not sufficient on linux (that is, even if $(EXEEXT) 'should' (but not 'must', as .exe suffix is not necessary) be added on widnows, it does not work on linux) Vincent Torri
Re: reword documentation about symbol stripping (was: default -g ??!?)
Karl, what do you think about this rewording The second hunk adds real information, so I'll go ahead and install that. The first hunk, though, I just can't agree with, and I feel pretty sure that rms would not approve of such a change either. "Helpless" is a good description of people faced with crashing executables without debugging symbols. Empowering everyone on equal terms is part of the overall GNU philosophy, after all. I personally would not have written it that way in the first place, but given that it is there now, I don't want to simply replace it with bland text, or occupy rms's time with it, either. Best, karl
Re: reword documentation about symbol stripping
You need to remember the original target audience of GNU software was a group of people that wanted to share free software. Most of them were students or researchers that generally built software distributed in source form. Only in the last 10 years has Linux become generally popular. Before that time, it and all the software that ran on it were pretty much relegated to programmers. That being the case, "users" were programmers, and programmers are indeed helpless without debug symbols during a crash - that is, unless you're one of those rare types that loves to dig into a good assembly debug session. In any case, it makes complete sense why the GNU standards were written this way when you understand the history. John On 11/21/2010 12:25 PM, MK wrote: > On Sun, 21 Nov 2010 17:44:10 +0100 > Ralf Wildenhues wrote: >> Oh well. This thread has been so noisy and unproductive, maybe we >> should seize the opportunity to take a bit of good away from it. >> >> Karl, what do you think about this rewording (against the gnulib copy >> of make-stds.texi) that makes the text slightly less subjective and >> slightly less tongue-in-cheek? > Wow-wee is that refreshing gang, thanks. I do recognize that I could > have done more of my own homework here, but: as a neophyte programmer, > that is endlessly true (of an endless array of topics -- I think > otherwise known as an "infinite regress"), and it is always nice to find > something spelled out in a clear, concise manner. Then I can move on > quickly to the next conundrum, rather than having to investigate some > vague insinuation at every step, potentially wasting other people's > time in the process. > >> May we have a real name please to credit in the ChangeLog entry? > I would be "Mark T. Eriksen". >
Re: on Windows, BUILT_SOURCES does not append .exe
On Sun, Nov 21, 2010 at 22:44 UTC, Vincent Torri wrote: > On Sun, 21 Nov 2010, Ralf Wildenhues wrote: >> * Vincent Torri wrote on Sun, Nov 21, 2010 at 11:14:23PM CET: >>> If I don't use BUILT_SOURCES, cmapdump binary is not built before >>> libcmaps, hence cmap_tounicode.c is not created, and compilation of >>> libcmaps fails. >>> >>> Is there another solution ? >> >> Yes: just specify cmapdump$(EXEEXT) as prerequisite to cmap_tounicode.c. > > isn't what the line: > > cmap_tounicode.c: cmapdump $(cmap_tounicode_files) > > does ? Note that having that rule is not sufficient on linux (that is, even > if $(EXEEXT) 'should' (but not 'must', as .exe suffix is not necessary) be > added on widnows, it does not work on linux) In that case you may need to add cmap_tounicode.c to BUILT_SOURCES, leaving cmapdump out of same. Cheers, Dave Hart
Re: reword documentation about symbol stripping
John Calcote writes: > You need to remember the original target audience of GNU software was > a group of people that wanted to share free software. Most of them > were students or researchers that generally built software distributed > in source form. ... > That being the case, "users" were programmers, and programmers are > indeed helpless without debug symbols during a crash - that is, unless > you're one of those rare types that loves to dig into a good assembly > debug session. I think the basic goal is still quite valid though -- the point is not to _require_ that users be programmers, or even to _expect_ them to be, but to _enable_ them to go as far as they want to go. So In cases where some extra power can be granted to the user at little cost, it's good to do so, even if many users never use that power. The concept of system utilities/libraries/etc of being "magic you're not supposed to touch or look at, even if you want to" has been a problem for decades, though the lines have shifted. -Miles -- Politeness, n. The most acceptable hypocrisy.
Re: reword documentation about symbol stripping
k...@freefriends.org (Karl Berry) writes: > I personally would not have written it that way in the first place, but > given that it is there now, I don't want to simply replace it with bland > text, or occupy rms's time with it, either. Yeah, I think there's nothing particularly offensive about that text, but maybe it could be more explanatory -- it addresses an issue that some people may not know about, so maybe it would be good to briefly explain further? -Miles -- "Suppose He doesn't give a shit? Suppose there is a God but He just doesn't give a shit?" [George Carlin]
Re: reword documentation about symbol stripping
On 11/21/2010 07:09 PM, Miles Bader wrote: > John Calcote writes: >> You need to remember the original target audience of GNU software was >> a group of people that wanted to share free software. Most of them >> were students or researchers that generally built software distributed >> in source form. > ... >> That being the case, "users" were programmers, and programmers are >> indeed helpless without debug symbols during a crash - that is, unless >> you're one of those rare types that loves to dig into a good assembly >> debug session. > I think the basic goal is still quite valid though -- the point is not > to _require_ that users be programmers, or even to _expect_ them to be, > but to _enable_ them to go as far as they want to go. So In cases where > some extra power can be granted to the user at little cost, it's good to > do so, even if many users never use that power. > > The concept of system utilities/libraries/etc of being "magic you're not > supposed to touch or look at, even if you want to" has been a problem > for decades, though the lines have shifted. Don't get me wrong - I completely agree with you. I like this approach personally but perhaps that just because I'm a programmer myself. Regardless, there is always value in providing a lot of information (as long as it doesn't get in the way of normal use - and it most definitely doesn't in this case). Additionally, in the open source community, where user feedback is essential, its important to empower users to give you valuable information about your product - such as stack traces with symbols. So even if you don't debug problems yourself, you can hardly give good crash reports without symbol information - especially in a situation where the developer can't know the binary version well enough to apply symbols to numeric addresses correctly every time (as the user may have received his version of your program from any number of sources that may have modified it themselves). John
Re: on Windows, BUILT_SOURCES does not append .exe
* Dave Hart wrote on Mon, Nov 22, 2010 at 12:34:47AM CET: > On Sun, Nov 21, 2010 at 22:44 UTC, Vincent Torri wrote: > > On Sun, 21 Nov 2010, Ralf Wildenhues wrote: > >> * Vincent Torri wrote on Sun, Nov 21, 2010 at 11:14:23PM CET: > >>> If I don't use BUILT_SOURCES, cmapdump binary is not built before > >>> libcmaps, hence cmap_tounicode.c is not created, and compilation of > >>> libcmaps fails. > >>> > >>> Is there another solution ? > >> > >> Yes: just specify cmapdump$(EXEEXT) as prerequisite to cmap_tounicode.c. > > > > isn't what the line: > > > > cmap_tounicode.c: cmapdump $(cmap_tounicode_files) > > > > does ? Note that having that rule is not sufficient on linux (that is, even > > if $(EXEEXT) 'should' (but not 'must', as .exe suffix is not necessary) be > > added on widnows, it does not work on linux) > > In that case you may need to add cmap_tounicode.c to BUILT_SOURCES, > leaving cmapdump out of same. That shouldn't be necessary; you generally only need to list headers there for which automake cannot know that they are generated and needed early. On GNU/Linux, the example code from the second post from Vincent should just work. Vincent, I'm afraid you might just have to debug the makefile (make -d etc); maybe there is a 'cmapdump' file that make finds to be matching? Cheers, Ralf
