fix inadequate Windows implementation of tmpfile
Under Windows, the tmpfile function is defined to always create its temporary file in the root directory. Most users don't have permission to do that, so it will often fail. I'm proposing a "tmpfile" module that detects this situation and replaces tmpfile with a better implementation. The implementation I provide is Windows-specific, because the common Unix implementation of temporary files (open then immediately unlink) doesn't work on Windows, which doesn't permit deleting open files. I using the clean-temp module instead, as a more portable solution, but then I needed to interpose upon fclose (and perhaps close as well) to delete the temporary file when it was closed, which quickly turned into a mess. Using the Windows "delete on close" _O_TEMPORARY flag is a simpler solution. I've tested this solution on mingw/Wine and it works well there. I haven't tested it on a real Windows machine because I don't have easy access to one. All feedback is appreciated. Index: MODULES.html.sh === RCS file: /cvsroot/gnulib/gnulib/MODULES.html.sh,v retrieving revision 1.194 diff -u -p -r1.194 MODULES.html.sh --- MODULES.html.sh 15 Feb 2007 03:07:04 - 1.194 +++ MODULES.html.sh 15 Feb 2007 15:18:20 - @@ -1539,6 +1539,16 @@ func_all_modules () func_module pagealign_alloc func_end_table + element="Input/Output " + element=`printf "%s" "$element" | sed -e "$sed_lt" -e "$sed_gt"` + func_section_wrap ansic_enh_stdio + func_wrap H3 + func_echo "$element" + + func_begin_table + func_module tmpfile + func_end_table + element="Sorting functions " element=`printf "%s" "$element" | sed -e "$sed_lt" -e "$sed_gt"` func_section_wrap ansic_enh_stdlib_sorting Index: lib/tmpfile.c === RCS file: lib/tmpfile.c diff -N lib/tmpfile.c --- /dev/null 1 Jan 1970 00:00:00 - +++ lib/tmpfile.c 15 Feb 2007 15:18:21 - @@ -0,0 +1,89 @@ +/* Replacement for inadequate Windows implementation of tmpfile. + Copyright (C) 2007 Free Software Foundation, Inc. + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 2, or (at your option) + any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software Foundation, + Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. */ + +/* Written by Ben Pfaff. */ + +#include + +#include +#include +#include +#include +#include +#include + +#include "pathmax.h" +#include "xallocsa.h" +#include "tempname.h" +#include "tmpdir.h" + +#define WIN32_LEAN_AND_MEAN +#include + +FILE * +tmpfile (void) +{ + char *dir; + char *xtemplate; + DWORD retval; + size_t len; + int fd; + FILE *file = NULL; + + dir = (char *) xallocsa (PATH_MAX); + xtemplate = (char *) xallocsa (PATH_MAX); + + /* Find Windows temporary file directory. + We provide this as the directory argument to path_search + because Windows defines P_tmpdir to "\\" and will therefore + try to put all temporary files in the root (unless $TMPDIR + is set). */ + retval = GetTempPath (PATH_MAX, dir); + if (retval == 0 || retval >= PATH_MAX - 1) +goto done; + + if (path_search (xtemplate, PATH_MAX, dir, NULL, true)) +goto done; + len = strlen (xtemplate); + + do +{ + strcpy (&xtemplate[len - 6], "XX"); + if (gen_tempname (xtemplate, GT_NOCREATE)) +goto done; + + fd = _open (xtemplate, + _O_BINARY | _O_CREAT | _O_TEMPORARY | _O_EXCL | _O_RDWR, + _S_IREAD | _S_IWRITE); +} + while (fd < 0 && errno == EEXIST); + if (fd < 0) +goto done; + + file = _fdopen (fd, "w+b"); + if (file == NULL) +{ + int save_errno = errno; + _close (fd); + errno = save_errno; +} + + done: + freesa (xtemplate); + freesa (dir); + return file; +} Index: m4/tmpfile.m4 === RCS file: m4/tmpfile.m4 diff -N m4/tmpfile.m4 --- /dev/null 1 Jan 1970 00:00:00 - +++ m4/tmpfile.m4 15 Feb 2007 15:18:21 - @@ -0,0 +1,39 @@ +# Check whether tmpfile will work for ordinary users who do not have +# permissions to write in the root directory. + +# Copyright (C) 2007 Free Software Foundation, Inc. +# This file is free software; the Free Software Foundation +# gives unlimited permission to copy and/or distribute it, +# with or without modifications, as long as this notice is preserved. + +# Written by Ben
predefined preprocessor macros on Windows platforms
Hi, This is the yearly FAQ posting about the symbols defined on C/C++ compilations on Windows. 1) The compiler indicators: - Mingw, Cygwin: __GNUC__ - MSVC: _MSC_VER - Borland:__TURBOC__, __BORLANDC__ 2) The CPU indicators for x86: - Mingw, Cygwin: _X86_, __i386__ - MSVC, Borland: _M_IX86 The CPU indicators for x86_64: - Mingw, Cygwin: __x86_64__ - MSVC: _M_X64 3) Operating system: - Cygwin default: __CYGWIN32__ - Cygwin when the installer wants to use native Woe32 API (option -mwin32): __CYGWIN32__, _WIN32 - Mingw: _WIN32 - MSVC: _WIN32 - Borland:__WIN32__ So, to test whether native Woe32 API is available, use defined _WIN32 || defined __WIN32__ And to test whether native Woe32 API should be used when there is also an equivalent POSIX API, use (defined _WIN32 || defined __WIN32__) && ! defined __CYGWIN32__ Bruno References: [1] http//predef.sourceforge.net/ [2] GNU clisp's lispbibl.d
Re: fix inadequate Windows implementation of tmpfile
Bruno Haible <[EMAIL PROTECTED]> writes: > So, to test whether native Woe32 API is available, use > > defined _WIN32 || defined __WIN32__ Ben Pfaff <[EMAIL PROTECTED]> writes: > + [[#ifdef __WIN32__ > +choke me > +#endif ]])], Oops. Thanks Bruno, I'll fix up that part of my patch. -- Ben Pfaff [EMAIL PROTECTED] http://benpfaff.org
Re: fix inadequate Windows implementation of tmpfile
Hello Ben, Thanks for proposing this tmpfile override, and for bringing _O_TEMPORARY to our attention. However, in gnulib we usually try to not override a function unless it's clearly broken. (We didn't do that with 'malloc' and 'realloc' for some time, and it caused hassles, because some modules were expecting a glibc compatible malloc and some weren't... Likewise for strstr more recently.) Here, it's debatable whether the tmpfile() function in MSVCRT is broken: On one hand, ISO C 99 says "It should be possible to open at least TMP_MAX temporary files during the lifetime of the program..." which isn't fulfilled, as you say, if the user doesn't have the permissions. (Btw, such situations can also occur on Unix! KDE becomes unusable when /tmp is full.) On the other hand, Microsoft's doc of tmpfile http://msdn2.microsoft.com/en-us/library/x8x7sakw(VS.80).aspx and of tmpfile_s http://msdn2.microsoft.com/en-us/library/b3dz6009(VS.80).aspx explicitly says "The temporary file is created in the root directory." So, your replacement doesn't fulfill the Microsoft doc. Therefore, I think you should choose a different name for your function, since you want it to behave differently than the documented behaviour of tmpfile() on Windows. - The _O_TEMPORARY flag is interesting. It appears to be portable to all versions of Windows since Woe95. What can you say about its reliability? The native tmpfile() uses this flag as well. Is a file created with tmpfile() on Windows deleted when you interrupt the program with Ctrl-C / Ctrl-Break? - > + [[#ifdef __WIN32__ This #if doesn't take into account several Windows platforms. See the other mail of today. - > I using the clean-temp module instead, as a > more portable solution, but then I needed to interpose upon > fclose (and perhaps close as well) to delete the temporary file > when it was closed, which quickly turned into a mess. Can you elaborate, please? clean-temp guarantees that the file will be removed in most cases. You want it removed earlier, as soon as the fclose happens? To save disk space? Bruno
Re: predefined preprocessor macros on Windows platforms
Bruno Haible wrote: > 3) Operating system: > - Cygwin default: __CYGWIN32__ > - Cygwin when the installer wants to use native Woe32 API (option -mwin32): > __CYGWIN32__, _WIN32 This should be __CYGWIN__ and not __CYGWIN32__. The former is the canonical name, the latter is an outdated name that is still defined for compatibility but should not be used. Brian
Re: predefined preprocessor macros on Windows platforms
Brian Dessent wrote: > This should be __CYGWIN__ and not __CYGWIN32__. Yes, you are right. My bad. Here is the corrected summary: 1) The compiler indicators: - Mingw, Cygwin: __GNUC__ - MSVC: _MSC_VER - Borland:__TURBOC__, __BORLANDC__ 2) The CPU indicators for x86: - Mingw, Cygwin: _X86_, __i386__ - MSVC, Borland: _M_IX86 The CPU indicators for x86_64: - Mingw, Cygwin: __x86_64__ - MSVC: _M_X64 3) Operating system: - Cygwin default: __CYGWIN__ - Cygwin when the installer wants to use native Woe32 API (option -mwin32): __CYGWIN__, _WIN32 - Mingw: _WIN32, __MINGW32__ - MSVC: _WIN32 - Borland:__WIN32__ So, to test whether native Woe32 API is available, use defined _WIN32 || defined __WIN32__ And to test whether native Woe32 API should be used when there is also an equivalent POSIX API, use (defined _WIN32 || defined __WIN32__) && ! defined __CYGWIN__ Bruno
Re: more m4 underquotations
Eric Blake wrote: > there is too much risk that a user > will use those characters in his local encoding, but not the > byte sequence that M4 recognizes based on a different > encoding of those characters. Indeed. I thought this problem could be solved by having every file declare its quote characters at the beginning (and reset then to [ ] at the end), but this doesn't work: when a macro is defined in a.m4 and invoked from b.m4, the quotes in effect for b.m4 will be used for parsing the inner definition of the macro - while the quotes for a.m4 apply to the outermost quotes. Oh well :-( > Although I agree that using non-ASCII quotes would eliminate ambiguity, > it would make debugging tougher, when the character appears correctly > in the editor but is in the wrong encoding. Indeed. Bruno
BSD m4 (was: more m4 underquotations)
Eric Blake wrote: > One more misfeature of changequote - changequote(,) is not > portable. The BSD folks have been trying to get their m4 > implementation to copy enough of GNU m4's traits that it > can be used in place of GNU M4 to support autoconf. But we don't need to care about that. We aren't putting clever features into our tools, just to later fall back to the least common denominator. We have been telling people that they need GNU m4 for autoconf and automake for years - there is no reason to stop doing so. > I DO know that in the > current source code of BSD m4, "changequote(,)" behaves > like "changequote" (ie. sets the quotes to ` and ') rather than > the GNU behavior of disabling quotes. This sounds like a POSIX [1] violation of BSD m4. Bruno [1] http://www.opengroup.org/susv3xcu/m4.html
Re: fix inadequate Windows implementation of tmpfile
Bruno Haible <[EMAIL PROTECTED]> writes: > Here, it's debatable whether the tmpfile() function in MSVCRT is broken: > On one hand, ISO C 99 says >"It should be possible to open at least TMP_MAX temporary files during > the lifetime of the program..." > which isn't fulfilled, as you say, if the user doesn't have the permissions. > (Btw, such situations can also occur on Unix! KDE becomes unusable when > /tmp is full.) > > On the other hand, Microsoft's doc of tmpfile > http://msdn2.microsoft.com/en-us/library/x8x7sakw(VS.80).aspx > and of tmpfile_s > http://msdn2.microsoft.com/en-us/library/b3dz6009(VS.80).aspx > explicitly says > "The temporary file is created in the root directory." > > So, your replacement doesn't fulfill the Microsoft doc. Therefore, I think > you should choose a different name for your function, since you want it > to behave differently than the documented behaviour of tmpfile() on Windows. I agree that the Windows implementation is, arguably, C99 and POSIX compliant. But I claim that it has an unreasonably poor quality of implementation, bad enough that we should replace it. If a Unix-like system implemented its tmpfile in a similar way, by always attempting to create a file in the root directory and failing if permissions were lacking, I imagine that we'd all accept that this is a poor implementation that should be replaced. To me the Microsoft documentation seems like a red herring. The way I've always thought about Gnulib is that it allows a GNU program to make assumptions about its environment that GNU programmers find comfortable, by covering up the differences where it is possible. For example, GNU programmers find it comfortable to assume that malloc(0) returns non-null, so Gnulib helps with that. > The _O_TEMPORARY flag is interesting. It appears to be portable to all > versions of Windows since Woe95. What can you say about its reliability? > The native tmpfile() uses this flag as well. Is a file created with tmpfile() > on Windows deleted when you interrupt the program with Ctrl-C / Ctrl-Break? I've only tested it under Wine, not under any version of Windows, so I can't really say for sure. There seems to be some kind of problem with FILE_FLAG_DELETE_ON_CLOSE, with which _O_TEMPORARY is presumably implemented, under Windows 95: http://cygwin.com/ml/cygwin-cvs/2003-q4/msg00025.html In fact, it appears that FILE_FLAG_DELETE_ON_CLOSE works on NT derivatives, not on the older Windows systems, based on the has_delete_on_close definitions here (sorry about citing a patch when a source file would probably be better, this is based on a very quick search): http://www.cygwin.com/ml/cygwin-patches/2003-q1/msg00170.html Where it works, it appears to be implemented such that it always works, even if a program is terminated abruptly, based on what I've read on the web. >> I using the clean-temp module instead, as a >> more portable solution, but then I needed to interpose upon >> fclose (and perhaps close as well) to delete the temporary file >> when it was closed, which quickly turned into a mess. > > Can you elaborate, please? clean-temp guarantees that the file will be > removed in most cases. You want it removed earlier, as soon as the fclose > happens? To save disk space? Yes: PSPP uses temporary files for transposing big matrices, out-of-core merge sorting, and so on, so it's a good idea to free up disk space when it can. -- Ben Pfaff [EMAIL PROTECTED] http://benpfaff.org
Re: fix inadequate Windows implementation of tmpfile
Ben Pfaff wrote: > There seems to be some kind of problem with > FILE_FLAG_DELETE_ON_CLOSE, with which _O_TEMPORARY is presumably > implemented, under Windows 95: > http://cygwin.com/ml/cygwin-cvs/2003-q4/msg00025.html > > In fact, it appears that FILE_FLAG_DELETE_ON_CLOSE works on NT > derivatives, not on the older Windows systems, based on the > has_delete_on_close definitions here (sorry about citing a patch > when a source file would probably be better, this is based on a > very quick search): > http://www.cygwin.com/ml/cygwin-patches/2003-q1/msg00170.html > > Where it works, it appears to be implemented such that it always > works, even if a program is terminated abruptly, based on what > I've read on the web. Thanks for these explanations. It appears worth to use this in the clean-temp module. 2007-02-15 Bruno Haible <[EMAIL PROTECTED]> * lib/clean-temp.c [WIN32 && !CYGWIN]: Include . (supports_delete_on_close): New function. (open_temp, fopen_temp): Use _O_TEMPORARY when supported. *** lib/clean-temp.c14 Jan 2007 11:32:10 - 1.11 --- lib/clean-temp.c16 Feb 2007 02:38:34 - *** *** 1,5 /* Temporary directories and temporary files with automatic cleanup. !Copyright (C) 2001, 2003, 2006 Free Software Foundation, Inc. Written by Bruno Haible <[EMAIL PROTECTED]>, 2006. This program is free software; you can redistribute it and/or modify --- 1,5 /* Temporary directories and temporary files with automatic cleanup. !Copyright (C) 2001, 2003, 2006-2007 Free Software Foundation, Inc. Written by Bruno Haible <[EMAIL PROTECTED]>, 2006. This program is free software; you can redistribute it and/or modify *** *** 30,35 --- 30,40 #include #include + #if (defined _WIN32 || defined __WIN32__) && ! defined __CYGWIN__ + # define WIN32_LEAN_AND_MEAN /* avoid including junk */ + # include + #endif + #include "error.h" #include "fatal-signal.h" #include "pathmax.h" *** *** 563,568 --- 568,600 } + #if (defined _WIN32 || defined __WIN32__) && ! defined __CYGWIN__ + + /* On Windows, opening a file with _O_TEMPORARY has the effect of passing +the FILE_FLAG_DELETE_ON_CLOSE flag to CreateFile(), which has the effect +of deleting the file when it is closed - even when the program crashes. +But (according to the Cygwin sources) it works only on Windows NT or newer. +So we cache the info whether we are running on Windows NT or newer. */ + + static bool + supports_delete_on_close () + { + static int known; /* 1 = yes, -1 = no, 0 = unknown */ + if (!known) + { + OSVERSIONINFO v; + + if (GetVersionEx (&v)) + known = (v.dwPlatformId == VER_PLATFORM_WIN32_NT ? 1 : -1); + else + known = -1; + } + return (known > 0); + } + + #endif + + /* Register a file descriptor to be closed. */ static void register_fd (int fd) *** *** 598,604 int saved_errno; block_fatal_signals (); ! fd = open (file_name, flags, mode); /* actually open or open_safer */ saved_errno = errno; if (fd >= 0) register_fd (fd); --- 630,644 int saved_errno; block_fatal_signals (); ! /* Note: 'open' here is actually open() or open_safer(). */ ! #if (defined _WIN32 || defined __WIN32__) && ! defined __CYGWIN__ ! /* Use _O_TEMPORARY when possible, to increase the chances that the ! temporary file is removed when the process crashes. */ ! if (supports_delete_on_close ()) ! fd = open (file_name, flags | _O_TEMPORARY, mode); ! else ! #endif ! fd = open (file_name, flags, mode); saved_errno = errno; if (fd >= 0) register_fd (fd); *** *** 616,623 int saved_errno; block_fatal_signals (); ! fp = fopen (file_name, mode); /* actually fopen or fopen_safer */ ! saved_errno = errno; if (fp != NULL) { /* It is sufficient to register fileno (fp) instead of the entire fp, --- 656,683 int saved_errno; block_fatal_signals (); ! /* Note: 'fopen' here is actually fopen() or fopen_safer(). */ ! #if (defined _WIN32 || defined __WIN32__) && ! defined __CYGWIN__ ! /* Use _O_TEMPORARY when possible, to increase the chances that the ! temporary file is removed when the process crashes. */ ! if (supports_delete_on_close ()) ! { ! size_t mode_len = strlen (mode); ! char *augmented_mode = (char *) xallocsa (mode_len + 2); ! memcpy (augmented_mode, mode, mode_len); ! memcpy (augmented_mode + mode_len, "D", 2); ! ! fp = fopen (file_name, augmented_mode); ! saved_errno = errno; ! ! freesa (augmented_mode); ! } ! else ! #endif ! { ! fp = fopen (file_name, mode); ! saved_errno = errno; ! } if (fp != NULL) { /* It is sufficient to register fileno (fp) instead of the entire fp,
Re: fix inadequate Windows implementation of tmpfile
Ben Pfaff wrote: > I agree that the Windows implementation is, arguably, C99 and > POSIX compliant. But I claim that it has an unreasonably poor > quality of implementation, bad enough that we should replace it. > If a Unix-like system implemented its tmpfile in a similar way, > by always attempting to create a file in the root directory and > failing if permissions were lacking, I imagine that we'd all > accept that this is a poor implementation that should be > replaced. > > To me the Microsoft documentation seems like a red herring. The > way I've always thought about Gnulib is that it allows a GNU > program to make assumptions about its environment that GNU > programmers find comfortable, by covering up the differences > where it is possible. For example, GNU programmers find it > comfortable to assume that malloc(0) returns non-null, so Gnulib > helps with that. Paul, what's your opinion on this? Should gnulib includes overrides under the same function name, even if they contradict the respective platform's documentation? > > Can you elaborate, please? clean-temp guarantees that the file will be > > removed in most cases. You want it removed earlier, as soon as the fclose > > happens? To save disk space? > > Yes: PSPP uses temporary files for transposing big matrices, > out-of-core merge sorting, and so on, so it's a good idea to free > up disk space when it can. But with the clean-temp module, you can arrange yourself to free the disk space as soon as possible: just call cleanup_temp_file right after close_temp. The only feature that _O_TEMPORARY provides is that the file is removed if the process crashes. Bruno
check for C99-compliant snprintf
Pre-C99 versions of snprintf often had an interface different from the C99 interface: they would return -1 when the buffer was too small, and the "size" argument was not necessarily interpreted the same way. However, the current Gnulib snprintf module doesn't check whether snprintf is C99-compliant, it just checks for its presence. This is a problem with, e.g. Windows, which has such a noncompliant snprintf. The following patch attempts to remedy the situation. It works fine for me with mingw32/Wine. I suspect that the patch to vasnprintf.c is not the right way to go about things, but I don't know what the preferred method is. vsnprintf probably wants something similar. Comments? Index: lib/vasnprintf.c === RCS file: /sources/gnulib/gnulib/lib/vasnprintf.c,v retrieving revision 1.22 diff -u -p -r1.22 vasnprintf.c --- lib/vasnprintf.c30 Jan 2007 01:07:22 - 1.22 +++ lib/vasnprintf.c16 Feb 2007 05:01:05 - @@ -27,6 +27,9 @@ # include #endif +/* If we're replacing snprintf with rpl_snprintf, avoid a loop. */ +#undef snprintf + /* Specification. */ #if WIDE_CHAR_VERSION # include "vasnwprintf.h" Index: m4/snprintf.m4 === RCS file: /sources/gnulib/gnulib/m4/snprintf.m4,v retrieving revision 1.3 diff -u -p -r1.3 snprintf.m4 --- m4/snprintf.m4 23 Jan 2005 08:06:57 - 1.3 +++ m4/snprintf.m4 16 Feb 2007 05:01:06 - @@ -1,12 +1,27 @@ -# snprintf.m4 serial 2 -dnl Copyright (C) 2002, 2003, 2004 Free Software Foundation, Inc. +# snprintf.m4 serial 3 +dnl Copyright (C) 2002, 2003, 2004, 2007 Free Software Foundation, Inc. dnl This file is free software; the Free Software Foundation dnl gives unlimited permission to copy and/or distribute it, dnl with or without modifications, as long as this notice is preserved. AC_DEFUN([gl_FUNC_SNPRINTF], [ - AC_REPLACE_FUNCS(snprintf) + AC_CACHE_CHECK([for C99-compliant snprintf], +[gl_cv_func_snprintf], +[AC_RUN_IFELSE( + [AC_LANG_PROGRAM( + [[#include ]], + [[char s[2]; +return !(snprintf (s, 2, "foo") == 3 && s[0] == 'f' && s[1] == '\0');]])], + [gl_cv_func_snprintf=yes], + [gl_cv_func_snprintf=no], + [gl_cv_func_snprintf=no])]) + + if test $gl_cv_func_snprintf = no; then +AC_LIBOBJ(snprintf) +AC_DEFINE(snprintf, rpl_snprintf, + [Define to rpl_snprintf if the replacement function should be used.]) + fi AC_CHECK_DECLS_ONCE(snprintf) gl_PREREQ_SNPRINTF ]) -- "In the PARTIES partition there is a small section called the BEER. Prior to turning control over to the PARTIES partition, the BIOS must measure the BEER area into PCR[5]." --TCPA PC Specific Implementation Specification
Re: check for C99-compliant snprintf
It's a bit of a pain that this will reject all cross-compiled snprintfs. Is there some way you can test for this at compile-time? Does the nonstandard snprintf have exactly the same signature as the C99 snprintf? If not, we should be able to catch this at compile-time. Does the nonstandard implementation have a vsnprintf? If not, we could test for that at link-time and cross-compiles wouldn't have to substitute snprintf except on clearly pre-C99 systems. Basically, I'm looking for any way we can distinguish the nonstandard implementation without running it.
Re: predefined preprocessor macros on Windows platforms
Bruno Haible <[EMAIL PROTECTED]> writes: > Brian Dessent wrote: >> This should be __CYGWIN__ and not __CYGWIN32__. > > Yes, you are right. My bad. Here is the corrected summary: This is very useful, and I have been referencing to your posts describing this a few times. Would it be possible to put this in the manual? /Simon