Re: isatty fails on Windows 8

[Michael Goffioul]

On Sun, Dec 23, 2012 at 3:51 PM, Michael Goffioul <
michael.goffi...@gmail.com> wrote:

> On Sun, Dec 23, 2012 at 3:25 PM, Eli Zaretskii <e...@gnu.org> wrote:
>
>> > Date: Thu, 20 Dec 2012 19:02:03 -0500
>> > From: Michael Goffioul <michael.goffi...@gmail.com>
>> >
>> > We've got a bug report in octave [1] that seems to indicate that
>> gnulib's
>> > replacement of isatty is incorrect on Windows 8 (it's fine up to Windows
>> > 7). Looking at the implementation, it first calls the Windows _isatty
>> > version, then checks the last 2 bits of the handle associated with the
>> file
>> > descriptor, based on the assumption described in [2] (a console HANDLE
>> is
>> > not a multiple of 4). It seems that this last assumption is not valid
>> > anymore on Windows 8.
>>
>> Does the following work on Windows 8?
>>
>>   #define ISATTY(fd) (isatty(fd) && lseek(fd,SEEK_CUR,0) == -1)
>>
>> and then use ISATTY instead of isatty.
>>
>> (The lseek test is to reject other character devices, most prominently
>> the null device, with MS implementation of isatty doesn't reject.)
>>
>
> I'll ask the bug reported to try it out (I don't have Windows 8). Paolo,
> note that there's an error in your suggested code, one should test "!= 0"
> instead of equality: GetConsoleMode returns non zero on success.
>
>
Someone has finally tested the suggestions under Windows 8. Paolo's
suggestion works OK, Eli's one does not. I attached the test utility. The
results are:

test-tty.exe:

isatty (stdin): 64
IsConsoleHandle (stdin): 0
IsConsoleHandle2 (stdin): 1
ISATTY (stdin): 0
isatty (stdout): 64
IsConsoleHandle (stdout): 0
IsConsoleHandle2 (stdout): 1
ISATTY (stdout): 0

test-tty.exe <NUL:

isatty (stdin): 64
IsConsoleHandle (stdin): 0
IsConsoleHandle2 (stdin): 0
ISATTY (stdin): 0
isatty (stdout): 64
IsConsoleHandle (stdout): 0
IsConsoleHandle2 (stdout): 1
ISATTY (stdout): 0


Note, the lseek call, as suggested, is incorrect [1]. The 2nd and 3rd
argument should be swapped. However it didn't make any difference in the
test results.

Michael.

[1] http://msdn.microsoft.com/en-us/library/1yee101t(v=vs.100).aspx

#include <io.h>
#include <stdio.h>
#define WIN32_LEAN_AND_MEAN 1
#include <windows.h>

#define IsConsoleHandle(h) (((long) (h) & 3) == 3)

BOOL IsConsoleHandle2 (HANDLE h)
{
  DWORD mode;
  return GetConsoleMode (h, &mode) != 0;
}

#define ISATTY(fd) (isatty(fd) && lseek(fd,SEEK_CUR,0) == -1)

int main (int argc, char **argv)
{
  printf ("isatty (stdin): %d\n", isatty (fileno (stdin)));
  printf ("IsConsoleHandle (stdin): %d\n",
          IsConsoleHandle ((HANDLE) _get_osfhandle (fileno (stdin))));
  printf ("IsConsoleHandle2 (stdin): %d\n",
          IsConsoleHandle2 ((HANDLE) _get_osfhandle (fileno (stdin))));
  printf ("ISATTY (stdin): %d\n", ISATTY (fileno (stdin)));

  printf ("isatty (stdout): %d\n", isatty (fileno (stdout)));
  printf ("IsConsoleHandle (stdout): %d\n",
          IsConsoleHandle ((HANDLE) _get_osfhandle (fileno (stdout))));
  printf ("IsConsoleHandle2 (stdout): %d\n",
          IsConsoleHandle2 ((HANDLE) _get_osfhandle (fileno (stdout))));
  printf ("ISATTY (stdout): %d\n", ISATTY (fileno (stdout)));

  return 0;
}