(Please Cc me when replying, I'm not subscribed to the list.)
Hi!
I suspect that there is a bug in Cygwin:
1. Create a pipe with both ends in blocking mode (O_NONBLOCK
is not set).
2. The writer sets its end to non-blocking mode.
3. The writer writes to the pipe.
4. The writer restores its end of the pipe to blocking mode
before the reader has read anything from the pipe.
5. The writer closes its end of the pipe.
6. The reader reads from the pipe in blocking mode. The last
bytes written by the writer never appear at the reader,
thus data is silently lost.
Omitting the step 4 above makes the problem go away.
A small test case is attached. I have tested it under Cygwin64 that was
installed a few days ago on Windows 7. Usage:
gcc -std=gnu99 -Wall -Wextra writer.c -o writer
gcc -std=gnu99 -Wall -Wextra reader.c -o reader
./writer | ./reader
The reader only gets the first 65536 bytes. The remaining 5555 bytes
are never seen by the reader even though all write() calls succeed at
the writer.
I found this problem because I had heard a rumor that there is something
wrong with xz 5.2.0 on Cygwin. Since the file I/O code had been changed
in 5.2.0, I investigated that first. There also seems to be something
wrong with xz 5.2.0 in threaded mode under Cygwin which needs more
investigation (random pthread_join() failures cause assertion failures
if xz was built with --enable-debug).
--
Lasse Collin | IRC: Larhzu @ IRCnet & Freenode
// gcc -std=gnu99 -Wall -Wextra reader.c -o reader
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
static void
fail(const char *str)
{
fprintf(stderr, "R: %s\n", str);
exit(EXIT_FAILURE);
}
int
main(void)
{
// It is assumed that stdin is a pipe open for reading.
//
// Read 8 KiB at a time.
static unsigned char buf[8 << 10];
// Total amount read
unsigned total = 0;
for (unsigned i = 0; i < 10; ++i) {
fprintf(stderr, "R: Sleeping\n");
sleep(1);
int amount = read(STDIN_FILENO, buf, sizeof(buf));
fprintf(stderr, "R: read() returned %d\n", amount);
if (amount >= 0) {
total += amount;
fprintf(stderr, "R: Total amount read is %u\n",
total);
}
}
if (close(STDIN_FILENO))
fail("Error closing stdin");
return EXIT_SUCCESS;
}
// gcc -std=gnu99 -Wall -Wextra writer.c -o writer
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
static void
fail(const char *str)
{
fprintf(stderr, "W: %s\n", str);
exit(EXIT_FAILURE);
}
int
main(void)
{
// It is assumed that stdout is a pipe open for writing.
//
// Set stdout to non-blocking mode.
int flags = fcntl(STDOUT_FILENO, F_GETFL);
if (flags == -1)
fail("Cannot get stdout file status flags");
if (fcntl(STDOUT_FILENO, F_SETFL, flags | O_NONBLOCK) == -1)
fail("Setting stdout to non-blocking mode failed");
// Writing 64 KiB works as expected.
static unsigned char buf[64 << 10];
if (write(STDOUT_FILENO, buf, sizeof(buf)) != sizeof(buf))
fail("The first write() failed");
fprintf(stderr, "W: Wrote %zu bytes\n", sizeof(buf));
// Writing more returns successfully but the reader won't see
// these bytes. The reason appears to be the fcntl() below.
const unsigned small_size = 5555;
if (write(STDOUT_FILENO, buf, small_size) != (ssize_t)small_size)
fail("The second write() failed");
fprintf(stderr, "W: Wrote %u bytes\n", small_size);
// Restore the original file status flags to stdout.
// For some reason this makes the bytes from the last
// write() disappear. If this is commented out, all
// data is visible at the reader.
if (fcntl(STDOUT_FILENO, F_SETFL, flags) == -1)
fail("Restoring stdout file status flags failed");
if (close(STDOUT_FILENO))
fail("Error closing stdout");
return EXIT_SUCCESS;
}
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
