Darryl Miles wrote:
There maybe other ways to deal with that write() but as far as I
understand the NT kernel does not provide a true non-blocking mechanism
to work from with pipes. This is where you can offer to the kernel the
data and if the buffers are full the kernel will reject the data without
blocking leaving the application holding it. Overlapped I/O as I
understand it does not work like this.
Okay the biggest factor I overlooked here was that the memory buffer
inside application space is pinned until IO completion is signalled, so
the theory here must be that for any real world usage I'd run out of
application address space before the kernel hit any internal problems.
As there is little overhead inside the NT kernel per overlapped IO that
is queued up compared to the 1kb/4kb buffer within application address
space.
If this is the case then for any application using Overlapped I/O on
Win32 it must manage its own outstanding queue and perform its own
throttling to stop it chewing itself to pieces.
But with POSIX in both blocking and non-blocking mode the kernel
implements that rate limiting by virue of design.
So I think even more that a solution is possible with overlapping I/O
(after creating a temporary private copy of the application data) with a
byte count outstanding threshold driving select/poll writability
indication is the way to go.
If the byte count if over the threshold:
* When a POSIX write(2) is made on a blocking fd (O_NONBLOCK is reset)
then we implement the blocking all the way until the IO completion arrives.
* When a POSIX write(2) is made on a non-blocking fd (O_NONBLOCK is
set) then we return -1 EAGAIN.
* Select justs need to lookup the outstanding by count and update the
write_ready state based on the threshold.
* The outstanding byte count needs to be protected by a mutex.
If WriteFileEx() can never block for overlapped I/O there is no need to
have a writer thread. The purpose of the writer thread then thinking
out loud before was to offload the possibility of the
WriteFile()/WriteFileEx() from blocking. Offloading it away from the
application thread of execution making the POSIX write(2) call.
Another point I have thought about when we call close(2) on a pipe with
outstanding IO, we need to close the POSIX side but keep the W32API side
active until the IO completes. Call it an unlinked IO if you will.
This means that the IO completion routine needs to check and handle
deallocation of the "fhandle_pipe" class since it maybe the last to be
using it. Maybe this is already accounted for in some other way.
Darryl
--
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Problem reports: http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ: http://cygwin.com/faq/
