Hi Matthew,
Sorry for the delay in replying.
On 6/17/2021 7:08 PM, Matthew Flatt wrote:
At Wed, 16 Jun 2021 17:33:56 -0400, George Neuner wrote:
>
> Dumb question ... why should non-blocking I/O worry about "flush" at
> all. Why not behave like native I/O where writes are guaranteed only to
> get to the I/O buffer?
If I understand what you mean, then that's the intent. But I think
you're using "buffer" in the sense of the OS layer. That's different
than the buffer that you have in, say, the C library. The Racket port
model uses "buffer" to mean the one like the one implemented in a C
library, and "flush" is used also used in the C library sense, and not
in terms of a buffer that might exist in the next layer (which might be
the OS or might not).
I was not necessarily thinking about any particular layer, but I was
thinking about open files, TCP connections, etc. I know that both [or
neither] the OS and the I/O library may implement buffering.
My point was that the docs for write-bytes-avail et al specifically
mention "flush" of data, and in a way that implies (to me) that there is
expected to be something else underlying the buffer to "flush" to, e.g.,
storage media, network connection, etc., ... something. But, if I
understand correctly, Racket pipes are a library construct and really
just a named block of memory.
So it seems like "flush" is being used in the docs to mean insertion of
data into the buffer ... which is confusing (to me) and at odds with how
the word typically is used in discussions of I/O. To reveal prejudices,
I've been programming for 40 years and Racket is an umpteenth language.
Probably this became more clear with experiments that Shu-Hung and you
reported, but that's what happens, except that trying to write to a
pipe whose buffer is full will then grow the buffer. Naturally, the
buffer grows by doubling --- up to the pipe's capacity limit, if any.
An OS pipe tends to have a fixed size (OS-level) buffer that is
allocated up front, and that serves as both the maximum granularity of
writes and the capacity of the pipe. Racket pipes are meant to work in
those kinds of cases, but also adapt to cases where pipes are small and
numerous or where they have no capacity limit, so the write granularity
and capacity are not so tightly coupled.
So the limit value passed to make-pipe is only a maximum size for the
data buffer, which starts [way too] small and then grows as needed up to
the maximum. Although that does make some sense - I think starting at
16 bytes is a wee bit restrictive: many (most?) real world uses would
have to grow it very quickly.
I think many (most?) of us expected that Racket pipes would behave like
OS pipes - a fixed length allocation up front - and that a single call
to write-bytes-avail would fill the buffer. [It turns out that it
/does/ fill the buffer, but the buffer isn't what we thought it would be.]
There's likely room for improvement. Racket BC pipes absorb more in a
non-blocking write, because MzScheme was very slow in the old days, and
it was worth looping and trying harder in the low layers instead of
having to go back out to slow MzScheme code. For Racket CS, that
slow--fast boundary isn't there, and so the lower layer core tries to
stay as fast as possible by branching and looping less internally, but
I don't really know if that's the best choice.
Growing the buffer is expensive in any case and particularly when
existing data has to be copied. With pre-allocated buffer, the I/O call
will spend only what time is necessary to copy new data. That would
seem to be the fastest implementation possible.
Anyway, thank you very much for explaining it.
George
--
You received this message because you are subscribed to the Google Groups "Racket
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/racket-users/71be924c-e88e-fd91-ea94-af6694c90a0e%40comcast.net.
