Re: [PATCH] Initialize IO_STATUS_BLOCK for pread, pwrite

[Corinna Vinschen]

On Dec  1 13:46, Mark Geisert wrote:
> On Fri, 1 Dec 2017, Corinna Vinschen wrote:
> > On Dec  1 10:30, Corinna Vinschen wrote:
> > > In terms of pread/pwrite, the new handle shares the same settings with
> > > the original handle.  However, if you use cygwin_attach_handle_to_fd,
> > > there's a chance information got lost.  Nobody actually uses this call ;)
> > > 
> > > In terms of FILE_SYNCHRONOUS_IO_NONALERT, this is stored in
> > > fhandler_base::options, utilizing the get_options/set_options methods.
> > > I have a hunch that cygwin_attach_handle_to_fd fails to call set_options,
> > > thus options is 0 when you call pwrite, thus the new handle is opened
> > > without FILE_SYNCHRONOUS_IO_NONALERT and all the other option flags
> > > we use by default.
> > 
> > It's more than a hunch.  Of course the info gets lost since
> > none of the functions called by cygwin_attach_handle_to_fd calls
> > NtQueryInformationFile(FileModeInformation) to fetch the options.
> 
> Bang.  There it is.  Let me fix the offending program to just use
> Cygwin-supplied handles and make sure this bug of mine is squashed.  I'll
> report back but it might be a few days.
> 
> I'm open to using overlapped I/O for the usual read & write cases of aio but
> there are some extensions I have in mind that don't allow for overlapped so
> I think I need to have threads handle them.  I might combine the two.

I'm just a bit concerned in terms of calling lots of aio_read/write at
the same time or lio_listio with lots of entries.  One thread for each
entry?

> Using
> overlapped for the common case would, I think, allow me to reduce the number
> of worker threads hanging around.  Thanks for the input!

I wonder if APCs are the way to go for this use case.  As you might
know, Nt{Read,Write}File provide a way to specify an APC routine and a
APC context pointer (struct aiocb *?), so instead of waiting actively
for the event, you could just lean back and wait for the APC routine to
be called.

As a sidenote, I think we could use APCs in other scenarios, too, but
somehow we never got around to it.


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


signature.asc
Description: PGP signature

Re: [PATCH] Initialize IO_STATUS_BLOCK for pread, pwrite

[Mark Geisert]

On Sat, 2 Dec 2017, Corinna Vinschen wrote:

On Dec  1 13:46, Mark Geisert wrote:

I'm open to using overlapped I/O for the usual read & write cases of aio but
there are some extensions I have in mind that don't allow for overlapped so
I think I need to have threads handle them.  I might combine the two.


I'm just a bit concerned in terms of calling lots of aio_read/write at
the same time or lio_listio with lots of entries.  One thread for each
entry?


No, first use of aio_* or lio_* causes a pool of AIO_MAX worker threads to 
be created.  The actual request(s) is/are placed on a FIFO queue.  Each 
enqueue bumps a semaphore to wake up a worker thread to dequeue and 
process a request.  More requests than threads?  It's OK, requests are

queued until there's a worker thread available to field it.


Using
overlapped for the common case would, I think, allow me to reduce the number
of worker threads hanging around.  Thanks for the input!


I wonder if APCs are the way to go for this use case.  As you might
know, Nt{Read,Write}File provide a way to specify an APC routine and a
APC context pointer (struct aiocb *?), so instead of waiting actively
for the event, you could just lean back and wait for the APC routine to
be called.

As a sidenote, I think we could use APCs in other scenarios, too, but
somehow we never got around to it.


Good points.  Requires a little architecting.  Have to wait in an 
interruptible wait.  Hmm.  I could consider this direction too.


..mark