Re: AIO v2.3

2025-02-18 Thread Robert Haas
On Mon, Feb 17, 2025 at 4:01 PM Andres Freund wrote: > If one instead integrates with resowners, that kind of thing works, because > exec_simple_query() calls PortalDrop(), which in turn calls > ResourceOwnerRelease(). Hmm, so maybe that's a reason to do it via resowner.c, then. The fact that it'

Re: AIO v2.3

2025-02-17 Thread Andres Freund
Hi, On 2025-02-12 13:00:22 -0500, Robert Haas wrote: > On Tue, Feb 11, 2025 at 4:43 PM Andres Freund wrote: > > One way we could avoid the need for a mechanism to reset-batch-in-progress > > would be to make batch submission controlled by a flag on the IO. Something > > like > > pgaio_io_set_

Re: AIO v2.3

2025-02-13 Thread Jakub Wartak
On Tue, Feb 11, 2025 at 12:10 AM Andres Freund wrote: >> TLDR; in terms of SELECTs the master vs aioworkers looks very solid! > Phew! Weee! Yay. Another good news: I've completed a full 24h pgbench run on the same machine and it did not fail or report anything suspicious. FYI, patchset didn't n

Re: AIO v2.3

2025-02-12 Thread James Hunter
On Tue, Feb 11, 2025 at 1:44 PM Andres Freund wrote: > ... > Alternatively we could make pgaio_batch_begin() basically start a critical > section, but that doesn't seem like a good idea, because too much that needs > to happen around buffered IO isn't compatible with critical sections. > > > Does

Re: AIO v2.3

2025-02-12 Thread Robert Haas
On Tue, Feb 11, 2025 at 4:43 PM Andres Freund wrote: > Alternatively we could make pgaio_batch_begin() basically start a critical > section, but that doesn't seem like a good idea, because too much that needs > to happen around buffered IO isn't compatible with critical sections. A critical secti

Re: AIO v2.3

2025-02-11 Thread Andres Freund
Hi, On 2025-02-11 11:48:38 +1300, Thomas Munro wrote: > Would the API be better like this?: When you want to create a batch > of I/Os submitted together, you wrap the work in pgaio_begin_batch() > and pgaio_submit_batch(), eg the loop in read_stream_lookahead(). One annoying detail is that an AP

Re: AIO v2.3

2025-02-11 Thread Robert Haas
On Tue, Feb 11, 2025 at 12:11 PM James Hunter wrote: > I like this idea. If we want to submit a batch, then just submit a batch. Sounds good to me, too. -- Robert Haas EDB: http://www.enterprisedb.com

Re: AIO v2.3

2025-02-11 Thread James Hunter
On Mon, Feb 10, 2025 at 2:40 PM Thomas Munro wrote: > ... > Problem statement: You want to be able to batch I/O submission, ie > make a single call to ioring_enter() (and other mechanisms) to start > several I/Os, but the code that submits is inside StartReadBuffers() > and the code that knows how

Re: AIO v2.3

2025-02-10 Thread Andres Freund
Hi, On 2025-02-06 11:50:04 +0100, Jakub Wartak wrote: > Hi Andres, OK, so I've hastily launched AIO v2.3 (full, 29 patches) > patchset probe run before going for short vacations and here results > are attached*. Thanks for doing that work! > TLDR; in terms of SELECTs the mas

Re: AIO v2.3

2025-02-10 Thread Andres Freund
Hi, On 2025-02-11 11:48:38 +1300, Thomas Munro wrote: > On Thu, Jan 23, 2025 at 5:29 PM Andres Freund wrote: > > - Heikki doesn't love pgaio_submit_staged(), suggested pgaio_kick_staged() > > or > > such. I don't love that name though. > > Problem statement: You want to be able to batch I/O su

Re: AIO v2.3

2025-02-10 Thread Thomas Munro
On Thu, Jan 23, 2025 at 5:29 PM Andres Freund wrote: +/* caller will issue more io, don't submit */ +#define READ_BUFFERS_MORE_MORE_MORE (1 << 3) > - Heikki doesn't love pgaio_submit_staged(), suggested pgaio_kick_staged() or > such. I don't love that name though. Problem statement: You want to