Re: Allow io_combine_limit up to 1MB

2025-04-25 Thread Thomas Munro
On Sat, Apr 26, 2025 at 5:43 AM Tom Lane wrote: > Andres Freund writes: > > It's kinda sad to not have any test that tests a larger > > io_combine_limit/io_max_combine_limit - as evidenced by this bug that'd be > > good. However, not all platforms have PG_IOV_MAX > 16, so it seems like it'd > > b

Re: Allow io_combine_limit up to 1MB

2025-04-25 Thread Tom Lane
Andres Freund writes: > It's kinda sad to not have any test that tests a larger > io_combine_limit/io_max_combine_limit - as evidenced by this bug that'd be > good. However, not all platforms have PG_IOV_MAX > 16, so it seems like it'd > be somewhat painful to test? Maybe just skip the test if th

Re: Allow io_combine_limit up to 1MB

2025-04-25 Thread Andres Freund
Hi, On 2025-04-25 10:26:09 -0400, Tom Lane wrote: > Andres Freund writes: > > void > > assign_io_max_combine_limit(int newval, void *extra) > > { > > io_max_combine_limit = newval; > > io_combine_limit = Min(io_max_combine_limit, io_combine_limit_guc); > > } > > void > > assign_io_combine

Re: Allow io_combine_limit up to 1MB

2025-04-25 Thread Tom Lane
Andres Freund writes: > void > assign_io_max_combine_limit(int newval, void *extra) > { > io_max_combine_limit = newval; > io_combine_limit = Min(io_max_combine_limit, io_combine_limit_guc); > } > void > assign_io_combine_limit(int newval, void *extra) > { > io_combine_limit_guc

Re: Allow io_combine_limit up to 1MB

2025-04-25 Thread Andres Freund
Hi, On 2025-03-18 16:18:17 +1300, Thomas Munro wrote: > Here's a new version that also adjusts the code that just landed in > da722699: Something isn't quite right with this code. If I just add -c io_combine_limit=32 to the options and do a seqscan, I get odd failures. Mostly assertion failures

Re: Allow io_combine_limit up to 1MB

2025-03-17 Thread Thomas Munro
Here's a new version that also adjusts the code that just landed in da722699: - /* -* Each IO handle can have an PG_IOV_MAX long iovec. -* -* XXX: Right now the amount of space available for each IO is PG_IOV_MAX. -* While it's tempting to use the io_combine_l

Re: Allow io_combine_limit up to 1MB

2025-03-03 Thread Thomas Munro
On Wed, Feb 12, 2025 at 3:24 PM Thomas Munro wrote: > On Wed, Feb 12, 2025 at 3:22 PM Andres Freund wrote: > > On 2025-02-12 13:59:21 +1300, Thomas Munro wrote: > > > How about just maintaining it in a new variable > > > effective_io_combine_limit, whenever either of them is assigned? > > > > Yea

Re: Allow io_combine_limit up to 1MB

2025-02-14 Thread Andres Freund
Hi, On 2025-02-14 09:32:32 +0100, Jakub Wartak wrote: > On Wed, Feb 12, 2025 at 1:03 AM Andres Freund wrote: > > FWIW, I see substantial performance *regressions* with *big* IO sizes using > > fio. Just looking at cached buffered IO. > > > > for s in 4 8 16 32 64 128 256 512 1024 2048 4096 8192;d

Re: Allow io_combine_limit up to 1MB

2025-02-14 Thread Jakub Wartak
On Wed, Feb 12, 2025 at 1:03 AM Andres Freund wrote: > > Hi, > > On 2025-02-11 13:12:17 +1300, Thomas Munro wrote: > > Tomas queried[1] the limit of 256kB (or really 32 blocks) for > > io_combine_limit. Yeah, I think we should increase it and allow > > experimentation with larger numbers. Note t

Re: Allow io_combine_limit up to 1MB

2025-02-12 Thread Andres Freund
Hi, On 2025-02-12 15:24:21 +1300, Thomas Munro wrote: > On Wed, Feb 12, 2025 at 3:22 PM Andres Freund wrote: > > On 2025-02-12 13:59:21 +1300, Thomas Munro wrote: > > > How about just maintaining it in a new variable > > > effective_io_combine_limit, whenever either of them is assigned? > > > > Y

Re: Allow io_combine_limit up to 1MB

2025-02-11 Thread Thomas Munro
On Wed, Feb 12, 2025 at 3:22 PM Andres Freund wrote: > On 2025-02-12 13:59:21 +1300, Thomas Munro wrote: > > How about just maintaining it in a new variable > > effective_io_combine_limit, whenever either of them is assigned? > > Yea, that's probably the least bad way. > > I wonder if we should ju

Re: Allow io_combine_limit up to 1MB

2025-02-11 Thread Andres Freund
Hi, On 2025-02-12 13:59:21 +1300, Thomas Munro wrote: > How about just maintaining it in a new variable > effective_io_combine_limit, whenever either of them is assigned? Yea, that's probably the least bad way. I wonder if we should just name that variable io_combine_limit and have the GUC be _r

Re: Allow io_combine_limit up to 1MB

2025-02-11 Thread Thomas Munro
On Wed, Feb 12, 2025 at 1:03 PM Andres Freund wrote: > On 2025-02-11 13:12:17 +1300, Thomas Munro wrote: > > I was also > > anticipating future code that would need to multiply that number by other > > terms to allocate shared memory, but after some off-list discussion, that > > seems OK: such cod

Re: Allow io_combine_limit up to 1MB

2025-02-11 Thread Andres Freund
Hi, On 2025-02-11 13:12:17 +1300, Thomas Munro wrote: > Tomas queried[1] the limit of 256kB (or really 32 blocks) for > io_combine_limit. Yeah, I think we should increase it and allow > experimentation with larger numbers. Note that real hardware and > protocols have segment and size limits that

Allow io_combine_limit up to 1MB

2025-02-10 Thread Thomas Munro
Hi, Tomas queried[1] the limit of 256kB (or really 32 blocks) for io_combine_limit. Yeah, I think we should increase it and allow experimentation with larger numbers. Note that real hardware and protocols have segment and size limits that can force the kernel to split your I/Os, so it's not at a