On Thu, 9 Jan 2025 at 22:53, Andres Freund wrote:
> Workstation w/ 2x Xeon Gold 6442Y:
>
>march memresult
> native 100246.13766ms @ 33.282 GB/s
> native 10456.08080ms @ 17.962 GB
On Thu, 9 Jan 2025 at 18:25, Andres Freund wrote:
> > I'm curious about this because the checksum code should be fast enough
> > to easily handle that throughput.
>
> It seems to top out at about ~5-6 GB/s on my 2x Xeon Gold 6442Y
> workstation. But we don't have a good ready-made way of testing t
Hi,
On 2025-01-09 20:10:24 +0200, Ants Aasma wrote:
> On Thu, 9 Jan 2025 at 18:25, Andres Freund wrote:
> > > I'm curious about this because the checksum code should be fast enough
> > > to easily handle that throughput.
> >
> > It seems to top out at about ~5-6 GB/s on my 2x Xeon Gold 6442Y
> >
Hi,
On 2025-01-09 10:59:22 +0200, Ants Aasma wrote:
> On Wed, 8 Jan 2025 at 22:58, Andres Freund wrote:
> > master: ~18 GB/s
> > patch, buffered:~20 GB/s
> > patch, direct, worker: ~28 GB/s
> > patch, direct, uring: ~35 GB/s
> >
> >
> > This was with io_workers=32, io_m
On Wed, 8 Jan 2025 at 22:58, Andres Freund wrote:
> master: ~18 GB/s
> patch, buffered:~20 GB/s
> patch, direct, worker: ~28 GB/s
> patch, direct, uring: ~35 GB/s
>
>
> This was with io_workers=32, io_max_concurrency=128,
> effective_io_concurrency=1000 (doesn't need to
Hi,
On 2025-01-08 15:04:39 +0100, Jakub Wartak wrote:
> On Mon, Jan 6, 2025 at 5:28 PM Andres Freund wrote:
> > I didn't think that pg_stat_* was quite the right namespace, given that it
> > shows not stats, but the currently ongoing IOs. I am going with pg_aios for
> > now, but I don't particul
On Mon, Jan 6, 2025 at 5:28 PM Andres Freund wrote:
>
> Hi,
>
> On 2024-12-19 17:29:12 -0500, Andres Freund wrote:
> > > Not about patch itself, but questions about related stack functionality:
> > > --
Hi,
On 2024-12-19 17:29:12 -0500, Andres Freund wrote:
> > Not about patch itself, but questions about related stack functionality:
> >
> >
> >
> > 7. Is pg_stat_aios still on the table or not ? (A
Hi,
On 2024-12-20 18:27:13 +0100, Jelte Fennema-Nio wrote:
> On Fri, 20 Dec 2024 at 01:54, Andres Freund wrote:
> > Arguably the configuration *did* tell us, by having a higher hard limit...
> >
> > But opting into a higher rlimit, while obviously adhering to the hard limit
> > and perhaps some
On Fri, 20 Dec 2024 at 01:54, Andres Freund wrote:
> Arguably the configuration *did* tell us, by having a higher hard limit...
>
> But opting into a higher rlimit, while obviously adhering to the hard limit
> and perhaps some other config knob, seems fine?
Yes, totally fine. That's exactly the
Hi,
On 2024-12-19 17:34:29 -0500, Tom Lane wrote:
> Andres Freund writes:
> > My current thoughts around this are that we should generally, independent of
> > io_uring, increase the FD limit ourselves.
>
> I'm seriously down on that, because it amounts to an assumption that
> we own the machine a
Andres Freund writes:
> My current thoughts around this are that we should generally, independent of
> io_uring, increase the FD limit ourselves.
I'm seriously down on that, because it amounts to an assumption that
we own the machine and can appropriate all its resources. If ENFILE
weren't a thi
Hi,
Sorry for loosing track of your message for this long, I saw it just now
because I was working on posting a new version.
On 2024-11-18 13:19:58 +0100, Jakub Wartak wrote:
> On Fri, Sep 6, 2024 at 9:38 PM Andres Freund wrote:
> Thank You for worth admiring persistence on this. Please do not
On Fri, Sep 6, 2024 at 9:38 PM Andres Freund wrote:
> Hi,
>
> Attached is the next version of the patchset. (..)
Hi Andres,
Thank You for worth admiring persistence on this. Please do not take it as
criticism, just more like set of questions regarding the patchset v2.1 that
I finally got litt
On Mon, Sep 30, 2024 at 10:49:17AM -0400, Andres Freund wrote:
> We also discussed the topic at
> https://postgr.es/m/20240925020022.c5.nmisch%40google.com
> > ... neither BM_SETTING_HINTS nor keeping bounce buffers looks like a bad
> > decision. From what I've heard so far of the performance eff
On Mon, 30 Sept 2024 at 16:49, Andres Freund wrote:
> On 2024-09-17 11:08:19 -0700, Noah Misch wrote:
> > > - I am worried about the need for bounce buffers for writes of checksummed
> > > buffers. That quickly ends up being a significant chunk of memory,
> > > particularly when using a small
Hi,
On 2024-09-17 11:08:19 -0700, Noah Misch wrote:
> > - I am worried about the need for bounce buffers for writes of checksummed
> > buffers. That quickly ends up being a significant chunk of memory,
> > particularly when using a small shared_buffers with a higher than default
> > number o
On Mon, Sep 16, 2024 at 01:51:42PM -0400, Andres Freund wrote:
> On 2024-09-16 07:43:49 -0700, Noah Misch wrote:
> > On Fri, Sep 06, 2024 at 03:38:16PM -0400, Andres Freund wrote:
> > Reattaching descriptors and memory in each child may work, or one could just
> > block io_method=io_uring under EX
Hi,
On 2024-09-12 14:55:49 -0700, Robert Pang wrote:
> Hi Andres
>
> Thanks for the AIO patch update. I gave it a try and ran into a FATAL
> in bgwriter when executing a benchmark.
>
> 2024-09-12 01:38:00.851 PDT [2780939] PANIC: no more bbs
> 2024-09-12 01:38:00.854 PDT [2780473] LOG: backgro
Hi,
Thanks for the review!
On 2024-09-16 07:43:49 -0700, Noah Misch wrote:
> On Fri, Sep 06, 2024 at 03:38:16PM -0400, Andres Freund wrote:
> > There's plenty more to do, but I thought this would be a useful checkpoint.
>
> I find patches 1-5 are Ready for Committer.
Cool!
> > +typedef enum Pg
On Fri, Sep 06, 2024 at 03:38:16PM -0400, Andres Freund wrote:
> There's plenty more to do, but I thought this would be a useful checkpoint.
I find patches 1-5 are Ready for Committer.
> +typedef enum PgAioHandleState
This enum clarified a lot for me, so I wish I had read it before anything
else
Hi Andres
Thanks for the AIO patch update. I gave it a try and ran into a FATAL
in bgwriter when executing a benchmark.
2024-09-12 01:38:00.851 PDT [2780939] PANIC: no more bbs
2024-09-12 01:38:00.854 PDT [2780473] LOG: background writer process
(PID 2780939) was terminated by signal 6: Aborted
Hi,
On 2024-09-05 01:37:34 +0800, 陈宗志 wrote:
> I hope there can be a high-level design document that includes a
> description, high-level architecture, and low-level design.
> This way, others can also participate in reviewing the code.
Yep, that was already on my todo list. The version I just po
On Sun, 1 Sept 2024 at 18:28, Andres Freund wrote:
> 0 workers 1 worker2 workers
> 4 workers
> master: 65.753 33.246 21.095
> 12.918
> aio v2.0, worker: 21.519
I hope there can be a high-level design document that includes a
description, high-level architecture, and low-level design.
This way, others can also participate in reviewing the code.
For example, which paths were modified in the AIO module? Is it the
path for writing WAL logs, or the path for fl
Hi,
On 2024-09-02 13:03:07 +0300, Heikki Linnakangas wrote:
> On 01/09/2024 09:27, Andres Freund wrote:
> > In the next few days I'll add a bunch more documentation and comments as
> > well
> > as some better perf numbers (assuming my workstation survived...).
>
> Yeah, a high-level README would
On 01/09/2024 09:27, Andres Freund wrote:
The main reason I had previously implemented WAL AIO etc was to know the
design implications - but now that they're somewhat understood, I'm planning
to keep the patchset much smaller, with the goal of making it upstreamable.
+1 on that approach.
To s
27 matches
Mail list logo
