> 2021年9月1日 13:56，Andres Freund 写道：
>
> Hi,
>
> Attached is an updated patch AIO series. The major changes are:
> - rebased onto master (Andres)
> - lots of progress on posix AIO backend (Thomas)
> - lots of progress towards a windows native AIO implementation - not yet quite
> merged (Thomas
Hi,
On 2022-10-12 14:45:26 +0900, Michael Paquier wrote:
> On Tue, Aug 31, 2021 at 10:56:59PM -0700, Andres Freund wrote:
> > I've attached the code for posterity, but the series is large enough that I
> > don't think it makes sense to do that all that often... The code is at
> > https://github.co
On Tue, Aug 31, 2021 at 10:56:59PM -0700, Andres Freund wrote:
> I've attached the code for posterity, but the series is large enough that I
> don't think it makes sense to do that all that often... The code is at
> https://github.com/anarazel/postgres/tree/aio
I don't know what's the exact status
On Wed, Sep 1, 2021 at 11:27 AM Andres Freund wrote:
>
> Hi,
>
> Attached is an updated patch AIO series. The major changes are:
Hi Andres, is there a plan to get fallocate changes alone first? I think
fallocate API can help parallel inserts work (bulk relation extension
currently writes zero fil
Hi Andres,
> > The code is at
> > https://github.com/anarazel/postgres/tree/aio
>
> Just FYI the cfbot says that this version of the patchset doesn't
> apply anymore, and it seems that your branch was only rebased to
> 43c1c4f (Sept. 21th) which doesn't rebase cleanly:
After watching your recent
Hi,
On Wed, Sep 1, 2021 at 1:57 PM Andres Freund wrote:
>
> I've attached the code for posterity, but the series is large enough that I
> don't think it makes sense to do that all that often...
Agreed.
> The code is at
> https://github.com/anarazel/postgres/tree/aio
Just FYI the cfbot says tha
On Wed, Sep 1, 2021 at 5:57 PM Andres Freund wrote:
> - lots of progress on posix AIO backend (Thomas)
A quick note on this piece: Though it's still a work in progress with
a few things that need to be improved, I've tested this on a whole lot
of different OSes now. I originally tried to use re
On Wed, Jul 28, 2021 at 1:37 PM Melanie Plageman
wrote:
>
> On Tue, Feb 23, 2021 at 5:04 AM Andres Freund wrote:
> >
> > ## AIO API overview
> >
> > The main steps to use AIO (without higher level helpers) are:
> >
> > 1) acquire an "unused" AIO: pgaio_io_get()
> >
> > 2) start some IO, this is d
Thomas Munro writes:
> In any case, we already do it which is why wrasse (Sun Studio
> compiler) warns about indkey in pg_index.h. Curiously, indkey is not
> always the final member of the containing struct, depending on
> CATALOG_VARLEN...
Hm? CATALOG_VARLEN is never to be defined, see genbki.
On Sat, Jul 31, 2021 at 7:52 AM Andres Freund wrote:
> On 2021-07-30 15:35:30 -0400, Melanie Plageman wrote:
> > * I think that having TBMIterateResult inside of TBMIterator is not
> > well-defined C language behavior. In [1], it says
> >
> > "Structures with flexible array members (or unions
Hi,
On 2021-07-30 15:35:30 -0400, Melanie Plageman wrote:
> * I think that having TBMIterateResult inside of TBMIterator is not
> well-defined C language behavior. In [1], it says
>
> "Structures with flexible array members (or unions who have a
> recursive-possibly structure member with fl
On Wed, Jul 28, 2021 at 2:10 PM Andres Freund wrote:
> On 2021-07-28 13:37:48 -0400, Melanie Plageman wrote:
>
> > Each IO will have its own TBMIterateResult allocated and returned by the
> > PgStreamingRead helper and freed later by
> > heapam_scan_bitmap_next_block() before requesting the next b
Hi,
On 2021-07-28 13:37:48 -0400, Melanie Plageman wrote:
> Attached is a patch on top of the AIO branch which does bitmapheapscan
> prefetching using the PgStreamingRead helper already used by sequential
> scan and vacuum on the AIO branch.
>
> The prefetch iterator is removed and the main iterat
On Tue, Feb 23, 2021 at 5:04 AM Andres Freund wrote:
>
> ## AIO API overview
>
> The main steps to use AIO (without higher level helpers) are:
>
> 1) acquire an "unused" AIO: pgaio_io_get()
>
> 2) start some IO, this is done by functions like
>pgaio_io_start_(read|write|fsync|flush_range)_(smg
Sorry for another late reply, finally found some time to formulate couple of
thoughts.
> On Thu, Feb 25, 2021 at 09:22:43AM +0100, Dmitry Dolgov wrote:
> > On Wed, Feb 24, 2021 at 01:45:10PM -0800, Andres Freund wrote:
> >
> > > I'm curious if it makes sense
> > > to explore possibility to have th
Hi,
On 25.02.2021 02:03, Andres Freund wrote:
pg_stat_aio_backends.
This stat is based on COUNTERs, which is great, but the issue here is that
its lifespan is limited by the lifespan of the backend processes - once the
backend exits the stat will no longer be available - which could be
inappropr
> On Wed, Feb 24, 2021 at 01:45:10PM -0800, Andres Freund wrote:
>
> > I'm curious if it makes sense
> > to explore possibility to have these sort of "backpressure", e.g. if
> > number of inflight requests is too large calculate inflight_limit a bit
> > lower than possible (to avoid hard performanc
Hi,
On 2021-02-24 21:41:16 +0100, Dmitry Dolgov wrote:
> I'm curious about control knobs for this feature, it's somewhat related
> to the stats questions also discussed in this thread. I guess most
> important of those are max_aio_in_flight, io_max_concurrency etc, and
> they're going to be a hard
Hi,
On 2021-02-24 14:59:19 -0500, Greg Stark wrote:
> I guess what I would be looking for in stats would be a way to tell
> what the bandwidth, latency, and queue depth is. Ideally one day
> broken down by relation/index and pg_stat_statement record.
I think doing it at that granularity will like
Hi,
On 2021-02-24 21:15:14 +0500, Alexey Lesovsky wrote:
> pg_stat_aios.
> In general, this stat is a set of text values, and at the same time it looks
> GAUGE-like (similar to pg_stat_activity or pg_locks), and is only relevant
> for the moment when the user is looking at it. I think it would be
> On Tue, Feb 23, 2021 at 02:03:44AM -0800, Andres Freund wrote:
>
> over the last ~year I spent a lot of time trying to figure out how we could
> add AIO (asynchronous IO) and DIO (direct IO) support to postgres. While
> there's still a *lot* of open questions, I think I now have a decent handle o
I guess what I would be looking for in stats would be a way to tell
what the bandwidth, latency, and queue depth is. Ideally one day
broken down by relation/index and pg_stat_statement record.
I think seeing the actual in flight async requests in a connection is
probably not going to be very usefu
Hi,
Thank you for the amazing and great work.
On 23.02.2021 15:03, Andres Freund wrote:
## Stats
There are two new views: pg_stat_aios showing AIOs that are currently
in-progress, pg_stat_aio_backends showing per-backend statistics about AIO.
As a DBA I would like to propose a few amendments
On Tue, Feb 23, 2021 at 11:03 PM Andres Freund wrote:
> over the last ~year I spent a lot of time trying to figure out how we could
> add AIO (asynchronous IO) and DIO (direct IO) support to postgres. While
> there's still a *lot* of open questions, I think I now have a decent handle on
> most of
Hi,
On 2021-02-23 14:58:32 -0500, Greg Stark wrote:
> So firstly this is all just awesome work and I have questions but I
> don't want them to come across in any way as criticism or as a demand
> for more work.
I posted it to get argued with ;).
> The callbacks make me curious about two questio
On Tue, 23 Feb 2021 at 05:04, Andres Freund wrote:
>
> ## Callbacks
>
> In the core AIO pieces there are two different types of callbacks at the
> moment:
>
> Shared callbacks, which can be invoked by any backend (normally the issuing
> backend / the AIO workers, but can be other backends if they
Hi,
over the last ~year I spent a lot of time trying to figure out how we could
add AIO (asynchronous IO) and DIO (direct IO) support to postgres. While
there's still a *lot* of open questions, I think I now have a decent handle on
most of the bigger architectural questions. Thus this long email.
27 matches
Mail list logo
