Re: minor fix related to Auxiliary processes and IO workers

2025-05-06 Thread Cédric Villemain
On 07/05/2025 07:56, Michael Paquier wrote: On Wed, May 07, 2025 at 03:25:07AM +0200, Cédric Villemain wrote: Sure, modified this way. I have detected two more of these under NumProcSignalSlots and NumProcStateSlots. NUM_AUXILIARY_PROCS is adjusted in both cases with MAX_IO_WORKERS, which is

Re: Add time spent in posix_fadvise() in I/O read time ?

2025-05-06 Thread Cédric Villemain
uler also change the behavior of the queue, with mq-deadline there is 25% of the queue reserved for sync requests. See https://github.com/torvalds/linux/commit/07757588e5076748308dd95ee2e3cd0b82ebb8c4 * https://wiki.postgresql.org/wiki/Readahead - if you want to read/check/enhance its con

Re: minor fix related to Auxiliary processes and IO workers

2025-05-06 Thread Cédric Villemain
On 07/05/2025 01:52, Michael Paquier wrote: On Tue, May 06, 2025 at 06:25:24PM +0200, Cédric Villemain wrote: @@ -353,7 +353,7 @@ typedef enum BackendType * Auxiliary processes. These have PGPROC entries, but they are not * attached to any particular database, and cannot run

minor fix related to Auxiliary processes and IO workers

2025-05-06 Thread Cédric Villemain
Hello Andres, I noticed a comment in "include/miscadmin.h" which might be need a fix: it states that there is a single auxiliary process of each kind running at once. However, with IO workers it's not true anymore I believe. See minor patch attached. --- Cédric Villemain +3

Re: WAL-logging facility for pgstats kinds

2025-02-15 Thread Cédric Villemain
unction correctly during or after a switchover, and those (stats or) metrics that are irrelevant to PostgreSQL itself. --- Cédric Villemain +33 6 20 30 22 52 https://www.Data-Bene.io PostgreSQL Support, Expertise, Training, R&D

Re: PoC: history of recent vacuum/checkpoint runs (using new hooks)

2025-01-07 Thread Cédric Villemain
LRU or similar (the thing used to keep commits timestamp) ? I didn't checked all already available options in this area. [1] https://codeberg.org/Data-Bene/StatsMgr --- Cédric Villemain +33 6 20 30 22 52 https://www.Data-Bene.io PostgreSQL Support, Expertise, Training, R&D

Re: Crash: invalid DSA memory alloc request

2024-12-13 Thread Cédric Villemain
e interface will need to evolve as we come to understand the needs of different kinds of users. E.g support for iterators and incremental resizing is planned for later commits and the details of the callback signatures are likely to change. I'm unsure iterators and incremental resizing has

Re: Managing IO workers in postmaster's state machine

2024-12-13 Thread Cédric Villemain
x27;s a technical or historical reason for the current design, it would be useful to revisit that in this context. What prevent the number of background workers to be dynamically sized today ? --- Cédric Villemain +33 6 20 30 22 52 https://www.Data-Bene.io PostgreSQL Support, Expertise, Training, R&D

Re: Guidance Needed for Testing PostgreSQL Patch (CF-5044)

2024-11-29 Thread Cédric Villemain
k to https://www.postgresql.org/docs/current/source.html Which is a nice place to explore probably. I hope it'll help --- Cédric Villemain +33 6 20 30 22 52 https://www.Data-Bene.io PostgreSQL Support, Expertise, Training, R&D

Re: Change prefetch and read strategies to use range in pg_prewarm ... and raise a question about posix_fadvise WILLNEED

2024-03-07 Thread Cédric Villemain
Hi Nazir, On 07/03/2024 12:19, Nazir Bilal Yavuz wrote: On Wed, 6 Mar 2024 at 18:23, Cédric Villemain wrote: The behavior is 100% OK, and in fact it might a bad idea to prefetch block by block as the result is just to put more pressure on a system if it is already under pressure. Though

Re: Change prefetch and read strategies to use range in pg_prewarm ... and raise a question about posix_fadvise WILLNEED

2024-03-06 Thread Cédric Villemain
status, or you ensure to clean it first. [1] https://man7.org/linux/man-pages/man2/posix_fadvise.2.html#DESCRIPTION [2] https://elixir.bootlin.com/linux/latest/source/mm/readahead.c#L303 My apologize about the email address with sub-address which leads to undelivered email. Please update with the current one. --- Cédric Villemain +33 (0)6 20 30 22 52 https://Data-Bene.io PostgreSQL Expertise, Support, Training, R&D

linux cachestat in file Readv and Prefetch

2024-01-17 Thread Cédric Villemain
interest to go further in this direction. --- Cédric Villemain +33 (0)6 20 30 22 52 https://Data-Bene.io PostgreSQL Expertise, Support, Training, R&D From 4cdc5e952e0cd729bde472bdc3d7ff7bbc3009ea Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?C=C3=A9dric=20Villemain?= Date: Thu, 18 Jan 2024 01:00:37 +0

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2024-01-14 Thread Cédric Villemain
:25 AM, Cédric Villemain wrote: Hi Palak, I did a quick review of the patch: +CREATE FUNCTION pg_buffercache_invalidate(IN int, IN bool default true) +RETURNS bool +AS 'MODULE_PATHNAME', 'pg_buffercache_invalidate' +LANGUAGE C PARALLEL SAFE; --> Not enforced anywhere,

Re: doing also VM cache snapshot and restore with pg_prewarm, having more information of the VM inside PostgreSQL

2024-01-05 Thread Cédric Villemain
t proposing that PostgreSQL handles those tasks transparently or itself, but offering options to the users via extensions like we do with pg_prewarm and pg_buffercache. It's just the same for virtual memory/filesystem. --- Cédric Villemain +33 (0)6 20 30 22 52 https://Data-Bene.io PostgreSQL Expertise, Support, Training, R&D

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2024-01-03 Thread Cédric Villemain
} +    else +    { +        UnlockBufHdr(bufHdr, buf_state); +        return false; +    } Maybe safe to remove the else {} ... Maybe more tempting to start the big if with the following instead less nested...): +    if ((buf_state & BM_VALID) != BM_VALID) +    { +        UnlockBufHdr(bufHdr, buf

Re: Streaming I/O, vectored I/O (WIP)

2023-12-30 Thread Cédric Villemain
s patches here: * to use range prefetch in pg_prewarm (smgrprefetch only at the moment, using smgrreadv to come next). * to support nblocks=0 in smgrprefetch (posix_fadvise supports a len=0 to apply flag from offset to end of file). Should I add to commitfest ? --- Cédric Villemain +33 (0)6 20 3

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2023-11-22 Thread Cédric Villemain
counts', 0, true, true, B'111000'); relpath | os_page_size | os_pages_free | pages_loaded | pages_unloaded --+--+---+--+ base/11874/16447 | 4096 |408376 |3 | 3 ---