On Wed, Jan 22, 2025 at 01:00:08PM -0500, Andres Freund wrote:
> On 2025-01-22 11:43:20 -0600, Nathan Bossart wrote:
>> What is the purpose of syncing the file before the rename?
>
> It's from the general durable_rename() code. The reason it's there that it's
> required for "atomically replace a f
Hi,
On 2025-01-22 11:43:20 -0600, Nathan Bossart wrote:
> On Wed, Jan 22, 2025 at 11:21:03AM -0500, Andres Freund wrote:
> > fsync(open(oldname1));
> > fsync(open(oldname2));
> > ..
> > fsync(open(oldnameN));
> >
> > rename(oldname1, newname1);
> > rename(oldname2, newname2);
> > ..
> > rename(old
On Wed, Jan 22, 2025 at 11:21:03AM -0500, Andres Freund wrote:
> fsync(open(oldname1));
> fsync(open(oldname2));
> ..
> fsync(open(oldnameN));
>
> rename(oldname1, newname1);
> rename(oldname2, newname2);
> ..
> rename(oldnameN, newnameN);
>
> fsync(open(newname1));
> fsync(open(newname2));
> ..
Hi,
On 2025-01-22 01:14:22 +, Andy Fan wrote:
> Andres Freund writes:
> > FWIW, I've seen the fsyncs around recycling being a rather substantial
> > bottleneck. To the point of the main benefit of larger segments being the
> > reduction in number of fsyncs at the end of a checkpoint. I think
On Wed, Jan 22, 2025 at 01:14:22AM +, Andy Fan wrote:
> Andres Freund writes:
>> FWIW, I've seen the fsyncs around recycling being a rather substantial
>> bottleneck. To the point of the main benefit of larger segments being the
>> reduction in number of fsyncs at the end of a checkpoint. I t
On Tue, Jan 21, 2025 at 11:23:06AM -0500, Andres Freund wrote:
> On 2025-01-21 10:13:14 -0600, Nathan Bossart wrote:
>> On Tue, Jan 21, 2025 at 09:52:51AM -0600, Nathan Bossart wrote:
>> > On Tue, Jan 21, 2025 at 03:31:27AM +, Andy Fan wrote:
>> >> 3. Why is the purpose of preallocated_segments
Andres Freund writes:
Hi,
> FWIW, I've seen the fsyncs around recycling being a rather substantial
> bottleneck. To the point of the main benefit of larger segments being the
> reduction in number of fsyncs at the end of a checkpoint. I think we should
> be able to make the fsyncs a lot more ef
Hi,
On 2025-01-21 10:13:14 -0600, Nathan Bossart wrote:
> On Tue, Jan 21, 2025 at 09:52:51AM -0600, Nathan Bossart wrote:
> > On Tue, Jan 21, 2025 at 03:31:27AM +, Andy Fan wrote:
> >> 3. Why is the purpose of preallocated_segments directory? what in my
> >> mind is we just prellocate the norm
On Tue, Jan 21, 2025 at 09:52:51AM -0600, Nathan Bossart wrote:
> On Tue, Jan 21, 2025 at 03:31:27AM +, Andy Fan wrote:
>> 3. Why is the purpose of preallocated_segments directory? what in my
>> mind is we just prellocate the normal filename so that XLogWrite could
>> open it directly. This is
On Tue, Jan 21, 2025 at 03:31:27AM +, Andy Fan wrote:
> Come from [0] and thanks for working on this. Here are some design
> review/question after my first going through the patch.
Thanks for taking a look.
> 1. walwriter vs checkpointer? I prefer to walwriter for now because..
>
> a. chec
Hi Nathan,
Come from [0] and thanks for working on this. Here are some design
review/question after my first going through the patch.
1. walwriter vs checkpointer? I prefer to walwriter for now because..
a. checkpointer is hard to do it in a timely manner either because
checkpoint itself may
On Thu, Jul 14, 2022 at 11:34:07AM -0700, Nathan Bossart wrote:
> It's now been over a year since I first posted a patch in this thread, and
> I still sense very little interest for this feature. I intend to mark it
> as Withdrawn at the end of this commitfest.
Done.
--
Nathan Bossart
Amazon We
On Fri, Apr 08, 2022 at 01:30:03PM -0700, Nathan Bossart wrote:
> On Thu, Mar 17, 2022 at 04:12:12PM -0700, Nathan Bossart wrote:
>> It seems unlikely that this will be committed for v15, so I've adjusted the
>> commitfest entry to v16 and moved it to the next commitfest.
>
> rebased
It's now bee
On Thu, Mar 17, 2022 at 04:12:12PM -0700, Nathan Bossart wrote:
> It seems unlikely that this will be committed for v15, so I've adjusted the
> commitfest entry to v16 and moved it to the next commitfest.
rebased
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
>From 3781795f9b4e448
It seems unlikely that this will be committed for v15, so I've adjusted the
commitfest entry to v16 and moved it to the next commitfest.
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
On Tue, Mar 01, 2022 at 08:40:44AM -0600, Justin Pryzby wrote:
> FYI: this is currently failing in cfbot on linux.
>
> https://cirrus-ci.com/task/4934371210690560
> https://api.cirrus-ci.com/v1/artifact/task/4934371210690560/log/src/test/regress/regression.diffs
>
> DROP TABLESPACE regress_tblsp
On Thu, Dec 30, 2021 at 02:51:10PM +0300, Maxim Orlov wrote:
> I did check the patch too and found it to be ok. Check and check-world are
> passed.
FYI: this is currently failing in cfbot on linux.
https://cirrus-ci.com/task/4934371210690560
https://api.cirrus-ci.com/v1/artifact/task/493437121069
e
> > >> adding this feature. I did tests myself a couple of months ago and got
> > >> similar results.
> > >> Really don't know whether is it worth the effort.
> >
> > Thank you both for your review.
>
> It may have been discussed earlier
t; Really don't know whether is it worth the effort.
>
> Thank you both for your review.
It may have been discussed earlier, let me ask this here - IIUC the
whole point of pre-allocating WAL files is that creating new WAL files
of wal_segment_size requires us to write zero-filled empt
On 12/30/21, 3:52 AM, "Maxim Orlov" wrote:
> I did check the patch too and found it to be ok. Check and check-world are
> passed.
> Overall idea seems to be good in my opinion, but I'm not sure where is the
> optimal place to put the pre-allocation.
>
> On Thu, Dec 30, 2021 at 2:46 PM Pavel Bor
I did check the patch too and found it to be ok. Check and check-world are
passed.
Overall idea seems to be good in my opinion, but I'm not sure where is the
optimal place to put the pre-allocation.
On Thu, Dec 30, 2021 at 2:46 PM Pavel Borisov
wrote:
> > pre-allocating during checkpoints. I've
>
> > pre-allocating during checkpoints. I've done a few pgbench runs, and
> > it seems to work pretty well. Initialization is around 15% faster,
> > and I'm seeing about a 5% increase in TPS with a simple-update
> > workload with wal_recycle turned off. Of course, these improvements
> > go away
On 12/7/21, 9:35 AM, "Bossart, Nathan" wrote:
> On 12/7/21, 12:29 AM, "Bharath Rupireddy"
> wrote:
>> Why can't the walwriter pre-allocate some of the WAL segments instead
>> of a new background process? Of course, it might delay the main
>> functionality of the walwriter i.e. flush and sync the
On 12/7/21, 12:29 AM, "Bharath Rupireddy"
wrote:
> Why can't the walwriter pre-allocate some of the WAL segments instead
> of a new background process? Of course, it might delay the main
> functionality of the walwriter i.e. flush and sync the WAL files, but
> having checkpointer do the pre-alloc
On Thu, Nov 11, 2021 at 12:29 AM Bossart, Nathan wrote:
>
> On 10/8/21, 1:55 PM, "Bossart, Nathan" wrote:
> > Here is a first attempt at adding the pre-allocation logic to the
> > checkpointer. I went ahead and just used CheckpointWriteDelay() for
> > pre-allocating during checkpoints. I've don
On 10/6/21, 5:20 AM, "Maxim Orlov" wrote:
> We've looked through the code and everything looks good except few minor
> things:
> 1). Using dedicated bg worker seems not optimal, it introduces a lot of
> redundant code for little single action.
> We'd join initial proposal of Andres to implem
The following review has been posted through the commitfest application:
make installcheck-world: tested, passed
Implements feature: tested, failed
Spec compliant: not tested
Documentation:not tested
Hi!
We've looked through the code and everything looks good except f
On Mon, Jun 7, 2021 at 8:48 PM Bossart, Nathan wrote:
>
> On 12/25/20, 12:09 PM, "Andres Freund" wrote:
> > When running write heavy transactional workloads I've many times
> > observed that one needs to run the benchmarks for quite a while till
> > they get to their steady state performance. The
Hi,
When running write heavy transactional workloads I've many times
observed that one needs to run the benchmarks for quite a while till
they get to their steady state performance. The most significant reason
for that is that initially WAL files will not get recycled, but need to
be freshly initi
29 matches
Mail list logo
