Re: Get rid of WALBufMappingLock

2025-07-06 Thread Alexander Korotkov
On Sat, Jul 5, 2025 at 7:16 AM Noah Misch wrote: > > On Mon, Mar 31, 2025 at 09:18:30PM +0300, Alexander Korotkov wrote: > > I'm going to push the first patch ("nowalbuf") if no objections. > > I completed a post-commit review of this patch. I think the patch is correct. Great. I'm glad to hear

Re: Get rid of WALBufMappingLock

2025-07-04 Thread Noah Misch
On Mon, Mar 31, 2025 at 09:18:30PM +0300, Alexander Korotkov wrote: > I'm going to push the first patch ("nowalbuf") if no objections. I completed a post-commit review of this patch. I think the patch is correct. I found some of the variable names and comments challenging, so I made the attache

Re: Get rid of WALBufMappingLock

2025-04-05 Thread Alexander Korotkov
On Mon, Mar 31, 2025 at 1:42 PM Yura Sokolov wrote: > 14.03.2025 17:30, Tomas Vondra wrote: > > Hi, > > > > I've briefly looked at this patch this week, and done a bit of testing. > > I don't have any comments about the correctness - it does seem correct > > to me and I haven't noticed any crashes

Re: Get rid of WALBufMappingLock

2025-03-31 Thread Yura Sokolov
Good day, 14.03.2025 17:30, Tomas Vondra wrote: > Hi, > > I've briefly looked at this patch this week, and done a bit of testing. > I don't have any comments about the correctness - it does seem correct > to me and I haven't noticed any crashes/issues, but I'm not familiar > with the WALBufMappin

Re: Get rid of WALBufMappingLock

2025-03-21 Thread Yura Sokolov
Good day, Tomas 14.03.2025 17:30, Tomas Vondra wrote: > > Yes, with tmpfs the impact looks much more significant. For 8K the > speedup is ~1.3x, for 64K it's up to ~2x, for 1M it's ~1.1x. > > > That being said, I wonder how big is the impact for practical workloads. > ISTM this workload is pret

Re: Get rid of WALBufMappingLock

2025-03-14 Thread Tomas Vondra
Hi, I've briefly looked at this patch this week, and done a bit of testing. I don't have any comments about the correctness - it does seem correct to me and I haven't noticed any crashes/issues, but I'm not familiar with the WALBufMappingLock enough to have insightful opinions. I have however dec

Re: Get rid of WALBufMappingLock

2025-03-13 Thread Alexander Korotkov
On Fri, Mar 7, 2025 at 5:08 PM Alexander Korotkov wrote: > On Sun, Mar 2, 2025 at 1:58 PM Alexander Korotkov > wrote: > > > > On Fri, Feb 28, 2025 at 3:13 PM Álvaro Herrera > > wrote: > > > On 2025-Feb-28, Michael Paquier wrote: > > > > > > > Saying that, I have also done similar tests with yo

Re: Get rid of WALBufMappingLock

2025-03-07 Thread Alexander Korotkov
On Sun, Mar 2, 2025 at 1:58 PM Alexander Korotkov wrote: > > On Fri, Feb 28, 2025 at 3:13 PM Álvaro Herrera > wrote: > > On 2025-Feb-28, Michael Paquier wrote: > > > > > Saying that, I have also done similar tests with your v12 for a couple > > > of hours and this looks stable under installcheck

Re: Get rid of WALBufMappingLock

2025-03-02 Thread Alexander Korotkov
On Fri, Feb 28, 2025 at 3:44 PM Michael Paquier wrote: > > On Fri, Feb 28, 2025 at 02:13:23PM +0100, Álvaro Herrera wrote: > > On 2025-Feb-28, Michael Paquier wrote: > >> Saying that, I have also done similar tests with your v12 for a couple > >> of hours and this looks stable under installcheck-w

Re: Get rid of WALBufMappingLock

2025-03-02 Thread Alexander Korotkov
On Fri, Feb 28, 2025 at 3:13 PM Álvaro Herrera wrote: > On 2025-Feb-28, Michael Paquier wrote: > > > Saying that, I have also done similar tests with your v12 for a couple > > of hours and this looks stable under installcheck-world. I can see > > that you've reworked quite a bit the surroundings

Re: Get rid of WALBufMappingLock

2025-02-28 Thread Yura Sokolov
26.02.2025 11:52, Andrey Borodin wrote: >> On 25 Feb 2025, at 20:19, Alexander Korotkov wrote: >> >> > > > Hi! > > One little piece of code looks suspicious to me. But I was not raising > concern because I see similar code everywhere in the codebase. But know > Kirill asked to me explain what

Re: Get rid of WALBufMappingLock

2025-02-28 Thread Michael Paquier
On Fri, Feb 28, 2025 at 02:13:23PM +0100, Álvaro Herrera wrote: > On 2025-Feb-28, Michael Paquier wrote: >> Saying that, I have also done similar tests with your v12 for a couple >> of hours and this looks stable under installcheck-world. I can see >> that you've reworked quite a bit the surroundi

Re: Get rid of WALBufMappingLock

2025-02-28 Thread Álvaro Herrera
On 2025-Feb-28, Michael Paquier wrote: > Saying that, I have also done similar tests with your v12 for a couple > of hours and this looks stable under installcheck-world. I can see > that you've reworked quite a bit the surroundings of InitializedFrom > in this one. If you apply that once again

Re: Get rid of WALBufMappingLock

2025-02-28 Thread Yura Sokolov
26.02.2025 14:48, Alexander Korotkov пишет: > Hi, Michael! > > On Wed, Feb 26, 2025 at 3:04 AM Michael Paquier wrote: >> >> On Tue, Feb 25, 2025 at 05:19:29PM +0200, Alexander Korotkov wrote: >>> It seems that I managed to reproduce the issue on my Raspberry PI 4. >>> After running our test suite

Re: Get rid of WALBufMappingLock

2025-02-27 Thread Michael Paquier
On Wed, Feb 26, 2025 at 01:48:47PM +0200, Alexander Korotkov wrote: > Thank you for offering the help. Updated version of patch is attached > (I've added one memory barrier there just in case). I would > appreciate if you could run it on batta, hachi or similar hardware. Doing a revert of the re

Re: Get rid of WALBufMappingLock

2025-02-26 Thread Alexander Korotkov
Hi, Michael! On Wed, Feb 26, 2025 at 3:04 AM Michael Paquier wrote: > > On Tue, Feb 25, 2025 at 05:19:29PM +0200, Alexander Korotkov wrote: > > It seems that I managed to reproduce the issue on my Raspberry PI 4. > > After running our test suite in a loop for 2 days I found one timeout. > > Hmm.

Re: Get rid of WALBufMappingLock

2025-02-26 Thread Andrey Borodin
> On 25 Feb 2025, at 20:19, Alexander Korotkov wrote: > > Hi! One little piece of code looks suspicious to me. But I was not raising concern because I see similar code everywhere in the codebase. But know Kirill asked to me explain what is going on and I cannot. This seems to be relevan

Re: Get rid of WALBufMappingLock

2025-02-25 Thread Michael Paquier
On Tue, Feb 25, 2025 at 05:19:29PM +0200, Alexander Korotkov wrote: > It seems that I managed to reproduce the issue on my Raspberry PI 4. > After running our test suite in a loop for 2 days I found one timeout. Hmm. It's surprising to not see a higher occurence. My buildfarm host has caught tha

Re: Get rid of WALBufMappingLock

2025-02-25 Thread Alexander Korotkov
On Tue, Feb 18, 2025 at 2:29 AM Alexander Korotkov wrote: > > On Tue, Feb 18, 2025 at 2:21 AM Michael Paquier wrote: > > > > On Mon, Feb 17, 2025 at 11:25:05AM -0500, Tom Lane wrote: > > > This timeout failure on hachi looks suspicious as well: > > > > > > https://buildfarm.postgresql.org/cgi-bin

Re: Get rid of WALBufMappingLock

2025-02-17 Thread Alexander Korotkov
On Tue, Feb 18, 2025 at 2:21 AM Michael Paquier wrote: > > On Mon, Feb 17, 2025 at 11:25:05AM -0500, Tom Lane wrote: > > This timeout failure on hachi looks suspicious as well: > > > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hachi&dt=2025-02-17%2003%3A05%3A03 > > > > Might be relev

Re: Get rid of WALBufMappingLock

2025-02-17 Thread Michael Paquier
On Mon, Feb 17, 2025 at 11:25:05AM -0500, Tom Lane wrote: > This timeout failure on hachi looks suspicious as well: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hachi&dt=2025-02-17%2003%3A05%3A03 > > Might be relevant that they are both aarch64? Just logged into the host. The log

Re: Get rid of WALBufMappingLock

2025-02-17 Thread Pavel Borisov
Oops, I send wrong patch as a fix. The right one is attached. Pavel On Mon, 17 Feb 2025 at 13:40, Pavel Borisov wrote: > > Hi, Kirill! > Per your report, I revised the comment to fix typos. Also some little > changes in grammar. > > Kind regards, > Pavel Borisov v2-0001-Fix-typo-and-grammar-in

Re: Get rid of WALBufMappingLock

2025-02-17 Thread Tom Lane
Alexander Korotkov writes: > I've spotted the failure on the buildfarm. > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=batta&dt=2025-02-17%2008%3A05%3A03 > I can't quickly guess the reason. I'm going to revert patch for now, > then we investigate This timeout failure on hachi looks su

Re: Get rid of WALBufMappingLock

2025-02-17 Thread Pavel Borisov
Hi, Victor! On Mon, 17 Feb 2025 at 12:47, Victor Yegorov wrote: > > Hey. > > I find “Get rid of WALBufMappingLock" commit message misleading, 'cos Lock > it's being replaced by CV, actually. > > Should the subject be changed to “Replace WALBufMappingLock with

Re: Get rid of WALBufMappingLock

2025-02-17 Thread Alexander Korotkov
On Mon, Feb 17, 2025 at 11:44 AM Pavel Borisov wrote: > Oops, I send wrong patch as a fix. > The right one is attached. > > Pavel I've spotted the failure on the buildfarm. https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=batta&dt=2025-02-17%2008%3A05%3A03 I can't quickly guess the reason.

Re: Get rid of WALBufMappingLock

2025-02-17 Thread Pavel Borisov
Hi, Kirill! Per your report, I revised the comment to fix typos. Also some little changes in grammar. Kind regards, Pavel Borisov 0001-Fix-typo-and-grammar-in-comment-introduced-by-6a2275.patch Description: Binary data

Re: Get rid of WALBufMappingLock

2025-02-17 Thread Pavel Borisov
On Mon, 17 Feb 2025 at 13:20, Pavel Borisov wrote: > > Hi, Victor! > > On Mon, 17 Feb 2025 at 12:47, Victor Yegorov wrote: > > > > Hey. > > > > I find “Get rid of WALBufMappingLock" commit message misleading, 'cos Lock > > it's bein

Re: Get rid of WALBufMappingLock

2025-02-17 Thread Victor Yegorov
Hey. I find “Get rid of WALBufMappingLock" commit message misleading, 'cos Lock it's being replaced by CV, actually. Should the subject be changed to “Replace WALBufMappingLock with ConditionVariable” instead? -- Victor Yegorov

Re: Get rid of WALBufMappingLock

2025-02-16 Thread Kirill Reshke
Hi! I spotted a typo in v10: + /* + * Page at nextidx wasn't initialized yet, so we cann't move + * InitializedUpto further. It will be moved by backend which + * will initialize nextidx. + */ cann't - > can't moved by backend -> moved by the backend -- Best regards, Kirill Reshke

Re: Get rid of WALBufMappingLock

2025-02-15 Thread Alexander Korotkov
Hi! On Fri, Feb 14, 2025 at 4:11 PM Yura Sokolov wrote: > 14.02.2025 17:09, Yura Sokolov пишет: > > 14.02.2025 13:24, Alexander Korotkov пишет: > >> On Fri, Feb 14, 2025 at 11:45 AM Pavel Borisov > >> wrote: > >>> On Fri, 14 Feb 2025 at 00:59, Alexander Korotkov > >>> wrote: > On Thu, Fe

Re: Get rid of WALBufMappingLock

2025-02-14 Thread Yura Sokolov
;>>> >>>> Oh, sorry, I really did wrong. I've done git format-patch for wrong >>>> local branch for v5 and v6. Patches I've sent for v5 and v6 are >>>> actually the same as my v1. This is really pity. Please, find the >>>> right version of patchset att

Re: Get rid of WALBufMappingLock

2025-02-14 Thread Yura Sokolov
d v6. Patches I've sent for v5 and v6 are >>> actually the same as my v1. This is really pity. Please, find the >>> right version of patchset attached. >> >> I've rechecked v7. In v6 a proposal from [1] was not reflected. Now it >> landed in v7. >&

Re: Get rid of WALBufMappingLock

2025-02-14 Thread Alexander Korotkov
On Fri, Feb 14, 2025 at 11:45 AM Pavel Borisov wrote: > On Fri, 14 Feb 2025 at 00:59, Alexander Korotkov wrote: > > On Thu, Feb 13, 2025 at 6:39 PM Pavel Borisov > > wrote: > > > On Thu, 13 Feb 2025 at 14:08, Alexander Korotkov > > > wrote: > > > > > > > > On Thu, Feb 13, 2025 at 11:45 AM Yur

Re: Get rid of WALBufMappingLock

2025-02-14 Thread Pavel Borisov
Hi, Alexander! On Fri, 14 Feb 2025 at 00:59, Alexander Korotkov wrote: > > Hi, Pavel! > > On Thu, Feb 13, 2025 at 6:39 PM Pavel Borisov wrote: > > On Thu, 13 Feb 2025 at 14:08, Alexander Korotkov > > wrote: > > > > > > On Thu, Feb 13, 2025 at 11:45 AM Yura Sokolov > > > wrote: > > > > 13.02.

Re: Get rid of WALBufMappingLock

2025-02-13 Thread Alexander Korotkov
Hi, Pavel! On Thu, Feb 13, 2025 at 6:39 PM Pavel Borisov wrote: > On Thu, 13 Feb 2025 at 14:08, Alexander Korotkov wrote: > > > > On Thu, Feb 13, 2025 at 11:45 AM Yura Sokolov > > wrote: > > > 13.02.2025 12:34, Alexander Korotkov пишет: > > > > On Wed, Feb 12, 2025 at 8:16 PM Yura Sokolov >

Re: Get rid of WALBufMappingLock

2025-02-13 Thread Pavel Borisov
Hi, Yura and Alexander! On Thu, 13 Feb 2025 at 14:08, Alexander Korotkov wrote: > > On Thu, Feb 13, 2025 at 11:45 AM Yura Sokolov > wrote: > > 13.02.2025 12:34, Alexander Korotkov пишет: > > > On Wed, Feb 12, 2025 at 8:16 PM Yura Sokolov > > > wrote: > > >> 08.02.2025 13:07, Alexander Korotko

Re: Get rid of WALBufMappingLock

2025-02-13 Thread Alexander Korotkov
On Thu, Feb 13, 2025 at 11:45 AM Yura Sokolov wrote: > 13.02.2025 12:34, Alexander Korotkov пишет: > > On Wed, Feb 12, 2025 at 8:16 PM Yura Sokolov > > wrote: > >> 08.02.2025 13:07, Alexander Korotkov пишет: > >>> On Fri, Feb 7, 2025 at 1:39 PM Alexander Korotkov > >>> wrote: > Good, than

Re: Get rid of WALBufMappingLock

2025-02-13 Thread Yura Sokolov
13.02.2025 12:34, Alexander Korotkov пишет: > On Wed, Feb 12, 2025 at 8:16 PM Yura Sokolov wrote: >> 08.02.2025 13:07, Alexander Korotkov пишет: >>> On Fri, Feb 7, 2025 at 1:39 PM Alexander Korotkov >>> wrote: Good, thank you. I think 0001 patch is generally good, but needs some furth

Re: Get rid of WALBufMappingLock

2025-02-13 Thread Alexander Korotkov
On Wed, Feb 12, 2025 at 8:16 PM Yura Sokolov wrote: > 08.02.2025 13:07, Alexander Korotkov пишет: > > On Fri, Feb 7, 2025 at 1:39 PM Alexander Korotkov > > wrote: > >> Good, thank you. I think 0001 patch is generally good, but needs some > >> further polishing, e.g. more comments explaining how

Re: Get rid of WALBufMappingLock

2025-02-12 Thread Yura Sokolov
-b3d4-67ff9a00d447%40postgrespro.ru --- regards Yura Sokolov aka funny-falconFrom d243176a52faf6a57edd5719cfcbe8b3d1bbb919 Mon Sep 17 00:00:00 2001 From: Yura Sokolov Date: Sat, 18 Jan 2025 23:50:09 +0300 Subject: [PATCH v4 1/2] Get rid of WALBufMappingLock Allow many backends to concurrently initi

Re: Get rid of WALBufMappingLock

2025-02-08 Thread Alexander Korotkov
On Fri, Feb 7, 2025 at 1:39 PM Alexander Korotkov wrote: > Good, thank you. I think 0001 patch is generally good, but needs some > further polishing, e.g. more comments explaining how does it work. Two things comes to my mind worth rechecking about 0001. 1) Are XLogCtl->InitializeReserved, XLogC

Re: Get rid of WALBufMappingLock

2025-02-07 Thread Alexander Korotkov
On Fri, Feb 7, 2025 at 1:24 PM Yura Sokolov wrote: > 07.02.2025 14:02, Yura Sokolov пишет: > > 07.02.2025 01:26, Alexander Korotkov пишет: > >> Hi! > >> > >> On Sun, Jan 19, 2025 at 2:11 AM Yura Sokolov > >> wrote: > >>> > >>> During discussion of Increasing NUM_XLOGINSERT_LOCKS [1], Andres Freu

Re: Get rid of WALBufMappingLock

2025-02-07 Thread Yura Sokolov
ched. > > I've changed condition for broadcast a bit ("less" instead "not equal"): > - buffer's border may already go into future, > - and then other backend will reach not yet initialized buffer and will > broadcast. 2. I've inserted abort if &q

Re: Get rid of WALBufMappingLock

2025-02-07 Thread Yura Sokolov
nd clearer, I agree to keep just single condition variable. Cleaned version is attached. I've changed condition for broadcast a bit ("less" instead "not equal"): - buffer's border may already go into future, - and then other backend will reach not yet initialized buff

Re: Get rid of WALBufMappingLock

2025-02-06 Thread Alexander Korotkov
Hi! On Sun, Jan 19, 2025 at 2:11 AM Yura Sokolov wrote: > > During discussion of Increasing NUM_XLOGINSERT_LOCKS [1], Andres Freund > used benchmark which creates WAL records very intensively. While I this > it is not completely fair (1MB log records are really rare), it pushed > me to analyze wr

Re: Get rid of WALBufMappingLock

2025-01-18 Thread Yura Sokolov
19.01.2025 03:11, Yura Sokolov пишет: Good day, hackers. During discussion of Increasing NUM_XLOGINSERT_LOCKS [1], Andres Freund used benchmark which creates WAL records very intensively. While I this it is not completely fair (1MB log records are really rare), it pushed me to analyze write-s

Get rid of WALBufMappingLock

2025-01-18 Thread Yura Sokolov
27 /2133 So, patched version with increased NUM_XLOGINSERT_LOCKS looks no worse than unpatched without increasing num of locks. --- regards Yura Sokolov aka funny-falconFrom 236b69ae1f524c7e8488da7244966e631324a0e3 Mon Sep 17 00:00:00 2001 From: Yura Sokolov Date: Sat, 18 Jan 2025