page_collect_tuples without long lock on page (Was Re: IPC/MultixactCreation on the Standby server)

y dirty variant is in attach. I've made it just for fun. It passes 'regress', 'isolation' and 'recovery'. But I didn't benchmark it. -- regards Yura Sokolov aka funny-falcondiff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c

Re: Add 64-bit XIDs into PostgreSQL 15

But still possible. More over, it is hard for the bug to trigger real problems, because INSERT/UPDATE/DELETE uses tuple numbers to describe operation, not offsets. So operations are executed correctly... unless there were switch of master, and replica did more insertions into same page, so it overflowed free space on previous master, and full page write were disabled. So, the bug is real. But looks like it is quite minor. (Disclaimer: I didn't analyze code again to write text above, so I could be mistaken somewhere). -- regards Yura Sokolov aka funny-falcon

Re: Read-Write optimistic lock (Re: sinvaladt.c: remove msgnumLock, use atomic operations on maxMsgNum)

rriers were changed to full, but one necessary at the place you'd pointed to. But you just answered then [2]: > I don't believe we have the whole story here. [1] https://www.postgresql.org/message-id/b798eb5e-35b7-40b5-a245-4170deab56f8%40postgrespro.ru [2] https://www.postgresql.

Re: Optimize shared LWLock acquisition for high-core-count systems

10.07.2025 18:57, Zhou, Zhiguo пишет: > > > On 7/9/2025 3:56 PM, Yura Sokolov wrote: >> 30.05.2025 14:30, Zhou, Zhiguo пишет: >>> Hi Hackers, >>> >>> I am reaching out to solicit your insights and comments on this patch >>> addressing a signi

Re: Optimize shared LWLock acquisition for high-core-count systems

x27;m wrong? When I tried to do same thing, I did sub_fetch immediately in case of acquisition failure. And did no add_fetch at all if lock is held in Exclusive mode. BTW, there's way to optimize EXCLUSIVE lock as well since there's no need to do CAS if lock is held by someone else. Se

Re: Add 64-bit XIDs into PostgreSQL 15

Therefore, "fully excluding repairFragmentation=false" is not wise. But... probably we could try to acquire "cleanup lock" to perform prune_freeze with "repairFragmentation=true", and fallback to "repairFragmentation=false" in case of acquire failure. Still, I repeat, "excluding repairFragmentation=false" completely doesn't look feasible to me. -- regards Yura Sokolov aka funny-falcon

Re: Add 64-bit XIDs into PostgreSQL 15

ween pd_lower and pd_upper). Before last version, empty space between tuples with "repair_fragmentation=false" was certainly possible source of difference not masked by standard mask procedure. But now those holes are zeroed. -- regards Yura Sokolov aka funny-falcon

Re: Read-Write optimistic lock (Re: sinvaladt.c: remove msgnumLock, use atomic operations on maxMsgNum)

04.06.2025 00:04, Andres Freund пишет: > Hi, > > On 2025-06-02 21:20:33 +0300, Yura Sokolov wrote: >> But still problem of spin lock contention is here. > > I still would like to see a reproducer for this. For problem in sinvaladt.c we have no synthetic reproducer. But

Re: Add 64-bit XIDs into PostgreSQL 15

will perform cleanup is allowed to PIN the page. UPDATE/INSERT/DELETE lock CONTENT LOCK in EXCLUSIVE mode because they may add new tuples. But they are not allowed to move tuples because concurrent backends allowed to read tuples from the page in exactly same moment. -- regards Yura Sokolov aka funny-falcon

Re: C11 / VS 2019

n't > think so at quick glance through the VS docs, unfortunately, but I may > be wrong, of course.. Will it mean we can implement atomics in term of C11 atomics? Aside for VS 2019, which has no support for. (But VS 2022 already has.) So instead of numerous variants we could just differ VS2019 vs plain C11. -- regards Yura Sokolov aka funny-falcon

Re: ARM/ARM64 SpinLockAcquire and SpinLockRelease are not full barriers

Good day, hackers. 13.05.2025 19:22, Yura Sokolov wrote: > Good day. > > During conversation about other patch, Andres pointed out SpinLockAcquire > have semantic of full memory barrier [1] . > > spin.h also states: > >> Load and stores operation in calling co

Re: SpinLockAcquire and SpinLockRelease is broken on ARM/ARM64? (Re: sinvaladt.c: remove msgnumLock, use atomic operations on maxMsgNum)

Good day, Andres. Good day, hackers. 05.05.2025 10:30, Yura Sokolov wrote: > 21.03.2025 19:33, Andres Freund пишет: >> Hi, >> >> On 2025-03-21 14:35:16 +0300, Yura Sokolov wrote: >>> From 080c9e0de5e6e10751347e1ff50b65df424744cb Mon Sep 17 00:00:00 2001 >>&g

Re: Speedup truncations of temporary relation forks

e for RelFileLocator+forknum + included datastructure for BlockNumber (hash table or radix tree). Then there will no be need to iterate whole local buffers for each relation. Given local buffers are not target for concurrent access, both hash tables could be implemented using simplehash. It will compensate two-stage lookup, given dynahash is much slower than simplehash. -- regards Yura Sokolov aka funny-falcon

ARM/ARM64 SpinLockAcquire and SpinLockRelease are not full barriers

ndex-_005f_005fsync_005flock_005ftest_005fand_005fset [3] https://developer.arm.com/documentation/100069/0606/Data-Transfer-Instructions/SWPA--SWPAL--SWP--SWPL--SWPAL--SWP--SWPL [4] https://godbolt.org/z/h5P9fjzWd [5] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65697 -- regards Yura Sokolov aka fun

Re: bug: virtual generated column can be partition key

06.05.2025 13:31, jian he пишет: > On Tue, May 6, 2025 at 5:57 PM Yura Sokolov wrote: >> >> 21.04.2025 05:30, jian he пишет: >>> hi. >>> While trying to make the virtual generated column be part of the partition >>> key, >>> I found this bug. &g

Re: bug: virtual generated column can be partition key

omments. [1] https://www.postgresql.org/docs/17/ddl-partitioning.html#DDL-PARTITIONING-DECLARATIVE [2] https://github.com/postgres/postgres/blob/caa76b91a60681dff0bf193b64d4dcdc1014e036/src/backend/commands/tablecmds.c#L19735-L19741 [3] https://github.com/postgres/postgres/blob/caa76b91a60681dff0bf193b64d4dcdc1014e036/src/backend/commands/tablecmds.c#L19821-L19828 -- regards Yura Sokolov aka funny-falcon

Re: Fix a race condition in ConditionVariableTimedSleep()

05.05.2025 17:05, Bertrand Drouvot пишет: > Hi, > > On Mon, May 05, 2025 at 02:15:15PM +0300, Yura Sokolov wrote: >> Interestingly, our colleague stepped into same problem recently [1] . It >> happened because he attempted to make overcomplex timeout (SIGALARM) handler. >&

Re: Fix a race condition in ConditionVariableTimedSleep()

hange.patch And I believe, his solution is more elegant. Doesn't it? But in first step, I doubt there should be any thing that cancels condition variable during WaitLatch. Most probably you did wrong thing. We convinced the colleague to rework the code to not trigger the issue in first place. -- regards Yura Sokolov aka funny-falcon

SpinLockAcquire and SpinLockRelease is broken on ARM/ARM64? (Re: sinvaladt.c: remove msgnumLock, use atomic operations on maxMsgNum)

21.03.2025 19:33, Andres Freund пишет: > Hi, > > On 2025-03-21 14:35:16 +0300, Yura Sokolov wrote: >> From 080c9e0de5e6e10751347e1ff50b65df424744cb Mon Sep 17 00:00:00 2001 >> From: Yura Sokolov >> Date: Mon, 3 Feb 2025 11:58:33 +0300 >> Subject: [PATCH v2] sinv

Re: [RFC] Lock-free XLog Reservation from WAL

Just rebase -- regards Yura Sokolov aka funny-falconFrom 4ea25d6feb655a072d1e9f40a547dc6aeab762ac Mon Sep 17 00:00:00 2001 From: Yura Sokolov Date: Sun, 19 Jan 2025 17:40:28 +0300 Subject: [PATCH v6] Lock-free XLog Reservation using lock-free hash-table Removed PrevBytePos to eliminate lock

Re: Built-in Raft replication

- setting up cluster, provisioning machines / VMs > - setting up networking > - starting PostgreSQL servers. > - spinning up and down clients, > - communicating current leader and replica set (could be done by DNS > with agreed conventions) -- regards Yura Sokolov aka funny-falcon

Re: Built-in Raft replication

but also for reliability as almost full featured third server. Certainly, it may become only "read-only master" - just to replicate WAL's tail it has and commit it by commiting record in new term/timeline. Then it should give leadership to other replica immediately. This idea is not a fantasy. BiHA does it. -- regards Yura Sokolov aka funny-falcon

Re: Built-in Raft replication

estart when PostgreSQL needs to rewind and restart. Yes, there are non-trivial amount of changes made into postmaster machinery. But it is doable. -- regards Yura Sokolov aka funny-falcon

Re: Built-in Raft replication

esenting > the project at conferences which costs money. Or perhaps people are > just happy with Patroni. I'm not sure in which state Stolon is today. But the key point: if PostgreSQL will be improved a bit, there will be no need neither in Patroni, nor in Stolon. Isn't it great? -- regards Yura Sokolov aka funny-falcon

Re: Built-in Raft replication

and up) as their primary storage. And it seemed to me as "God Send". Everything just worked. Replication was as reliable as one could imagine. It outlives several hardware incidents without manual intervention. It allowed cluster maintenance (software and hardware upgrades) without application downtime. I really dream PostgreSQL will be as reliable as MongoDB without need of external services. -- regards Yura Sokolov aka funny-falcon

Re: Built-in Raft replication

of the biggest issues is forced snapshot on replica promotion. It really slows down leader switch time. It looks like it is not really needed, or some small workaround should be enough. -- regards Yura Sokolov aka funny-falcon

Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

huge. Even when they are in "columnar" format, which usually consumes less space. And memory is still costs more than disk space. Certainly there are users who think they need "in-memory". But the truth is very few of them really need "in-memory". All of this is just

Re: [RFC] Lock-free XLog Reservation from WAL

of this patch is: Waiting on Author Thank you very much for pointing on! Yes, I've missed copying from XLogCtl as it is done for WALInsertLocks. Fixed. -- regards Yura Sokolov aka funny-falconFrom 8cc3f9c7d629ce7fedd15f224df7566fd723cb06 Mon Sep 17 00:00:00 2001 From: Yura Sokolov Date: Sun,

Re: Optimize truncation logic to reduce AccessExclusive lock impact

the question left in the comment below is "Yes". > > /* > * Space/time tradeoff parameters: do these need to be user-tunable? Well, the patch changes formula as well. It is harder to implement user-tunable formula. --- regards Yura Sokolov

Re: Get rid of WALBufMappingLock

27;improvements.out' . It shows, removing WALBufMappingLock almost always doesn't harm performance and usually gives measurable gain. (Numbers are average from 4 middle runs out of 6. i.e. I threw minimum

Re: sinvaladt.c: remove msgnumLock, use atomic operations on maxMsgNum

25.03.2025 13:52, Yura Sokolov пишет: > Good day, Andres > > 24.03.2025 16:08, Andres Freund wrote: >> On 2025-03-24 13:41:17 +0300, Yura Sokolov wrote: >>> 21.03.2025 19:33, Andres Freund wrote: >>>> I'd also like to know a bit more about the motivation

Re: sinvaladt.c: remove msgnumLock, use atomic operations on maxMsgNum

Hi, Andres 21.03.2025 19:33, Andres Freund wrote: > Hi, > > On 2025-03-21 14:35:16 +0300, Yura Sokolov wrote: >> From 080c9e0de5e6e10751347e1ff50b65df424744cb Mon Sep 17 00:00:00 2001 >> From: Yura Sokolov >> Date: Mon, 3 Feb 2025 11:58:33 +0300 >> Subject: [

Re: Get rid of WALBufMappingLock

ppingLock is also worth to be removed. [1] https://postgr.es/m/flat/3b11fdc2-9793-403d-b3d4-67ff9a00d447%40postgrespro.ru [2] https://postgr.es/m/c31158a3-7c26-4b26-90df-2df8f7bbe736%40postgrespro.ru --- regards Yura Sokolov aka funny-falcon

Re: Network failure may prevent promotion

I've just rebased patches and merged last two fix-commits (0003 and 0004) into 0002. -- regards Yura Sokolov aka funny-falconFrom f8923c9b8e2f470dd3caaa1e71fb3b931389148b Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas Date: Fri, 21 Mar 2025 16:03:30 +0300 Subject: [PATCH v5 1/3] Re

Re: Optimize truncation logic to reduce AccessExclusive lock impact

19.03.2025 19:05, Yura Sokolov wrote: > 19.03.2025 05:09, David Rowley wrote: >> On Tue, 18 Mar 2025 at 19:04, Stepan Neretin wrote: >>> We propose modifying the truncation condition in should_attempt_truncation >>> to avoid unnecessary full buffer scans. >> I&#

Re: sinvaladt.c: remove msgnumLock, use atomic operations on maxMsgNum

Just rebased the patch. --- regards Yura Sokolov aka funny-falconFrom 080c9e0de5e6e10751347e1ff50b65df424744cb Mon Sep 17 00:00:00 2001 From: Yura Sokolov Date: Mon, 3 Feb 2025 11:58:33 +0300 Subject: [PATCH v2] sinvaladt.c: use atomic operations on maxMsgNum msgnumLock spinlock could be

Re: ZStandard (with dictionaries) compression support for TOAST compression

level is 3 and dict > size is 4KB. For super fast mode level can be set to 1. No. Super-fast mode levels are negative. See parsing "--fast" parameter in `programs/zstdcli.c` in zstd's repository and definition of ZSTD_minCLevel(). So, to support "super-fast" mode you have to accept negative compression levels. I didn't check, probably you're already support them? --- regards Yura Sokolov aka funny-falcon

Re: Implement waiting for wal lsn replay: reloaded

10.03.2025 14:30, Alexander Korotkov пишет: > On Fri, Feb 28, 2025 at 3:55 PM Yura Sokolov wrote: >> 28.02.2025 16:03, Yura Sokolov пишет: >>> 17.02.2025 00:27, Alexander Korotkov wrote: >>>> On Thu, Feb 6, 2025 at 10:31 AM Yura Sokolov >>>> wrote:

Re: ZStandard (with dictionaries) compression support for TOAST compression

DbNS%40paquier.xyz> Overall idea is great. I just want to mention LZ4 also have API to use dictionary. Its dictionary will be as simple as "virtually prepended" text (in contrast to complex ZStd dictionary format). I mean, it would be great if "dictionary" will be common property for different algorithms. On the other hand, zstd have "super fast" mode which is actually a bit faster than LZ4 and compresses a bit better. So may be support for different algos is not essential. (But then we need a way to change compression level to that "super fast" mode.) --- regards Yura Sokolov aka funny-falcon

Re: [RFC] Lock-free XLog Reservation from WAL

05.03.2025 08:39, Zhou, Zhiguo пишет: > > > On 2/23/2025 8:03 PM, Yura Sokolov wrote: >> 14.02.2025 11:41, Zhou, Zhiguo пишет: >>> >>> >>> On 2/11/2025 9:25 AM, Japin Li wrote: >>>> On Mon, 10 Feb 2025 at 22:12, "Zhou, Zhiguo" wro

Re: Allow table AMs to define their own reloptions

04.03.2025 09:16, Julien Tachoires пишет: > Hi Yura, > > On Sun, Mar 02, 2025 at 06:20:07PM +0300, Yura Sokolov wrote: >> Your forgot another one attempt discussion with patch [1] with alive >> commitfest entry [2] >> >> [1] https://postgr.es/m/flat/3766675.

Re: Allow table AMs to define their own reloptions

.@mail.gmail.com >> [3] >> https://www.postgresql.org/message-id/flat/AMUA1wBBBxfc3tKRLLdU64rb.1.1683276279979.Hmail.wuhao%40hashdata.cn > > Please find a new version including minor fixes: 'TAM' terms are > replaced by 'table AM' Good day, Julien. Your forgot another one attempt discussion with patch [1] with alive commitfest entry [2] [1] https://postgr.es/m/flat/3766675.7eaCOWfIcx%40thinkpad-pgpro [2] https://commitfest.postgresql.org/patch/4688/ --- regards Yura Sokolov aka funny-falcon

Re: Get rid of WALBufMappingLock

leep and immediately returns. So actual condition of `while` loop is checked at least twice before going to sleep. > > I see about 20 other occurrences of similar code, so, perhaps, everything is > fine. But I would greatly appreciate a little pointers on why it works. --- regards Yura Sokolov aka funny-falcon

Re: Implement waiting for wal lsn replay: reloaded

28.02.2025 16:03, Yura Sokolov пишет: > 17.02.2025 00:27, Alexander Korotkov wrote: >> On Thu, Feb 6, 2025 at 10:31 AM Yura Sokolov >> wrote: >>> I briefly looked into patch and have couple of minor remarks: >>> >>> 1. I don't like `palloc`

Re: Implement waiting for wal lsn replay: reloaded

17.02.2025 00:27, Alexander Korotkov wrote: > On Thu, Feb 6, 2025 at 10:31 AM Yura Sokolov wrote: >> I briefly looked into patch and have couple of minor remarks: >> >> 1. I don't like `palloc` in the `WaitLSNWakeup`. I believe it wont issue >> problems, but still

Re: Get rid of WALBufMappingLock

ffering the help. Updated version of patch is attached > (I've added one memory barrier there just in case). I would > appreciate if you could run it on batta, hachi or similar hardware. Good day, Alexander. Checked your additions to patch. They're clear and robust. --- regards Yura Sokolov aka funny-falcon

Re: [RFC] Lock-free XLog Reservation from WAL

14.02.2025 11:41, Zhou, Zhiguo пишет: > > > On 2/11/2025 9:25 AM, Japin Li wrote: >> On Mon, 10 Feb 2025 at 22:12, "Zhou, Zhiguo" wrote: >>> On 2/5/2025 4:32 PM, Japin Li wrote: >>>> On Mon, 27 Jan 2025 at 17:30, "Zhou, Zhiguo"

Re: Get rid of WALBufMappingLock

14.02.2025 17:09, Yura Sokolov пишет: > 14.02.2025 13:24, Alexander Korotkov пишет: >> On Fri, Feb 14, 2025 at 11:45 AM Pavel Borisov >> wrote: >>> On Fri, 14 Feb 2025 at 00:59, Alexander Korotkov >>> wrote: >>>> On Thu, Feb 13, 2025 at 6:39 PM Pav

Re: Get rid of WALBufMappingLock

025 at 14:08, Alexander Korotkov >>>> wrote: >>>>> >>>>> On Thu, Feb 13, 2025 at 11:45 AM Yura Sokolov >>>>> wrote: >>>>>> 13.02.2025 12:34, Alexander Korotkov пишет: >>>>>>> On Wed, Feb 12, 2025 at

Re: Get rid of WALBufMappingLock

13.02.2025 12:34, Alexander Korotkov пишет: > On Wed, Feb 12, 2025 at 8:16 PM Yura Sokolov wrote: >> 08.02.2025 13:07, Alexander Korotkov пишет: >>> On Fri, Feb 7, 2025 at 1:39 PM Alexander Korotkov >>> wrote: >>>> Good, thank you. I think 0001

Re: Get rid of WALBufMappingLock

ss, it is hardly sensible to do more probes. 3 probes did better than 2 in other benchmark [1], although there were NUM_XLOGINSERT_LOCK increased. Excuse me for not bencmarking different choices here. I'll try to do measurements in next days. [1] https://postgr.es/m/3b11fdc2-9793-403d

Re: Get rid of WALBufMappingLock

07.02.2025 14:02, Yura Sokolov пишет: > 07.02.2025 01:26, Alexander Korotkov пишет: >> Hi! >> >> On Sun, Jan 19, 2025 at 2:11 AM Yura Sokolov >> wrote: >>> >>> During discussion of Increasing NUM_XLOGINSERT_LOCKS [1], Andres Freund >>> use

Re: Get rid of WALBufMappingLock

07.02.2025 01:26, Alexander Korotkov пишет: > Hi! > > On Sun, Jan 19, 2025 at 2:11 AM Yura Sokolov wrote: >> >> During discussion of Increasing NUM_XLOGINSERT_LOCKS [1], Andres Freund >> used benchmark which creates WAL records very intensively. While I this >> i

Re: Implement waiting for wal lsn replay: reloaded

atch's sense. So lets stick with `inHeap`, but move it a bit. Non-code question: do you imagine for `WAIT` command reuse for other cases? Is syntax rule in gram.y convenient enough for such reuse? I believe, `LSN` is not part of syntax to not introduce new keyword. But is it correct way? I have no answer or strong opinion. Otherwise, the patch looks quite strong to me. --- regards Yura Sokolov

Re: sinvaladt.c: remove msgnumLock, use atomic operations on maxMsgNum

03.02.2025 19:49, Heikki Linnakangas пишет: > On 03/02/2025 13:05, Yura Sokolov wrote: >> Investigating some performance issues of a client, our engineers found >> msgnumLock to be contended. >> >> Looking closer it is obvious it is not needed at all: it used only as &g

sinvaladt.c: remove msgnumLock, use atomic operations on maxMsgNum

pg_atomic_read_membarrier_u32() is implemented currently as a write operation, that's not good for contended place. - regards Yura Sokolov aka funny-falconFrom 351b3ecb5d085559beb13ffd6dd4416dab4b4a84 Mon Sep 17 00:00:00 2001 From: Yura Sokolov Date: Mon, 3 Feb 2025 11:58:33 +0300 Subject: [PATCH v0] sinvaladt.c

Re: Optimize scram_SaltedPassword performance

03.02.2025 14:17, Daniel Gustafsson пишет: >> On 3 Feb 2025, at 08:45, Yura Sokolov wrote: >> >> 03.02.2025 10:11, Zixuan Fu пишет: >>> Hi Hackers, >>> >>> While profiling a program with `perf`, I noticed that `scram_SaltedPassword` >>

Re: Optimize scram_SaltedPassword performance

03.02.2025 10:11, Zixuan Fu пишет: > Hi Hackers, > > While profiling a program with `perf`, I noticed that `scram_SaltedPassword` > consumed more CPU time than expected. After some investigation, I found > that the function performs many HMAC iterations (4096 rounds for > SCRAM-SHA-256), and eac

Re: hash_search_with_hash_value is high in "perf top" on a replica

31.01.2025 17:25, Álvaro Herrera пишет: > On 2025-Jan-31, Dmitry Koterov wrote: > >> PG "startup recovering" eats up a lot of CPU (like 65 %user and 30 %sys), >> which is a little surprising (what is it doing with all those CPU cycles? >> it looked like WAL replay should be more IO bound than CPU

Re: EvictUnpinnedBuffer and buffer free list

31.01.2025 08:15, Ashutosh Bapat пишет: Hi All, EvictUnpinnedBuffer() calls InvalidateVictimBuffer() followed by UnpinBuffer() before returning. None of those functions put the buffer back into the free list. Its freeNext remains set to FREENEXT_NOT_IN_LIST. I think then that buffer will never be

Re: New process of getting changes into the commitfest app

23.01.2025 15:57, Jelte Fennema-Nio пишет: (Resent because sending to both -hackers and -www gets emails put in the moderation queue, and I don't want to introduce that delay to all replies. If you received the previous version because you're in the CC please only reply to this one) # Background

Re: [RFC] Lock-free XLog Reservation from WAL

24.01.2025 12:07, Japin Li wrote: On Thu, 23 Jan 2025 at 21:44, Japin Li wrote: On Thu, 23 Jan 2025 at 15:03, Yura Sokolov wrote: 23.01.2025 11:46, Japin Li пишет: On Wed, 22 Jan 2025 at 22:44, Japin Li wrote: On Wed, 22 Jan 2025 at 17:02, Yura Sokolov wrote: I believe, I know why it

Re: [RFC] Lock-free XLog Reservation from WAL

24.01.2025 12:07, Japin Li пишет: On Thu, 23 Jan 2025 at 21:44, Japin Li wrote: On Thu, 23 Jan 2025 at 15:03, Yura Sokolov wrote: 23.01.2025 11:46, Japin Li пишет: On Wed, 22 Jan 2025 at 22:44, Japin Li wrote: On Wed, 22 Jan 2025 at 17:02, Yura Sokolov wrote: I believe, I know why it

Re: Increase NUM_XLOGINSERT_LOCKS

may) arise from increasing it, 64 seems to be sweet spot. Probably in a future it could be increased more after other places will be optimized. On Thu, Jan 23, 2025 at 10:30 AM Japin Li <mailto:japi...@hotmail.com>> wrote: On Sat, 18 Jan 2025 at 14:53, Yura Sokolov mailto:y.sok

Re: [RFC] Lock-free XLog Reservation from WAL

23.01.2025 11:46, Japin Li пишет: On Wed, 22 Jan 2025 at 22:44, Japin Li wrote: On Wed, 22 Jan 2025 at 17:02, Yura Sokolov wrote: I believe, I know why it happens: I was in hurry making v2 by cherry-picking internal version. I reverted some changes in CalcCuckooPositions manually and forgot

Re: [RFC] Lock-free XLog Reservation from WAL

22.01.2025 10:54, Japin Li wrote: On Wed, 22 Jan 2025 at 10:25, Yura Sokolov wrote: 22.01.2025 09:09, Japin Li пишет: Hi, Yura Sokolov Thanks for updating the patch. I test the v2 patch using BenchmarkSQL 1000 warehouse, and here is the tpmC result: case | min| avg

Re: [RFC] Lock-free XLog Reservation from WAL

22.01.2025 15:37, Japin Li пишет: On Wed, 22 Jan 2025 at 16:49, Japin Li wrote: On Wed, 22 Jan 2025 at 11:22, Yura Sokolov wrote: 22.01.2025 10:54, Japin Li пишет: On Wed, 22 Jan 2025 at 10:25, Yura Sokolov wrote: 22.01.2025 09:09, Japin Li пишет: Hi, Yura Sokolov Thanks for updating the

Re: [RFC] Lock-free XLog Reservation from WAL

22.01.2025 10:54, Japin Li пишет: On Wed, 22 Jan 2025 at 10:25, Yura Sokolov wrote: 22.01.2025 09:09, Japin Li пишет: Hi, Yura Sokolov Thanks for updating the patch. I test the v2 patch using BenchmarkSQL 1000 warehouse, and here is the tpmC result: case | min| avg

Re: [RFC] Lock-free XLog Reservation from WAL

22.01.2025 09:09, Japin Li пишет: On Sun, 19 Jan 2025 at 17:56, Yura Sokolov wrote: 17.01.2025 17:00, Zhou, Zhiguo пишет: On 1/16/2025 10:00 PM, Yura Sokolov wrote: Good day, Zhiguo. Excuse me, I feel sneaky a bit, but I've started another thread just about increase of NUM_XLOGINSERT

Re: int64 support in List API

20.01.2025 07:36, Tom Lane пишет: Gurjeet Singh writes: I wanted to use the list api from pg_list.h. It has special implementations for int, oid, pointer, and xid types, which help with lower code overhead (no need to create structures whose sole member is of one of these types) and better per

Re: [RFC] Lock-free XLog Reservation from WAL

17.01.2025 17:00, Zhou, Zhiguo пишет: On 1/16/2025 10:00 PM, Yura Sokolov wrote: Good day, Zhiguo. Excuse me, I feel sneaky a bit, but I've started another thread just about increase of NUM_XLOGINSERT_LOCK, because I can measure its effect even on my working notebook (it is anothe

Re: Get rid of WALBufMappingLock

19.01.2025 03:11, Yura Sokolov пишет: Good day, hackers. During discussion of Increasing NUM_XLOGINSERT_LOCKS [1], Andres Freund used benchmark which creates WAL records very intensively. While I this it is not completely fair (1MB log records are really rare), it pushed me to analyze write

Re: [RFC] Lock-free XLog Reservation from WAL

17.01.2025 17:00, Zhou, Zhiguo пишет: On 1/16/2025 10:00 PM, Yura Sokolov wrote: Good day, Zhiguo. Excuse me, I feel sneaky a bit, but I've started another thread just about increase of NUM_XLOGINSERT_LOCK, because I can measure its effect even on my working notebook (it is anothe

Get rid of WALBufMappingLock

27 /2133 So, patched version with increased NUM_XLOGINSERT_LOCKS looks no worse than unpatched without increasing num of locks. --- regards Yura Sokolov aka funny-falconFrom 236b69ae1f524c7e8488da7244966e631324a0e3 Mon Sep 17 00:00:00 2001 From: Yura Sokolov Date: Sat, 18 Jan 2025

Re: Increase NUM_XLOGINSERT_LOCKS

Since it seems Andres missed my request to send answer's copy, here it is: On 2025-01-16 18:55:47 +0300, Yura Sokolov wrote: > 16.01.2025 18:36, Andres Freund пишет: >> Hi, >> >> On 2025-01-16 16:52:46 +0300, Yura Sokolov wrote: >>> Good day, hackers. >&g

Re: Increase NUM_XLOGINSERT_LOCKS

5 18:36, Andres Freund wrote: > Hi, > > On 2025-01-16 16:52:46 +0300, Yura Sokolov wrote: >> Good day, hackers. >> >> Zhiguo Zhow proposed to transform xlog reservation to lock-free algorighm to >> increment NUM_XLOGINSERT_LOCKS on very huge (480vCPU) servers. [1]

Re: [RFC] Lock-free XLog Reservation from WAL

14.01.2025 17:49, Zhou, Zhiguo пишет: Good day, Yura! On 1/10/2025 8:42 PM, Yura Sokolov wrote: If you consider hash-table fillrate, than 256 is quite enough for 128 concurrent inserters. The profile of your patch didn't show significant hotspots in the hash table functions, so I be

Increase NUM_XLOGINSERT_LOCKS

lock-free reservation should be looked at closer. [1] https://www.postgresql.org/message-id/flat/PH7PR11MB5796659F654F9BE983F3AD97EF142%40PH7PR11MB5796.namprd11.prod.outlook.com - regards Yura Sokolov aka funny-falconFrom 93a4d4a7e2219a952c2a544047c19db9f0f0f5c0 Mon Sep 17 00:00:00 20

Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

07.10.2024 17:53, Aya Iwata (Fujitsu) wrote: Hi All, Suggestions == When analyzing real-time data collected by PostgreSQL, it can be difficult to tune the current PostgreSQL server for satisfactory performance. Therefore, we propose Vertical Clustered Indexing (VCI), an in-memory

Re: POC, WIP: OR-clause support for indexes

Вс, 12 янв. 2025 г. в 21:39, Alexander Korotkov :On Fri, Nov 29, 2024 at 9:54 AM Alexander Korotkov wrote: > On Fri, Nov 29, 2024 at 7:51 AM Alena Rybakina > wrote: > > > > On 29.11.2024 03:04, Alexander Korotkov wrote: > > > On Thu, Nov 28, 2024 at 9:33 PM Alena Rybakina > > > wrote: > > >> On

Re: [RFC] Lock-free XLog Reservation from WAL

10.01.2025 19:53, Matthias van de Meent пишет: On Fri, 10 Jan 2025 at 13:42, Yura Sokolov wrote: BTW, your version could make alike trick for guaranteed atomicity: - change XLogRecord's `XLogRecPtr xl_prev` to `uint32 xl_prev_offset` and store offset to prev record's start. -

Re: [RFC] Lock-free XLog Reservation from WAL

, and leave the increase of NUM_XLOGINSERT_LOCKS for a future patch, where we would provide more quantitative evidence for the various implementations. WDYT? On Fri, 3 Jan 2025 at 20:36, Yura Sokolov <mailto:y.soko...@postgrespro.ru><mailto:y.soko...@postgrespro.ru <m

Re: Meson bug in detection of 64 atomics

drafted but never synced with “configure”. Patch that syncs meson check to configure’s one attached. It fixes the issue at least on my computer (Mac M1). —— regards Yura Sokolov aka funny-falcon

Re: [RFC] Lock-free XLog Reservation from WAL

, but looking closer to your patch today I see it is superior to mine (if atomic access will be fixed). regards, Yura Sokolov aka funny-falcon

Bug in v13 due to "Fix corruption when relation truncation fails."

//git.postgresql.org/gitweb/?p=postgresql.git;h=2280912165d [1] https://git.postgresql.org/gitweb/?p=postgresql.git;h=4c8e00ae9ae -- regards, Yura Sokolov aka funny-falcon

Re: Fix bank selection logic in SLRU

19.12.2024 15:10, Andrey M. Borodin wrote: On 19 Dec 2024, at 15:37, Yura Sokolov wrote: So, there's no more than 8192 banks at the moment. OK, but still current type indicates bitwise usage, while struct member is used as a number. Ok, I agree. Here's version with type cha

Re: Fix bank selection logic in SLRU

19.12.2024 13:10, Andrey Borodin пишет: On 19 Dec 2024, at 15:01, Yura Sokolov wrote: - `&` takes 0.69ns - `mult-rec` takes 2.94ns - `%` takes 3.24ns. Thanks, Yura, for benchmarks and off-list conversation. I’ve reproduced similar numbers on my Apple M2. I agree that additional 3-4ns

Re: Fix bank selection logic in SLRU

10.12.2024 17:07, Dilip Kumar wrote: On Tue, 10 Dec 2024 at 6:32 PM, Andrey M. Borodin <mailto:x4...@yandex-team.ru>> wrote: > On 10 Dec 2024, at 15:39, Yura Sokolov mailto:y.soko...@postgrespro.ru>> wrote: > > It is not critical bug, since it doesn

Fix bank selection logic in SLRU

does it pay for or not. [1] https://www.postgresql.org/message-id/flat/CAFiTN-vzDvNz%3DExGXz6gdyjtzGixKSqs0mKHMmaQ8sOSEFZ33A%40mail.gmail.com Regards, Yura Sokolov aka funny-falconFrom 4b438e67a79d77bf2caf3e5a0386bb7700d329ba Mon Sep 17 00:00:00 2001 From: Yura Sokolov Date: Tue, 10 Dec 2024 15:

Re: Maybe don't process multi xmax in FreezeMultiXactId() if it is already marked as invalid?

en we safely can? If we couldn't rely on hint, then hint is completely meaningless. have a nice day Yura Sokolov

Re: Suggest two small improvements for PITR.

11.01.2024 19:58, Yura Sokolov пишет: Good day, hackers. Here I am to suggest two small improvements to Point In Time Recovery. First is ability to recover recovery-target-time with timestamp stored in XLOG_RESTORE_POINT. Looks like historically this ability did exist and were removed

Suggest two small improvements for PITR.

or both is permitted. If idea is accepted, patches for tests will be applied as well. [1] https://git.postgresql.org/gitweb/?p=postgresql.git;a=patch;h=c945af80 --- Yura Sokolov.From 173cfc3762a97c300b618f863fd23433909cdb81 Mon Sep 17 00:00:00 2001 From: Yura Sokolov Date: Wed, 3 May 2023 18

Re: Vectorization of some functions and improving pg_list interface

06.09.2023 13:24, Yura Sokolov wrote: 24.08.2023 17:07, Maxim Orlov wrote: Hi! Recently, I've been playing around with pg_lists and realize how annoying (maybe, I was a bit tired) some stuff related to the lists. For an example, see this code List *l1 = list_make4(1, 2,

Re: Vectorization of some functions and improving pg_list interface

24.08.2023 17:07, Maxim Orlov wrote: Hi! Recently, I've been playing around with pg_lists and realize how annoying (maybe, I was a bit tired) some stuff related to the lists. For an example, see this code List *l1 = list_make4(1, 2, 3, 4), *l2 = list_make4(5, 6, 7, 8),

Re: When IMMUTABLE is not.

15.06.2023 17:49, Tom Lane пишет: "David G. Johnston" writes: The failure to find and execute the function code itself is not a failure mode that these markers need be concerned with. Assuming one can execute the function an immutable function will give the same answer for the same input for

Re: When IMMUTABLE is not.

15.06.2023 16:58, c...@anastigmatix.net пишет: On 2023-06-15 09:21, Tom Lane wrote: Yura Sokolov writes: not enough to be sure function doesn't manipulate data. Of course not.  It is the user's responsibility to mark functions properly. And also, isn't it the case that I

Re: When IMMUTABLE is not.

15.06.2023 16:21, Tom Lane wrote: Yura Sokolov writes: I found, than declaration of function as IMMUTABLE/STABLE is not enough to be sure function doesn't manipulate data. Of course not. It is the user's responsibility to mark functions properly. Trying to enforce that compl

Re: When IMMUTABLE is not.

is check is necessary, but it should be FATAL instead of ERROR. And ERROR should be generated at same place, when it is generated for `immutable_direct`, but with check of "read_only" status through whole call stack instead of just direct function kind. - regards, Yura Sokolov Postgres Professional

When IMMUTABLE is not.

And ERROR should be generated at same place, when it is generated for `immutable_direct`, but with check of "read_only" status through whole call stack instead of just direct function kind. - regards, Yura Sokolov Postgres Professional immutable_not.sql Description: applicatio

Re: Let's make PostgreSQL multi-threaded

macros */ #define my_hello_var (*(my_hello_var_t*)(CurSessionVar(my_hello_var_offset))) /* module.c */ size_t my_hello_var_offset = 0; void PG_init() { RegisterSessionVar(sizeof(my_hello_var_t), &my_hello_var_offset); } For security reasons, offset could be mangled. -- regards, Yura Sokolov

  1   2   3   >