Re: RFC: Lock-free XLog Reservation from WAL

2025-01-05 Thread Zhou, Zhiguo
Hi Japin, Thanks so much for your test and review. As you may have noticed, this patch has implemented the initial optimization idea and has passed only the basic regression tests. We have planned to extend the validation to include TAP tests after aligning the expectations with the community,

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-06 Thread Zhou, Zhiguo
qiu wrote: HI Zhiguo     Thank you for your reply ,Then you'll have to prove that 128 is the optimal value, otherwise they'll have a hard time agreeing with you on this patch. Thanks On Mon, Jan 6, 2025 at 2:46 PM Zhou, Zhiguo <mailto:zhiguo.z...@intel.com>> wrote:

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-05 Thread Zhou, Zhiguo
Hi Yura and Wenhui, Thanks for kindly reviewing this work! On 1/3/2025 9:01 PM, wenhui qiu wrote: Hi     Thank you for your path,NUM_XLOGINSERT_LOCKS increase to 128,I think it will be challenged,do we make it guc ? I noticed there have been some discussions (for example, [1] and its res

Re: [RFC] Lock-free XLog Reservation from WAL

2025-02-10 Thread Zhou, Zhiguo
On 2/5/2025 4:32 PM, Japin Li wrote: On Mon, 27 Jan 2025 at 17:30, "Zhou, Zhiguo" wrote: On 1/26/2025 10:59 PM, Yura Sokolov wrote: 24.01.2025 12:07, Japin Li пишет: On Thu, 23 Jan 2025 at 21:44, Japin Li wrote: On Thu, 23 Jan 2025 at 15:03, Yura Sokolov wrote: 23.01.2025 11

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-20 Thread Zhou, Zhiguo
On 1/19/2025 10:56 PM, Yura Sokolov wrote: 17.01.2025 17:00, Zhou, Zhiguo пишет: On 1/16/2025 10:00 PM, Yura Sokolov wrote: Good day, Zhiguo. Excuse me, I feel sneaky a bit, but I've started another thread just about increase of NUM_XLOGINSERT_LOCK, because I can measure its e

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-27 Thread Zhou, Zhiguo
On 1/26/2025 10:59 PM, Yura Sokolov wrote: 24.01.2025 12:07, Japin Li пишет: On Thu, 23 Jan 2025 at 21:44, Japin Li wrote: On Thu, 23 Jan 2025 at 15:03, Yura Sokolov wrote: 23.01.2025 11:46, Japin Li пишет: On Wed, 22 Jan 2025 at 22:44, Japin Li wrote: On Wed, 22 Jan 2025 at 17:02, Yura

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-14 Thread Zhou, Zhiguo
Good day, Yura! On 1/10/2025 8:42 PM, Yura Sokolov wrote: If you consider hash-table fillrate, than 256 is quite enough for 128 concurrent inserters. The profile of your patch didn't show significant hotspots in the hash table functions, so I believe the 256 entries should be enough. I wi

RFC: Lock-free XLog Reservation from WAL

2025-01-02 Thread Zhou, Zhiguo
Hi all, I am reaching out to solicit your insights and comments on a recent proposal regarding the "Lock-free XLog Reservation from WAL." We have identified some challenges with the current WAL insertions, which require space reservations in the WAL buffer which involve updating two shared-memo

[RFC] Lock-free XLog Reservation from WAL

2025-01-02 Thread Zhou, Zhiguo
Hi all, I am reaching out to solicit your insights and comments on a recent proposal regarding the "Lock-free XLog Reservation from WAL." We have identified some challenges with the current WAL insertions, which require space reservations in the WAL buffer which involve updating two shared-memo

[RFC] Lock-free XLog Reservation from WAL

2025-01-02 Thread Zhou, Zhiguo
Hi all, I am reaching out to solicit your insights and comments on a recent proposal regarding the "Lock-free XLog Reservation from WAL." We have identified some challenges with the current WAL insertions, which require space reservations in the WAL buffer which involve updating two shared-memo

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-17 Thread Zhou, Zhiguo
On 1/16/2025 10:00 PM, Yura Sokolov wrote: Good day, Zhiguo. Excuse me, I feel sneaky a bit, but I've started another thread just about increase of NUM_XLOGINSERT_LOCK, because I can measure its effect even on my working notebook (it is another one: Ryzen 5825U limited to @2GHz). http:/

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-09 Thread Zhou, Zhiguo
On 1/7/2025 10:49 AM, Юрий Соколов wrote: On 6 Jan 2025, at 09:46, Zhou, Zhiguo wrote: Hi Yura and Wenhui, Thanks for kindly reviewing this work! On 1/3/2025 9:01 PM, wenhui qiu wrote: Hi Thank you for your path,NUM_XLOGINSERT_LOCKS increase to 128,I think it will be challenged,do we

RE: [RFC] Lock-free XLog Reservation from WAL

2025-01-02 Thread Zhou, Zhiguo
This message is a duplicate of ph7pr11mb5796659f654f9be983f3ad97ef...@ph7pr11mb5796.namprd11.prod.outlook.com. Please consider dropping this thread and review the original one instead. Sorry for your inconvenience. -Original Message- From: Zhou, Zhiguo Sent: Thursday, January 2, 2025

RE: RFC: Lock-free XLog Reservation from WAL

2025-01-02 Thread Zhou, Zhiguo
This message is a duplicate of ph7pr11mb5796659f654f9be983f3ad97ef...@ph7pr11mb5796.namprd11.prod.outlook.com. Please consider dropping this thread and review the original one instead. Sorry for your inconvenience. -Original Message- From: Zhou, Zhiguo Sent: Thursday, January 2, 2025

Re: [RFC] Lock-free XLog Reservation from WAL

2025-02-14 Thread Zhou, Zhiguo
On 2/11/2025 9:25 AM, Japin Li wrote: On Mon, 10 Feb 2025 at 22:12, "Zhou, Zhiguo" wrote: On 2/5/2025 4:32 PM, Japin Li wrote: On Mon, 27 Jan 2025 at 17:30, "Zhou, Zhiguo" wrote: On 1/26/2025 10:59 PM, Yura Sokolov wrote: 24.01.2025 12:07, Japin Li пишет: On Thu,

Re: [RFC] Lock-free XLog Reservation from WAL

2025-03-04 Thread Zhou, Zhiguo
On 2/23/2025 8:03 PM, Yura Sokolov wrote: 14.02.2025 11:41, Zhou, Zhiguo пишет: On 2/11/2025 9:25 AM, Japin Li wrote: On Mon, 10 Feb 2025 at 22:12, "Zhou, Zhiguo" wrote: On 2/5/2025 4:32 PM, Japin Li wrote: On Mon, 27 Jan 2025 at 17:30, "Zhou, Zhiguo" wrote: O

Optimize LWLock scalability via ReadBiasedLWLock for heavily-shared locks

2025-06-25 Thread Zhou, Zhiguo
Hi Hackers, This patch addresses severe LWLock contention observed on high-core systems where hundreds of processors concurrently access frequently-shared locks. Specifically for ProcArrayLock (exhibiting 93.5% shared-mode acquires), we implement a new ReadBiasedLWLock mechanism to eliminate the

Optimize shared LWLock acquisition for high-core-count systems

2025-05-30 Thread Zhou, Zhiguo
Hi Hackers, I am reaching out to solicit your insights and comments on this patch addressing a significant performance bottleneck in LWLock acquisition observed on high-core-count systems. During performance analysis of HammerDB/TPCC (192 virtual users, 757 warehouses) on a 384-vCPU Intel sys

Re: Optimize shared LWLock acquisition for high-core-count systems

2025-05-30 Thread Zhou, Zhiguo
Thanks David! I've added the patch referred by this link: https://commitfest.postgresql.org/patch/5784/ Regards, Zhiguo On 5/30/2025 8:02 PM, David Rowley wrote: On Fri, 30 May 2025 at 23:31, Zhou, Zhiguo wrote: Please review this patch for consideration in upcoming PostgreSQL rel

Re: Optimize shared LWLock acquisition for high-core-count systems

2025-07-10 Thread Zhou, Zhiguo
On 7/9/2025 3:56 PM, Yura Sokolov wrote: 30.05.2025 14:30, Zhou, Zhiguo пишет: Hi Hackers, I am reaching out to solicit your insights and comments on this patch addressing a significant performance bottleneck in LWLock acquisition observed on high-core-count systems. During performance

Re: Optimize LWLock scalability via ReadBiasedLWLock for heavily-shared locks

2025-07-03 Thread Zhou, Zhiguo
Hi Andres, On 7/1/2025 10:06 PM, Andres Freund wrote: Hi, On 2025-07-01 09:57:18 -0400, Andres Freund wrote: On 2025-06-26 13:07:49 +0800, Zhou, Zhiguo wrote: This patch addresses severe LWLock contention observed on high-core systems where hundreds of processors concurrently access

Re: Optimize LWLock scalability via ReadBiasedLWLock for heavily-shared locks

2025-07-01 Thread Zhou, Zhiguo
On 6/26/2025 1:07 PM, Zhou, Zhiguo wrote: Hi Hackers, This patch addresses severe LWLock contention observed on high-core systems where hundreds of processors concurrently access frequently-shared locks. Specifically for ProcArrayLock (exhibiting 93.5% shared-mode acquires), we implement a

Re: Optimize shared LWLock acquisition for high-core-count systems

2025-07-21 Thread Zhou, Zhiguo
On 7/11/2025 4:35 PM, Yura Sokolov wrote: 10.07.2025 18:57, Zhou, Zhiguo пишет: On 7/9/2025 3:56 PM, Yura Sokolov wrote: 30.05.2025 14:30, Zhou, Zhiguo пишет: Hi Hackers, I am reaching out to solicit your insights and comments on this patch addressing a significant performance bottleneck