Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-27 Thread Zhou, Zhiguo
On 1/26/2025 10:59 PM, Yura Sokolov wrote: 24.01.2025 12:07, Japin Li пишет: On Thu, 23 Jan 2025 at 21:44, Japin Li wrote: On Thu, 23 Jan 2025 at 15:03, Yura Sokolov wrote: 23.01.2025 11:46, Japin Li пишет: On Wed, 22 Jan 2025 at 22:44, Japin Li wrote: On Wed, 22 Jan 2025 at 17:02, Yura

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-26 Thread Yura Sokolov
24.01.2025 12:07, Japin Li wrote: On Thu, 23 Jan 2025 at 21:44, Japin Li wrote: On Thu, 23 Jan 2025 at 15:03, Yura Sokolov wrote: 23.01.2025 11:46, Japin Li пишет: On Wed, 22 Jan 2025 at 22:44, Japin Li wrote: On Wed, 22 Jan 2025 at 17:02, Yura Sokolov wrote: I believe, I know why it hap

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-26 Thread Yura Sokolov
24.01.2025 12:07, Japin Li пишет: On Thu, 23 Jan 2025 at 21:44, Japin Li wrote: On Thu, 23 Jan 2025 at 15:03, Yura Sokolov wrote: 23.01.2025 11:46, Japin Li пишет: On Wed, 22 Jan 2025 at 22:44, Japin Li wrote: On Wed, 22 Jan 2025 at 17:02, Yura Sokolov wrote: I believe, I know why it hap

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-24 Thread Japin Li
On Thu, 23 Jan 2025 at 21:44, Japin Li wrote: > On Thu, 23 Jan 2025 at 15:03, Yura Sokolov wrote: >> 23.01.2025 11:46, Japin Li пишет: >>> On Wed, 22 Jan 2025 at 22:44, Japin Li wrote: On Wed, 22 Jan 2025 at 17:02, Yura Sokolov wrote: > I believe, I know why it happens: I was in

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-23 Thread Japin Li
On Thu, 23 Jan 2025 at 15:03, Yura Sokolov wrote: > 23.01.2025 11:46, Japin Li пишет: >> On Wed, 22 Jan 2025 at 22:44, Japin Li wrote: >>> On Wed, 22 Jan 2025 at 17:02, Yura Sokolov wrote: I believe, I know why it happens: I was in hurry making v2 by cherry-picking internal version. I

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-23 Thread Yura Sokolov
23.01.2025 11:46, Japin Li пишет: On Wed, 22 Jan 2025 at 22:44, Japin Li wrote: On Wed, 22 Jan 2025 at 17:02, Yura Sokolov wrote: I believe, I know why it happens: I was in hurry making v2 by cherry-picking internal version. I reverted some changes in CalcCuckooPositions manually and forgot t

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-23 Thread Japin Li
On Wed, 22 Jan 2025 at 22:44, Japin Li wrote: > On Wed, 22 Jan 2025 at 17:02, Yura Sokolov wrote: >> I believe, I know why it happens: I was in hurry making v2 by >> cherry-picking internal version. I reverted some changes in >> CalcCuckooPositions manually and forgot to add modulo >> PREV_LINKS_

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-22 Thread Japin Li
On Wed, 22 Jan 2025 at 17:02, Yura Sokolov wrote: > I believe, I know why it happens: I was in hurry making v2 by > cherry-picking internal version. I reverted some changes in > CalcCuckooPositions manually and forgot to add modulo > PREV_LINKS_HASH_CAPA. > > Here's the fix: > > pos->pos[0

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-22 Thread Yura Sokolov
22.01.2025 10:54, Japin Li wrote: On Wed, 22 Jan 2025 at 10:25, Yura Sokolov wrote: 22.01.2025 09:09, Japin Li пишет: Hi, Yura Sokolov Thanks for updating the patch. I test the v2 patch using BenchmarkSQL 1000 warehouse, and here is the tpmC result: case | min| avg

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-22 Thread Yura Sokolov
22.01.2025 15:37, Japin Li пишет: On Wed, 22 Jan 2025 at 16:49, Japin Li wrote: On Wed, 22 Jan 2025 at 11:22, Yura Sokolov wrote: 22.01.2025 10:54, Japin Li пишет: On Wed, 22 Jan 2025 at 10:25, Yura Sokolov wrote: 22.01.2025 09:09, Japin Li пишет: Hi, Yura Sokolov Thanks for updating the

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-22 Thread Japin Li
On Wed, 22 Jan 2025 at 16:49, Japin Li wrote: > On Wed, 22 Jan 2025 at 11:22, Yura Sokolov wrote: >> 22.01.2025 10:54, Japin Li пишет: >>> On Wed, 22 Jan 2025 at 10:25, Yura Sokolov wrote: 22.01.2025 09:09, Japin Li пишет: > Hi, Yura Sokolov > Thanks for updating the patch. > I

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-22 Thread Japin Li
On Wed, 22 Jan 2025 at 11:22, Yura Sokolov wrote: > 22.01.2025 10:54, Japin Li пишет: >> On Wed, 22 Jan 2025 at 10:25, Yura Sokolov wrote: >>> 22.01.2025 09:09, Japin Li пишет: Hi, Yura Sokolov Thanks for updating the patch. I test the v2 patch using BenchmarkSQL 1000 warehouse, an

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-22 Thread Yura Sokolov
22.01.2025 10:54, Japin Li пишет: On Wed, 22 Jan 2025 at 10:25, Yura Sokolov wrote: 22.01.2025 09:09, Japin Li пишет: Hi, Yura Sokolov Thanks for updating the patch. I test the v2 patch using BenchmarkSQL 1000 warehouse, and here is the tpmC result: case | min| avg

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-21 Thread Japin Li
On Wed, 22 Jan 2025 at 10:25, Yura Sokolov wrote: > 22.01.2025 09:09, Japin Li пишет: >> Hi, Yura Sokolov >> Thanks for updating the patch. >> I test the v2 patch using BenchmarkSQL 1000 warehouse, and here is the tpmC >> result: >> case | min| avg| max >> -

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-21 Thread Yura Sokolov
22.01.2025 09:09, Japin Li пишет: On Sun, 19 Jan 2025 at 17:56, Yura Sokolov wrote: 17.01.2025 17:00, Zhou, Zhiguo пишет: On 1/16/2025 10:00 PM, Yura Sokolov wrote: Good day, Zhiguo. Excuse me, I feel sneaky a bit, but I've started another thread just about increase of NUM_XLOGINSERT_LOCK,

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-21 Thread Japin Li
On Sun, 19 Jan 2025 at 17:56, Yura Sokolov wrote: > 17.01.2025 17:00, Zhou, Zhiguo пишет: >> On 1/16/2025 10:00 PM, Yura Sokolov wrote: >>> >>> Good day, Zhiguo. >>> >>> Excuse me, I feel sneaky a bit, but I've started another thread >>> just about increase of NUM_XLOGINSERT_LOCK, because I can me

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-20 Thread Zhou, Zhiguo
On 1/19/2025 10:56 PM, Yura Sokolov wrote: 17.01.2025 17:00, Zhou, Zhiguo пишет: On 1/16/2025 10:00 PM, Yura Sokolov wrote: Good day, Zhiguo. Excuse me, I feel sneaky a bit, but I've started another thread just about increase of NUM_XLOGINSERT_LOCK, because I can measure its effect eve

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-19 Thread Yura Sokolov
17.01.2025 17:00, Zhou, Zhiguo пишет: On 1/16/2025 10:00 PM, Yura Sokolov wrote: Good day, Zhiguo. Excuse me, I feel sneaky a bit, but I've started another thread just about increase of NUM_XLOGINSERT_LOCK, because I can measure its effect even on my working notebook (it is another one: Ry

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-18 Thread Yura Sokolov
17.01.2025 17:00, Zhou, Zhiguo пишет: On 1/16/2025 10:00 PM, Yura Sokolov wrote: Good day, Zhiguo. Excuse me, I feel sneaky a bit, but I've started another thread just about increase of NUM_XLOGINSERT_LOCK, because I can measure its effect even on my working notebook (it is another one: Ry

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-17 Thread Zhou, Zhiguo
On 1/16/2025 10:00 PM, Yura Sokolov wrote: Good day, Zhiguo. Excuse me, I feel sneaky a bit, but I've started another thread just about increase of NUM_XLOGINSERT_LOCK, because I can measure its effect even on my working notebook (it is another one: Ryzen 5825U limited to @2GHz). http:/

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-16 Thread Yura Sokolov
14.01.2025 17:49, Zhou, Zhiguo пишет: Good day, Yura! On 1/10/2025 8:42 PM, Yura Sokolov wrote: If you consider hash-table fillrate, than 256 is quite enough for 128 concurrent inserters. The profile of your patch didn't show significant hotspots in the hash table functions, so I believe the

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-14 Thread Zhou, Zhiguo
Good day, Yura! On 1/10/2025 8:42 PM, Yura Sokolov wrote: If you consider hash-table fillrate, than 256 is quite enough for 128 concurrent inserters. The profile of your patch didn't show significant hotspots in the hash table functions, so I believe the 256 entries should be enough. I wi

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-10 Thread Yura Sokolov
10.01.2025 19:53, Matthias van de Meent пишет: On Fri, 10 Jan 2025 at 13:42, Yura Sokolov wrote: BTW, your version could make alike trick for guaranteed atomicity: - change XLogRecord's `XLogRecPtr xl_prev` to `uint32 xl_prev_offset` and store offset to prev record's start. -1, I don't think

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-10 Thread Matthias van de Meent
On Fri, 10 Jan 2025 at 13:42, Yura Sokolov wrote: > > BTW, your version could make alike trick for guaranteed atomicity: > - change XLogRecord's `XLogRecPtr xl_prev` to `uint32 xl_prev_offset` > and store offset to prev record's start. -1, I don't think that is possible without degrading what our

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-10 Thread Yura Sokolov
09.01.2025 19:03, Zhou, Zhiguo пишет: On 1/7/2025 10:49 AM, Юрий Соколов wrote: On 6 Jan 2025, at 09:46, Zhou, Zhiguo wrote: Hi Yura and Wenhui, Thanks for kindly reviewing this work! On 1/3/2025 9:01 PM, wenhui qiu wrote: Hi Thank you for your path,NUM_XLOGINSERT_LOCKS increase to 128,I

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-09 Thread Zhou, Zhiguo
On 1/7/2025 10:49 AM, Юрий Соколов wrote: On 6 Jan 2025, at 09:46, Zhou, Zhiguo wrote: Hi Yura and Wenhui, Thanks for kindly reviewing this work! On 1/3/2025 9:01 PM, wenhui qiu wrote: Hi Thank you for your path,NUM_XLOGINSERT_LOCKS increase to 128,I think it will be challenged,do we make

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-06 Thread Юрий Соколов
On 6 Jan 2025, at 09:46, Zhou, Zhiguo wrote:Hi Yura and Wenhui,Thanks for kindly reviewing this work!On 1/3/2025 9:01 PM, wenhui qiu wrote:Hi    Thank you for your path,NUM_XLOGINSERT_LOCKS increase to 128,I think it will be challenged,do we make it guc ?I noticed there have been some discussions

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-06 Thread wenhui qiu
HI Zhiguo > Maybe we could leave the NUM_XLOGINSERT_LOCKS unchanged in this patch, > as it is not a hard dependency of the lock-free algorithm. And when this > patch has been fully accepted, we could then investigate the more proper > way of increasing NUM_XLOGINSERT_LOCKS. WDYT? If the value is no

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-06 Thread Zhou, Zhiguo
Maybe we could leave the NUM_XLOGINSERT_LOCKS unchanged in this patch, as it is not a hard dependency of the lock-free algorithm. And when this patch has been fully accepted, we could then investigate the more proper way of increasing NUM_XLOGINSERT_LOCKS. WDYT? On 1/6/2025 4:35 PM, wenhui qiu

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-06 Thread wenhui qiu
HI Zhiguo Thank you for your reply ,Then you'll have to prove that 128 is the optimal value, otherwise they'll have a hard time agreeing with you on this patch. Thanks On Mon, Jan 6, 2025 at 2:46 PM Zhou, Zhiguo wrote: > Hi Yura and Wenhui, > > Thanks for kindly reviewing this work! > > On

Re: RFC: Lock-free XLog Reservation from WAL

2025-01-05 Thread Zhou, Zhiguo
Hi Japin, Thanks so much for your test and review. As you may have noticed, this patch has implemented the initial optimization idea and has passed only the basic regression tests. We have planned to extend the validation to include TAP tests after aligning the expectations with the community,

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-05 Thread Zhou, Zhiguo
Hi Yura and Wenhui, Thanks for kindly reviewing this work! On 1/3/2025 9:01 PM, wenhui qiu wrote: Hi     Thank you for your path,NUM_XLOGINSERT_LOCKS increase to 128,I think it will be challenged,do we make it guc ? I noticed there have been some discussions (for example, [1] and its res

Re: RFC: Lock-free XLog Reservation from WAL

2025-01-05 Thread Japin Li
Hi, Zhigou Thanks for the patch. On Thu, 02 Jan 2025 at 09:14, "Zhou, Zhiguo" wrote: > Hi all, > > I am reaching out to solicit your insights and comments on a recent > proposal regarding the "Lock-free XLog Reservation from WAL." We have > identified some challenges with the current WAL inser

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-03 Thread wenhui qiu
Hi Thank you for your path,NUM_XLOGINSERT_LOCKS increase to 128,I think it will be challenged,do we make it guc ? On Fri, 3 Jan 2025 at 20:36, Yura Sokolov wrote: > 02.01.2025 10:05, Zhou, Zhiguo wrote: > > Hi all, > > > > I am reaching out to solicit your insights and comments on a recent

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-03 Thread Yura Sokolov
02.01.2025 10:05, Zhou, Zhiguo wrote: Hi all, I am reaching out to solicit your insights and comments on a recent proposal regarding the "Lock-free XLog Reservation from WAL." We have identified some challenges with the current WAL insertions, which require space reservations in the WAL buffer

RE: RFC: Lock-free XLog Reservation from WAL

2025-01-02 Thread Zhou, Zhiguo
5:15 PM To: pgsql-hackers@lists.postgresql.org Subject: RFC: Lock-free XLog Reservation from WAL Hi all, I am reaching out to solicit your insights and comments on a recent proposal regarding the "Lock-free XLog Reservation from WAL." We have identified some challenges with the c

RE: [RFC] Lock-free XLog Reservation from WAL

2025-01-02 Thread Zhou, Zhiguo
3:20 PM To: pgsql-hackers@lists.postgresql.org Subject: [RFC] Lock-free XLog Reservation from WAL Hi all, I am reaching out to solicit your insights and comments on a recent proposal regarding the "Lock-free XLog Reservation from WAL." We have identified some challenges with the c

RFC: Lock-free XLog Reservation from WAL

2025-01-02 Thread Zhou, Zhiguo
Hi all, I am reaching out to solicit your insights and comments on a recent proposal regarding the "Lock-free XLog Reservation from WAL." We have identified some challenges with the current WAL insertions, which require space reservations in the WAL buffer which involve updating two shared-memo

[RFC] Lock-free XLog Reservation from WAL

2025-01-02 Thread Zhou, Zhiguo
Hi all, I am reaching out to solicit your insights and comments on a recent proposal regarding the "Lock-free XLog Reservation from WAL." We have identified some challenges with the current WAL insertions, which require space reservations in the WAL buffer which involve updating two shared-memo

[RFC] Lock-free XLog Reservation from WAL

2025-01-02 Thread Zhou, Zhiguo
Hi all, I am reaching out to solicit your insights and comments on a recent proposal regarding the "Lock-free XLog Reservation from WAL." We have identified some challenges with the current WAL insertions, which require space reservations in the WAL buffer which involve updating two shared-memo