Re: protocol-level wait-for-LSN

2024-11-04 Thread Robert Haas
On Wed, Oct 30, 2024 at 6:45 PM Tatsuo Ishii wrote: > doesn't work, right? I don't like clients need to know the exact order > of each protocol extensions. I agree with this criticism, at least for the most part. Years and years ago, the only way to specify EXPLAIN options was to say EXPLAIN [ANA

Re: protocol-level wait-for-LSN

2024-11-04 Thread Matthias van de Meent
On Wed, 30 Oct 2024, 18:45 Jelte Fennema-Nio, wrote: > > On Wed, 30 Oct 2024 at 18:18, Ants Aasma wrote: > > The idea is great, I have been wanting something like this for a long > > time. For future proofing it might be a good idea to not require the > > communicated-waited value to be a LSN. >

Re: protocol-level wait-for-LSN

2024-11-04 Thread Peter Eisentraut
On 30.10.24 10:03, Jelte Fennema-Nio wrote: On Mon, 28 Oct 2024 at 16:51, Peter Eisentraut wrote: Thoughts? + snprintf(xloc, sizeof(xloc), "%X/%X", LSN_FORMAT_ARGS(logptr)) + pq_sendstring(&buf, xloc); nit: I feel that sending the LSN as a string seems unn

Re: protocol-level wait-for-LSN

2024-10-31 Thread Jelte Fennema-Nio
On Thu, 31 Oct 2024 at 13:59, Jesper Pedersen wrote: > And, it will be confusing to clean-room implementations: When you have > this startup parameter then you get these message formats, when you have > this startup parameter then you get these message formats -- and what > about combinations ? Li

Re: protocol-level wait-for-LSN

2024-10-31 Thread Jesper Pedersen
Hi, On 10/30/24 3:49 PM, Jelte Fennema-Nio wrote: On Wed, 30 Oct 2024 at 19:04, Jesper Pedersen wrote: Having LSN would be nice, but to break all existing implementations, no. Having to specify with startup parameters how a core message format looks like sounds like a bad idea to me, It woul

Re: protocol-level wait-for-LSN

2024-10-30 Thread Tatsuo Ishii
>> So yes, each protocol extension needs to know about all the other >> protocol extensions that it can be used with. In practice we'll avoid >> doing crazy stuff so that the protocol extensions are orthogonal > > Just as an example, let's say we add a server-side query time to the > protocol (whi

Re: protocol-level wait-for-LSN

2024-10-30 Thread Jelte Fennema-Nio
On Wed, 30 Oct 2024 at 19:04, Jesper Pedersen wrote: > Having LSN would be nice, but to break all existing implementations, no. > Having to specify with startup parameters how a core message format > looks like sounds like a bad idea to me, It would really help if you would explain why you think

Re: protocol-level wait-for-LSN

2024-10-30 Thread Jesper Pedersen
Hi, On 10/30/24 1:45 PM, Jelte Fennema-Nio wrote: On Wed, 30 Oct 2024 at 18:18, Ants Aasma wrote: The idea is great, I have been wanting something like this for a long time. For future proofing it might be a good idea to not require the communicated-waited value to be a LSN. Yours and Matthi

Re: protocol-level wait-for-LSN

2024-10-30 Thread Jelte Fennema-Nio
On Wed, 30 Oct 2024 at 18:18, Ants Aasma wrote: > The idea is great, I have been wanting something like this for a long > time. For future proofing it might be a good idea to not require the > communicated-waited value to be a LSN. Yours and Matthias' feedback make total sense I think. From an im

Re: protocol-level wait-for-LSN

2024-10-30 Thread Ants Aasma
On Mon, 28 Oct 2024 at 17:51, Peter Eisentraut wrote: > This is something I hacked together on the way back from pgconf.eu. > It's highly experimental. > > The idea is to do the equivalent of pg_wal_replay_wait() on the protocol > level, so that it is ideally fully transparent to the application c

Re: protocol-level wait-for-LSN

2024-10-30 Thread Jelte Fennema-Nio
On Wed, 30 Oct 2024 at 12:53, Heikki Linnakangas wrote: > We might have made a mistake by calling this mechanism "protocol > extensions", because it makes people think of user-defined extensions. I think this is a real problem, that's probably worth fixing. I created a separate thread to address

Re: protocol-level wait-for-LSN

2024-10-30 Thread Tatsuo Ishii
> On Wed, 30 Oct 2024 at 07:49, Tatsuo Ishii wrote: >> > I think one would have to define that somehow. If it's useful, the >> > additional fields of both extensions could be appended, in some >> > defined order. But this is an interesting question to think about. >> >> I think this kind of exte

Re: protocol-level wait-for-LSN

2024-10-30 Thread Heikki Linnakangas
On 30/10/2024 13:34, Tatsuo Ishii wrote: On Wed, 30 Oct 2024 at 07:49, Tatsuo Ishii wrote: I think one would have to define that somehow. If it's useful, the additional fields of both extensions could be appended, in some defined order. But this is an interesting question to think about. I

Re: protocol-level wait-for-LSN

2024-10-30 Thread Tatsuo Ishii
>>> With this protocol extension, two things are changed: >>> >>> - The ReadyForQuery message sends back the current LSN. >> If other protocol extension X tries to add something to the >> ReadyForQuery message too, what would happen? > > I think one would have to define that somehow. If it's usef

Re: protocol-level wait-for-LSN

2024-10-30 Thread Jelte Fennema-Nio
On Mon, 28 Oct 2024 at 17:58, Heikki Linnakangas wrote: > > - The Query message sends an LSN to wait for. (This doesn't handle the > > extended query protocol yet.) > > I'd suggest adding a new message type for this, so that it works the > same with simple and extended query. Or if you just want

Re: protocol-level wait-for-LSN

2024-10-30 Thread Jelte Fennema-Nio
On Mon, 28 Oct 2024 at 16:51, Peter Eisentraut wrote: > Thoughts? + snprintf(xloc, sizeof(xloc), "%X/%X", LSN_FORMAT_ARGS(logptr)) + pq_sendstring(&buf, xloc); nit: I feel that sending the LSN as a string seems unnecessarily wasteful of bytes. I'd rather send

Re: protocol-level wait-for-LSN

2024-10-30 Thread Jelte Fennema-Nio
On Wed, 30 Oct 2024 at 07:49, Tatsuo Ishii wrote: > > I think one would have to define that somehow. If it's useful, the > > additional fields of both extensions could be appended, in some > > defined order. But this is an interesting question to think about. > > I think this kind of extension,

Re: protocol-level wait-for-LSN

2024-10-29 Thread Jesper Pedersen
Hi, On 10/29/24 12:03 PM, Jesper Pedersen wrote: On 10/28/24 12:58 PM, Heikki Linnakangas wrote: I'd suggest adding a new message type for this, so that it works the same with simple and extended query. Or if you just want to wait without issuing any query. I agree it is a good idea to have

Re: protocol-level wait-for-LSN

2024-10-29 Thread Jesper Pedersen
Hi Peter, On 10/28/24 12:58 PM, Heikki Linnakangas wrote: I'd suggest adding a new message type for this, so that it works the same with simple and extended query. Or if you just want to wait without issuing any query. I agree it is a good idea to have a feature like this. However, I agree w

Re: protocol-level wait-for-LSN

2024-10-29 Thread Matthias van de Meent
On Mon, 28 Oct 2024, 16:51 Peter Eisentraut, wrote: > > This is something I hacked together on the way back from pgconf.eu. > It's highly experimental. > > The idea is to do the equivalent of pg_wal_replay_wait() on the protocol > level, so that it is ideally fully transparent to the application c

Re: protocol-level wait-for-LSN

2024-10-28 Thread Peter Eisentraut
On 29.10.24 06:06, Tatsuo Ishii wrote: The patch adds a protocol extension called _pq_.wait_for_lsn as well as a libpq connection option wait_for_lsn to activate the same. (Use e.g., psql -d 'wait_for_lsn=1'.) With this protocol extension, two things are changed: - The ReadyForQuery message se

Re: protocol-level wait-for-LSN

2024-10-28 Thread Tatsuo Ishii
> The patch adds a protocol extension called _pq_.wait_for_lsn as well > as a libpq connection option wait_for_lsn to activate the same. (Use > e.g., psql -d 'wait_for_lsn=1'.) > > With this protocol extension, two things are changed: > > - The ReadyForQuery message sends back the current LSN.

Re: protocol-level wait-for-LSN

2024-10-28 Thread Jelte Fennema-Nio
On Mon, 28 Oct 2024 at 17:58, Heikki Linnakangas wrote: > > - The Query message sends an LSN to wait for. (This doesn't handle the > > extended query protocol yet.) > > I'd suggest adding a new message type for this, so that it works the > same with simple and extended query. Or if you just want

Re: protocol-level wait-for-LSN

2024-10-28 Thread Jelte Fennema-Nio
On Mon, 28 Oct 2024 at 16:51, Peter Eisentraut wrote: > The idea is to do the equivalent of pg_wal_replay_wait() on the protocol > level, so that it is ideally fully transparent to the application code. > The application just issues queries, and they might be serviced by a > primary or a standby,

Re: protocol-level wait-for-LSN

2024-10-28 Thread Heikki Linnakangas
On 28/10/2024 17:51, Peter Eisentraut wrote: This is something I hacked together on the way back from pgconf.eu. It's highly experimental. The idea is to do the equivalent of pg_wal_replay_wait() on the protocol level, so that it is ideally fully transparent to the application code. The appli

protocol-level wait-for-LSN

2024-10-28 Thread Peter Eisentraut
This is something I hacked together on the way back from pgconf.eu. It's highly experimental. The idea is to do the equivalent of pg_wal_replay_wait() on the protocol level, so that it is ideally fully transparent to the application code. The application just issues queries, and they might be