Re: Exposing the lock manager's WaitForLockers() to SQL

2024-01-11 Thread Will Mortensen
Here is a new series adding a single pg_wait_for_lockers() function that takes a boolean argument to control the interpretation of the lock mode. It omits LOCK's handling of descendant tables so it requires permissions directly on descendants in order to wait for locks on them. Not sure if that wou

Re: Exposing the lock manager's WaitForLockers() to SQL

2024-01-28 Thread Will Mortensen
On Fri, Jan 26, 2024 at 4:54 AM vignesh C wrote: > > CFBot shows that there is one warning as in [1]: > patching file doc/src/sgml/libpq.sgml > ... > [09:30:40.000] [943/2212] Compiling C object > src/backend/postgres_lib.a.p/storage_lmgr_lock.c.obj > [09:30:40.000] c:\cirrus\src\backend\storage\l

Re: Exposing the lock manager's WaitForLockers() to SQL

2024-01-28 Thread Will Mortensen
I guess the output of the deadlock test was unstable, so I simply removed it in v8 here, but I can try to fix it instead if it seems important to test that. v8-0001-Refactor-GetLockConflicts-into-more-general-GetLo.patch Description: Binary data v8-0003-Add-pg_wait_for_lockers-function.patch De

Re: Exposing the lock manager's WaitForLockers() to SQL

2024-01-28 Thread Will Mortensen
Minor style fix; sorry for the spam. v9-0001-Refactor-GetLockConflicts-into-more-general-GetLo.patch Description: Binary data v9-0002-Allow-specifying-single-lockmode-in-WaitForLocker.patch Description: Binary data v9-0003-Add-pg_wait_for_lockers-function.patch Description: Binary data

Re: [PATCH] LockAcquireExtended improvement

2024-03-26 Thread Will Mortensen
On Thu, Mar 14, 2024 at 1:15 PM Robert Haas wrote: > Seeing no further discussion, I have committed my version of this > patch, with your test case. This comment on ProcSleep() seems to have the values of dontWait backward (double negatives are tricky): * Result: PROC_WAIT_STATUS_OK if we ac

Re: Exposing the lock manager's WaitForLockers() to SQL

2024-03-26 Thread Will Mortensen
Rebased, fixed a couple typos, and reordered the isolation tests to put the most elaborate pair last. v11-0001-Refactor-GetLockConflicts-into-more-general-GetL.patch Description: Binary data v11-0002-Allow-specifying-single-lockmode-in-WaitForLocke.patch Description: Binary data v11-0003-Add-

Re: Exposing the lock manager's WaitForLockers() to SQL

2024-03-08 Thread Will Mortensen
Rebased and fixed conflicts. FWIW re: Andrey's comment in his excellent CF summary email[0]: we're currently using vanilla Postgres (via Gentoo) on single nodes, and not anything fancy like Citus. The Citus relationship is just that we were inspired by Marco's blog post there. We have a variety of

Exposing the lock manager's WaitForLockers() to SQL

2022-12-23 Thread Will Mortensen
Hi there, We'd like to be able to call the lock manager's WaitForLockers() and WaitForLockersMultiple() from SQL. Below I describe our use case, but basically I'm wondering if this: 1. Seems like a reasonable thing to do 2. Would be of interest upstream 3. Should be done with a new

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-01-11 Thread Will Mortensen
But I'm really no expert on SQL syntax or typical practice for things like this. Anything that works is fine with me. :-) As a possibly superfluous sidebar, I wanted to correct this part of my original message: > On Fri, Dec 23, 2022 at 11:43 AM Will Mortensen wrote: > > pg_s

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-01-11 Thread Will Mortensen
Hi Andres, On Wed, Jan 11, 2023 at 12:33 PM Andres Freund wrote: > I think such a function would still have to integrate enough with the lock > manager infrastructure to participate in the deadlock detector. Otherwise I > think you'd trivially end up with loads of deadlocks. Could you elaborate

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-01-12 Thread Will Mortensen
D And I don't know if there's any reason to use a _lockmode_ other than ACCESS SHARE. On Wed, Jan 11, 2023 at 11:03 PM Will Mortensen wrote: > > Hi Andres, > > On Wed, Jan 11, 2023 at 12:33 PM Andres Freund wrote: > > I think such a function would still have

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-01-12 Thread Will Mortensen
l need to integrate with > the deadlock detection. I see. What about it seems potentially unsuitable? > On 2023-01-11 23:03:30 -0800, Will Mortensen wrote: > > To my very limited understanding, from looking at the existing callers and > > the implementation of LOCK, that wou

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-01-12 Thread Will Mortensen
Hi Andres, On Thu, Jan 12, 2023 at 7:49 PM Andres Freund wrote: > Consider a scenario like this: > > tx 1: acquires RowExclusiveLock on tbl1 to insert rows > tx 2: acquires AccessShareLock on tbl1 > tx 2: WaitForLockers(ShareRowExclusiveLock, tbl1) ends up waiting for tx1 > tx 1: truncate tbl1 ne

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-12-23 Thread Will Mortensen
On Sun, Sep 3, 2023 at 11:16 PM Will Mortensen wrote: > I realized that for our use case, we'd ideally wait for holders of > RowExclusiveLock only, and not e.g. VACUUM holding > ShareUpdateExclusiveLock. Waiting for lockers in a specific mode seems > possible by general

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-12-23 Thread Will Mortensen
I meant to add that the example in the doc is adapted from Marco Slot's blog post linked earlier: https://www.citusdata.com/blog/2018/06/14/scalable-incremental-data-aggregation/

Re: Exposing the lock manager's WaitForLockers() to SQL

2024-01-06 Thread Will Mortensen
Simplified the code and docs, and rewrote the example with more prose instead of PL/pgSQL, which unfortunately made it longer, although it could be truncated. Not really sure what's best... v5-0001-Refactor-GetLockConflicts-into-more-general-GetLo.patch Description: Binary data v5-0003-Add-WAIT

Re: Exposing the lock manager's WaitForLockers() to SQL

2024-01-09 Thread Will Mortensen
Hi Laurenz, thanks for taking a look! On Sat, Jan 6, 2024 at 4:00 AM Laurenz Albe wrote: > While your original use case is valid, I cannot think of > any other use case. So it is a special-purpose statement that is only > useful for certain processing of append-only tables. It is definitely som

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-09-03 Thread Will Mortensen
I realized that for our use case, we'd ideally wait for holders of RowExclusiveLock only, and not e.g. VACUUM holding ShareUpdateExclusiveLock. Waiting for lockers in a specific mode seems possible by generalizing/duplicating WaitForLockersMultiple() and GetLockConflicts(), but I'd love to have a s

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-02-01 Thread Will Mortensen
Here is a first attempt at a WIP patch. Sorry about the MIME type. It doesn't take any locks on the tables, but I'm not super confident that that's safe, so any input would be appreciated. I omitted view support for simplicity, but if that seems like a requirement I'll see about adding it. I assu

[PATCH] fix doc example of bit-reversed MAC address

2022-06-03 Thread Will Mortensen
Pretty trivial since this is documenting something that Postgres *doesn't* do, but it incorrectly reversed only the bits of each nibble, not the whole byte. See e.g. https://www.ibm.com/docs/en/csfdcd/7.1?topic=ls-bit-ordering-in-mac-addresses for a handy table. diff --git a/doc/src/sgml/datatype.s

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-08-02 Thread Will Mortensen
Updated docs a bit. I'll see about adding this to the next CF in hopes of attracting a reviewer. :-) v3-0001-Add-WAIT-ONLY-option-to-LOCK-command.patch Description: Binary data

Re: [PATCH] LockAcquireExtended improvement

2024-05-17 Thread Will Mortensen
On Tue, Mar 26, 2024 at 7:14 PM Will Mortensen wrote: > This comment on ProcSleep() seems to have the values of dontWait > backward (double negatives are tricky): > > * Result: PROC_WAIT_STATUS_OK if we acquired the lock, > PROC_WAIT_STATUS_ERROR > * if not (if dontWait

Re: [PATCH] LockAcquireExtended improvement

2024-05-22 Thread Will Mortensen
Thanks! :-)

Re: Exposing the lock manager's WaitForLockers() to SQL

2024-05-30 Thread Will Mortensen
I got some very helpful off-list feedback from Robert Haas that this needed more self-contained explanation/motivation. So here goes. :-) This patch set adds a new SQL function pg_wait_for_lockers(), which waits for transactions holding specified table locks to commit or roll back. This can be use

Re: Exposing the lock manager's WaitForLockers() to SQL

2024-05-30 Thread Will Mortensen
I should add that the latest patches remove permissions checks because pg_locks doesn't have any, and improve the commit messages. Hope I didn't garble anything doing this late after the dev conference. :-) Robert asked me about other existing functions that could be leveraged, such as GetConflict

[PATCH] doc: fix markup indentation in MVCC

2023-06-19 Thread Will Mortensen
Trivial fix to make the indentation consistent. From 46977fbe5fa0a26ef77938a8fe30b9def062e8f8 Mon Sep 17 00:00:00 2001 From: Will Mortensen Date: Sat, 27 Aug 2022 17:07:11 -0700 Subject: [PATCH 1/6] doc: fix markup indentation in MVCC --- doc/src/sgml/mvcc.sgml | 16 1 file

[PATCH] doc: add missing mention of MERGE in MVCC

2023-06-19 Thread Will Mortensen
MERGE is now a data-modification command too. 0002-doc-add-missing-mention-of-MERGE-in-MVCC.patch Description: Binary data

Re: [PATCH] doc: add missing mention of MERGE in MVCC

2023-06-21 Thread Will Mortensen
I saw, thanks again! On Wed, Jun 21, 2023 at 4:08 PM Bruce Momjian wrote: > > On Mon, Jun 19, 2023 at 11:32:46PM -0700, Will Mortensen wrote: > > MERGE is now a data-modification command too. > > Yes, this has been applied too. > > -- > Bruce Momjian

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-07-04 Thread Will Mortensen
Updated patch with more tests and a first attempt at doc updates. As the commit message and doc now point out, using WaitForLockersMultiple() makes for a behavior difference with actually locking multiple tables, in that the combined set of conflicting locks is obtained only once for all tables, r

Re: Exposing the lock manager's WaitForLockers() to SQL

2024-07-21 Thread Will Mortensen
On Thu, May 30, 2024 at 12:01 AM Will Mortensen wrote: > FWIW, another solution might be to directly expose the functions that > WaitForLockers() calls, namely GetLockConflicts() (generalized to > GetLockers() in the first patch) to identify the transactions holding > th

README.tuplock and SHARE lock

2024-11-18 Thread Will Mortensen
README.tuplock says: > There is one exception > here: since infomask space is limited, we do not provide a separate bit > for SELECT FOR SHARE, so we have to use the extended info in a MultiXact in > that case. (The other cases, SELECT FOR UPDATE and SELECT FOR KEY SHARE, are > presumably more co

Re: README.tuplock and SHARE lock

2024-11-19 Thread Will Mortensen
Sounds good to me. :-)