Re: Expose lock group leader pid in pg_stat_activity

2020-03-15 Thread Michael Paquier
On Mon, Mar 16, 2020 at 12:43:41AM -0500, Justin Pryzby wrote: > I think I see. Julien's v3 patch did this: > https://www.postgresql.org/message-id/attachment/106429/pgsa_leader_pid-v3.diff > + if (proc->lockGroupLeader) > + values[29

Re: Expose lock group leader pid in pg_stat_activity

2020-03-15 Thread Justin Pryzby
On Sun, Mar 15, 2020 at 11:27:52PM -0500, Justin Pryzby wrote: > On Tue, Jan 28, 2020 at 12:36:41PM +0100, Julien Rouhaud wrote: > > So, AFAICT the LockHashPartitionLockByProc is required when > > iterating/modifying lockGroupMembers or lockGroupLink, but just > > getting the leader pid should be s

Re: Expose lock group leader pid in pg_stat_activity

2020-03-15 Thread Justin Pryzby
On Tue, Jan 28, 2020 at 12:36:41PM +0100, Julien Rouhaud wrote: > So, AFAICT the LockHashPartitionLockByProc is required when > iterating/modifying lockGroupMembers or lockGroupLink, but just > getting the leader pid should be safe. This still seems unsafe: git show -U11 -w --patience b025f32e0b

Re: Expose lock group leader pid in pg_stat_activity

2020-02-06 Thread Michael Paquier
On Thu, Feb 06, 2020 at 09:23:33AM +0100, Julien Rouhaud wrote: > While on the topic, is there any reason why the backend stays a group leader > for the rest of its lifetime, and should we change that? Nothing happens without a reason. a1c1af2 is the original commit, and the thread is here: https

Re: Expose lock group leader pid in pg_stat_activity

2020-02-06 Thread Julien Rouhaud
On Thu, Feb 06, 2020 at 09:24:16AM +0900, Michael Paquier wrote: > On Wed, Feb 05, 2020 at 07:57:20AM +0100, Julien Rouhaud wrote: > > This looks good, thanks a lot! > > Thanks for double-checking. And done. Thanks! While on the topic, is there any reason why the backend stays a group leader fo

Re: Expose lock group leader pid in pg_stat_activity

2020-02-05 Thread Michael Paquier
On Wed, Feb 05, 2020 at 07:57:20AM +0100, Julien Rouhaud wrote: > This looks good, thanks a lot! Thanks for double-checking. And done. -- Michael signature.asc Description: PGP signature

Re: Expose lock group leader pid in pg_stat_activity

2020-02-04 Thread Julien Rouhaud
On Wed, Feb 05, 2020 at 10:48:31AM +0900, Michael Paquier wrote: > On Tue, Feb 04, 2020 at 03:27:25PM +0100, Julien Rouhaud wrote: > > I added some code comments to remind that we don't guarantee any consistency > > here. > > That's mostly fine. I have moved the comment related to > AuxiliaryPidG

Re: Expose lock group leader pid in pg_stat_activity

2020-02-04 Thread Michael Paquier
On Tue, Feb 04, 2020 at 03:27:25PM +0100, Julien Rouhaud wrote: > I added some code comments to remind that we don't guarantee any consistency > here. That's mostly fine. I have moved the comment related to AuxiliaryPidGetProc() within the inner part of its the "if" (or the comment should be chan

Re: Expose lock group leader pid in pg_stat_activity

2020-02-04 Thread Julien Rouhaud
On Thu, Jan 30, 2020 at 10:03:01PM +0900, Michael Paquier wrote: > On Tue, Jan 28, 2020 at 02:52:08PM +0100, Tomas Vondra wrote: > > On Tue, Jan 28, 2020 at 02:26:34PM +0100, Julien Rouhaud wrote: > >> There were already some dependencies between the rows since parallel > >> queries were added, as

Re: Expose lock group leader pid in pg_stat_activity

2020-01-30 Thread Michael Paquier
On Tue, Jan 28, 2020 at 02:52:08PM +0100, Tomas Vondra wrote: > On Tue, Jan 28, 2020 at 02:26:34PM +0100, Julien Rouhaud wrote: >> There were already some dependencies between the rows since parallel >> queries were added, as you could see eg. a parallel worker while no >> query is currently active

Re: Expose lock group leader pid in pg_stat_activity

2020-01-28 Thread Tomas Vondra
On Tue, Jan 28, 2020 at 02:26:34PM +0100, Julien Rouhaud wrote: On Tue, Jan 28, 2020 at 2:09 PM Tomas Vondra wrote: I agree a separate "leader_id" column is easier to work with, as it does not require unnesting and so on. As for the consistency, I agree we probably can't make this perfect, as

Re: Expose lock group leader pid in pg_stat_activity

2020-01-28 Thread Julien Rouhaud
On Tue, Jan 28, 2020 at 2:09 PM Tomas Vondra wrote: > > I agree a separate "leader_id" column is easier to work with, as it does > not require unnesting and so on. > > As for the consistency, I agree we probably can't make this perfect, as > we're fetching and processing the PGPROC records one by

Re: Expose lock group leader pid in pg_stat_activity

2020-01-28 Thread Tomas Vondra
On Tue, Jan 28, 2020 at 12:36:41PM +0100, Julien Rouhaud wrote: On Sat, Jan 18, 2020 at 3:51 AM Michael Paquier wrote: On Fri, Jan 17, 2020 at 05:07:55PM +0100, Julien Rouhaud wrote: > Oh indeed. But unless we hold some LWLock during the whole function > execution, we cannot guarantee a consi

Re: Expose lock group leader pid in pg_stat_activity

2020-01-28 Thread Julien Rouhaud
On Sat, Jan 18, 2020 at 3:51 AM Michael Paquier wrote: > > On Fri, Jan 17, 2020 at 05:07:55PM +0100, Julien Rouhaud wrote: > > Oh indeed. But unless we hold some LWLock during the whole function > > execution, we cannot guarantee a consistent view right? > > Yep. That's possible. > > > And isn't

Re: Expose lock group leader pid in pg_stat_activity

2020-01-17 Thread Michael Paquier
On Fri, Jan 17, 2020 at 05:07:55PM +0100, Julien Rouhaud wrote: > Oh indeed. But unless we hold some LWLock during the whole function > execution, we cannot guarantee a consistent view right? Yep. That's possible. > And isn't it already possible to e.g. see a parallel worker in > pg_stat_activi

Re: Expose lock group leader pid in pg_stat_activity

2020-01-17 Thread Julien Rouhaud
On Thu, Jan 16, 2020 at 8:49 AM Michael Paquier wrote: > > On Thu, Jan 16, 2020 at 04:27:27PM +0900, Michael Paquier wrote: > > While looking at the code, I think that we could refactor things a bit > > for raw_wait_event, wait_event_type and wait_event which has some > > duplicated code for backe

Re: Expose lock group leader pid in pg_stat_activity

2020-01-17 Thread Julien Rouhaud
On Thu, Jan 16, 2020 at 8:28 AM Michael Paquier wrote: > > On Fri, Dec 27, 2019 at 10:15:33AM +0100, Julien Rouhaud wrote: > > I think that not using "parallel" to name this field will help to > > avoid confusion if the lock group infrastructure is eventually used > > for something else, but that'

Re: Expose lock group leader pid in pg_stat_activity

2020-01-15 Thread Michael Paquier
On Thu, Jan 16, 2020 at 04:27:27PM +0900, Michael Paquier wrote: > While looking at the code, I think that we could refactor things a bit > for raw_wait_event, wait_event_type and wait_event which has some > duplicated code for backend and auxiliary processes. What about > filling in the wait even

Re: Expose lock group leader pid in pg_stat_activity

2020-01-15 Thread Michael Paquier
On Fri, Dec 27, 2019 at 10:15:33AM +0100, Julien Rouhaud wrote: > I think that not using "parallel" to name this field will help to > avoid confusion if the lock group infrastructure is eventually used > for something else, but that's only true if indeed we explain what a > lock group is. As you a

Re: Expose lock group leader pid in pg_stat_activity

2019-12-27 Thread Julien Rouhaud
On Fri, Dec 27, 2019 at 10:01 AM Sergei Kornilov wrote: > > Hello > > > As I understand it, lock group is some infrastructure that is used by > > parallel queries, but could be used for something else too. So if > > more documentation is needed, we should say something like "For now, > > only para

Re: Expose lock group leader pid in pg_stat_activity

2019-12-27 Thread Sergei Kornilov
Hello > As I understand it, lock group is some infrastructure that is used by > parallel queries, but could be used for something else too. So if > more documentation is needed, we should say something like "For now, > only parallel queries can have a lock group" or something like that. If lockGr

Re: Expose lock group leader pid in pg_stat_activity

2019-12-26 Thread Julien Rouhaud
Hello, On Thu, Dec 26, 2019 at 12:18 PM Sergei Kornilov wrote: > > I doubt that "Process ID of the lock group leader" is enough for user > documentation. I think we need note: > - this field is related to parallel query execution > - leader_pid = pid if process is parallel leader > - leader_pid

Re: Expose lock group leader pid in pg_stat_activity

2019-12-26 Thread Sergei Kornilov
Hello I doubt that "Process ID of the lock group leader" is enough for user documentation. I think we need note: - this field is related to parallel query execution - leader_pid = pid if process is parallel leader - leader_pid would point to pid of the leader if process is parallel worker - leade

Re: Expose lock group leader pid in pg_stat_activity

2019-12-26 Thread Guillaume Lelarge
Le jeu. 26 déc. 2019 à 10:26, Julien Rouhaud a écrit : > On Thu, Dec 26, 2019 at 10:20 AM Guillaume Lelarge > wrote: > > > > Le jeu. 26 déc. 2019 à 09:49, Julien Rouhaud a > écrit : > >> > >> On Thu, Dec 26, 2019 at 9:08 AM Guillaume Lelarge > >> wrote: > >> > > >> > Le mer. 25 déc. 2019 à 19:

Re: Expose lock group leader pid in pg_stat_activity

2019-12-26 Thread Julien Rouhaud
On Thu, Dec 26, 2019 at 10:20 AM Guillaume Lelarge wrote: > > Le jeu. 26 déc. 2019 à 09:49, Julien Rouhaud a écrit : >> >> On Thu, Dec 26, 2019 at 9:08 AM Guillaume Lelarge >> wrote: >> > >> > Le mer. 25 déc. 2019 à 19:30, Julien Rouhaud a écrit : >> >> >> >> On Wed, Dec 25, 2019 at 7:03 PM Jul

Re: Expose lock group leader pid in pg_stat_activity

2019-12-26 Thread Guillaume Lelarge
Le jeu. 26 déc. 2019 à 09:49, Julien Rouhaud a écrit : > On Thu, Dec 26, 2019 at 9:08 AM Guillaume Lelarge > wrote: > > > > Le mer. 25 déc. 2019 à 19:30, Julien Rouhaud a > écrit : > >> > >> On Wed, Dec 25, 2019 at 7:03 PM Julien Rouhaud > wrote: > >> > > >> > Guillaume (in Cc) recently pointe

Re: Expose lock group leader pid in pg_stat_activity

2019-12-26 Thread Julien Rouhaud
On Thu, Dec 26, 2019 at 9:08 AM Guillaume Lelarge wrote: > > Le mer. 25 déc. 2019 à 19:30, Julien Rouhaud a écrit : >> >> On Wed, Dec 25, 2019 at 7:03 PM Julien Rouhaud wrote: >> > >> > Guillaume (in Cc) recently pointed out [1] that it's currently not >> > possible to retrieve the list of paral

Re: Expose lock group leader pid in pg_stat_activity

2019-12-26 Thread Guillaume Lelarge
Le mer. 25 déc. 2019 à 19:30, Julien Rouhaud a écrit : > On Wed, Dec 25, 2019 at 7:03 PM Julien Rouhaud wrote: > > > > Guillaume (in Cc) recently pointed out [1] that it's currently not > > possible to retrieve the list of parallel workers for a given backend > > at the SQL level. His use case

Re: Expose lock group leader pid in pg_stat_activity

2019-12-25 Thread Julien Rouhaud
On Wed, Dec 25, 2019 at 7:03 PM Julien Rouhaud wrote: > > Guillaume (in Cc) recently pointed out [1] that it's currently not > possible to retrieve the list of parallel workers for a given backend > at the SQL level. His use case was to develop a function in plpgsql > to sample a given query wait

Expose lock group leader pid in pg_stat_activity

2019-12-25 Thread Julien Rouhaud
Hello, Guillaume (in Cc) recently pointed out [1] that it's currently not possible to retrieve the list of parallel workers for a given backend at the SQL level. His use case was to develop a function in plpgsql to sample a given query wait event, but it's not hard to imagine other useful use cas