Re: How to run in parallel in Postgres

2019-12-06 Thread Justin Pryzby
On Thu, Dec 05, 2019 at 12:10:42PM +, Lars Aksel Opsahl wrote: > I have a function that prepares data, so the big job can be run it in > parallel. > > Since Postgres now supports parallel I was wondering if it's easy to trigger > parallel dynamically created SQL calls. > > If you look at >

Re: How to run in parallel in Postgres

2019-12-06 Thread Ondrej Ivanič
Hi Lars, I have two suggestions: - `xargs` almost always present and it can run in parallel (-P) but script needs to be changed: for((i=1;i<=28;i++)); do echo "SELECT find_overlap_gap_single_cell('test_data.overlap_gap_input_t1','geom',4258,'test_data.overlap_gap_input_t1_res',${I},28);"; done |

Re: autovacuum locking question

2019-12-06 Thread Jeff Janes
On Fri, Dec 6, 2019 at 12:50 PM MichaelDBA wrote: > And Just to reiterate my own understanding of this... > > autovacuum priority is less than a user-initiated request, so issuing a > manual vacuum (user-initiated request) will not result in being cancelled. > Somethings happen in some situation

Re: unexpected result for wastedbytes query after vacuum full

2019-12-06 Thread Justin Pryzby
On Fri, Dec 06, 2019 at 05:18:20PM +, Mike Schanne wrote: > Hi all, > > This question is somewhat related to my previous question: > https://www.postgresql.org/message-id/0871fcf35ceb4caa8a2204ca9c38e330%40USEPRDEX1.corp.kns.com > > I was attempting to measure the benefit of doing a VACUUM FU

Re: Legal disclaimers on emails to this group

2019-12-06 Thread Tim Cross
Craig James writes: > (I've changed the original subject, "autovacuum locking question", of the > sender's email so as not to hijack that thread.) > > On Thu, Dec 5, 2019 at 2:26 PM Mike Schanne wrote: > >> Hi, >> >> I am investigating a performance problem... >> ... This email is non-binding,

threads (Re: Legal disclaimers on emails to this group)

2019-12-06 Thread Justin Pryzby
On Fri, Dec 06, 2019 at 10:42:19AM -0800, Craig James wrote: > (I've changed the original subject, "autovacuum locking question", of the > sender's email so as not to hijack that thread.) Note that threads are defined by these headers, not by "Subject". References: <0871fcf35ceb4caa8a2204ca9c38e.

Re: Legal disclaimers on emails to this group

2019-12-06 Thread Tom Lane
Craig James writes: > Sorry to be off topic, but this bugs me. Language is important. This isn't > directed at you specifically, but I see these disclaimers all the time. How > can you post to a public newsgroup that automatically reproduces your email > to thousands of subscribers, and additional

Legal disclaimers on emails to this group

2019-12-06 Thread Craig James
(I've changed the original subject, "autovacuum locking question", of the sender's email so as not to hijack that thread.) On Thu, Dec 5, 2019 at 2:26 PM Mike Schanne wrote: > Hi, > > I am investigating a performance problem... > ... This email is non-binding, is subject to contract, and neither

Re: autovacuum locking question

2019-12-06 Thread MichaelDBA
And Just to reiterate my own understanding of this... autovacuum priority is less than a user-initiated request, so issuing a manual vacuum (user-initiated request) will not result in being cancelled. Regards, Michael Vitale Jeff Janes wrote on 12/6/2019 12:47 PM: On Fri, Dec 6, 2019 at 10:55

Re: autovacuum locking question

2019-12-06 Thread Justin Pryzby
On Thu, Dec 05, 2019 at 06:49:06PM -0500, Tom Lane wrote: > The only part that would get canceled in response to somebody taking a > non-exclusive lock is the last step, which is truncation of unused blocks at > the end of the table; that requires an exclusive lock. On Thu, Dec 05, 2019 at 06:55:0

Re: autovacuum locking question

2019-12-06 Thread Jeff Janes
On Fri, Dec 6, 2019 at 10:55 AM Mike Schanne wrote: > The error is not actually showing up very often (I have 8 occurrences from > 11/29 and none since then). So maybe I should not be concerned about it. > I suspect we have an I/O bottleneck from other logs (i.e. long checkpoint > sync times), s

Re: autovacuum locking question

2019-12-06 Thread Tom Lane
Mike Schanne writes: > The error is not actually showing up very often (I have 8 occurrences from > 11/29 and none since then). So maybe I should not be concerned about it. I > suspect we have an I/O bottleneck from other logs (i.e. long checkpoint sync > times), so this error may be a sympto

unexpected result for wastedbytes query after vacuum full

2019-12-06 Thread Mike Schanne
Hi all, This question is somewhat related to my previous question: https://www.postgresql.org/message-id/0871fcf35ceb4caa8a2204ca9c38e330%40USEPRDEX1.corp.kns.com I was attempting to measure the benefit of doing a VACUUM FULL on my database. I was using the query found here: https://wiki.postg

Re: autovacuum locking question

2019-12-06 Thread Tom Lane
Mike Schanne writes: > Is this what you are referring to? > - Prevent VACUUM from trying to freeze an old multixact ID involving a > still-running transaction (Nathan Bossart, Jeremy Schneider) > This case would lead to VACUUM failing until the old transaction terminates. > https://www.postgresql

RE: autovacuum locking question

2019-12-06 Thread Mike Schanne
The error is not actually showing up very often (I have 8 occurrences from 11/29 and none since then). So maybe I should not be concerned about it. I suspect we have an I/O bottleneck from other logs (i.e. long checkpoint sync times), so this error may be a symptom rather than the cause. From

RE: autovacuum locking question

2019-12-06 Thread Mike Schanne
Is this what you are referring to? - Prevent VACUUM from trying to freeze an old multixact ID involving a still-running transaction (Nathan Bossart, Jeremy Schneider) This case would lead to VACUUM failing until the old transaction terminates. https://www.postgresql.org/docs/release/9.6.16/ Than

Re: How to run in parallel in Postgres

2019-12-06 Thread Lars Aksel Opsahl
>From: Laurenz Albe >Sent: Thursday, December 5, 2019 5:42 PM >To: Lars Aksel Opsahl ; >pgsql-performance@lists.postgresql.org >Subject: Re: How to run in parallel in Postgres > >On Thu, 2019-12-05 at 12:10 +, Lars Aksel Opsahl wrote: >> have a function that prepares data, so the big j