Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-04-07 Thread Julien Rouhaud
On Mon, Apr 8, 2019 at 8:01 AM Fujii Masao wrote: > > 2019年4月8日(月) 14:22 Tsunakawa, Takayuki : >> >> From: Alvaro Herrera [mailto:alvhe...@2ndquadrant.com] >> > "vacuum_truncate" gets my vote too. >> >> +1 > > > +1 > ISTM that we have small consensus to > use "vacuum_truncate". I'm also fine with

Re: Re: A separate table level option to control compression

2019-04-07 Thread Michael Paquier
On Sat, Apr 06, 2019 at 09:18:18PM -0400, Andrew Dunstan wrote: > Yeah, I'd hoped for some more opinions. I agree we've run out of time :-( We are a couple of hours away from the freeze, so I have marked the patch as returned with feedback. Let's see if we can still do something for the other ite

Re: [HACKERS] Weaker shmem interlock w/o postmaster.pid

2019-04-07 Thread Noah Misch
On Thu, Apr 04, 2019 at 07:53:19AM -0700, Noah Misch wrote: > On Wed, Apr 03, 2019 at 07:05:43PM -0700, Noah Misch wrote: > > Pushed, but that broke two buildfarm members: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=idiacanthus&dt=2019-04-04%2000%3A33%3A14 > > https://buildfarm.post

Re: ToDo: show size of partitioned table

2019-04-07 Thread Amit Langote
Hi, On 2019/04/08 12:59, Pavel Stehule wrote: > ne 7. 4. 2019 v 21:13 odesílatel Alvaro Herrera > napsal: >> Added to the docs, and pushed. >> > > Thank you very much Thank you Alvaro for pushing this to completion. Also, thank you Justin and Pavel for reviewing it till the last minute. Looks

Re: change password_encryption default to scram-sha-256?

2019-04-07 Thread Michael Paquier
On Mon, Apr 08, 2019 at 09:08:05AM +0300, Heikki Linnakangas wrote: > I wouldn't hold my breath. That's the third PR to add SCRAM support already, > see also https://github.com/lib/pq/pull/788 and > https://github.com/lib/pq/pull/608. The project seems to lack the committer > manpower or round tuit

Re: clean up pg_checksums.sgml

2019-04-07 Thread Michael Paquier
On Sun, Apr 07, 2019 at 07:15:46PM -0500, Justin Pryzby wrote: > Sorry, the patch was on top of an brief effort I made to rename "check > checksums" to "verify checksums", before asking about the idea. > > PFA patch to master. Thanks for the patch, Justin. That looks indeed clearer after conside

Re: Strange coding in _mdfd_openseg()

2019-04-07 Thread Kyotaro HORIGUCHI
At Fri, 5 Apr 2019 18:44:15 +1300, Thomas Munro wrote in > On Thu, Apr 4, 2019 at 4:16 PM Kyotaro HORIGUCHI > wrote: > > At Wed, 3 Apr 2019 13:47:46 -0700, Andres Freund wrote > > in <20190403204746.2yumq7c2mirmo...@alap3.anarazel.de> > > > Yea, I totally agree it's weird. I'm not sure if I'd

Re: lazy_scan_heap() forgets to mark buffer dirty when setting all frozen?

2019-04-07 Thread Masahiko Sawada
On Mon, Apr 8, 2019 at 12:49 PM Andres Freund wrote: > > Hi, > > lazy_scan_heap() contains the following block: > > /* > * If the all-visible page is turned out to be all-frozen but > not > * marked, we should so mark it. Note that all_frozen is

Re: pg_rewind vs superuser

2019-04-07 Thread Michael Paquier
On Sun, Apr 07, 2019 at 03:06:56PM +0200, Magnus Hagander wrote: > So can we *detect* that this is the case? Because if so, we could perhaps > just wait for it to be done? Because there will always be one? Yes, this one is technically possible. We could add a timeout option which checks each N se

Re: change password_encryption default to scram-sha-256?

2019-04-07 Thread Heikki Linnakangas
On 08/04/2019 08:42, Andres Freund wrote: Seems go/pq might get it soon-ish: https://github.com/lib/pq/pull/833 I wouldn't hold my breath. That's the third PR to add SCRAM support already, see also https://github.com/lib/pq/pull/788 and https://github.com/lib/pq/pull/608. The project seems to

Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-04-07 Thread Fujii Masao
2019年4月8日(月) 14:22 Tsunakawa, Takayuki : > From: Alvaro Herrera [mailto:alvhe...@2ndquadrant.com] > > "vacuum_truncate" gets my vote too. > > +1 > +1 ISTM that we have small consensus to use "vacuum_truncate". regards,

Re: tableam scan-API patch broke foreign key validation

2019-04-07 Thread Andres Freund
Hi, On 2019-04-06 14:43:26 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2019-04-06 14:34:34 -0400, Tom Lane wrote: > >> Why should this code need to free anything? That'd be the responsibility > >> of the slot code, no? > > > Well, not really. If a slot doesn't hold heap tuples interna

Re: Shouldn't validateForeignKeyConstraint() reset memory context?

2019-04-07 Thread Andres Freund
Hi, On 2018-07-11 11:56:28 -0700, Andres Freund wrote: > while looking at the pluggable storage patch I noticed that > validateForeignKeyConstraint() calls RI_FKey_check() for each row > without resetting a memory / expression context. There's not too much > leakage in the called code, but there'

Re: Assert failure when validating foreign keys

2019-04-07 Thread Andres Freund
Hi, On 2019-03-25 11:04:05 -0700, Andres Freund wrote: > Hi, > > On 2019-03-24 23:54:53 +1300, David Rowley wrote: > > This results in an Assert failure on master and an elog ERROR prior to > > c2fe139c201: > > > > create role test_role with login; > > create table ref(a int primary key); > > gr

Re: change password_encryption default to scram-sha-256?

2019-04-07 Thread Andres Freund
Hi, On 2019-04-08 01:34:42 -0400, Tom Lane wrote: > Michael Paquier writes: > > From what I can see, the major drivers not using directly libpq > > support our SASL protocol: JDBC and npgsql. However I can count three > > of them which still don't support it: Crystal, pq (Go) and asyncpg. > > pq

Re: change password_encryption default to scram-sha-256?

2019-04-07 Thread Tom Lane
Michael Paquier writes: > From what I can see, the major drivers not using directly libpq > support our SASL protocol: JDBC and npgsql. However I can count three > of them which still don't support it: Crystal, pq (Go) and asyncpg. > pq and asyncpg are very popular on github, with at least 3000 s

Re: change password_encryption default to scram-sha-256?

2019-04-07 Thread Michael Paquier
On Sun, Apr 07, 2019 at 08:23:06PM +0200, David Fetter wrote: > Great idea! Does it make sense to test all, or at least some > significant fraction of the connectors listed in > https://wiki.postgresql.org/wiki/Client_Libraries by default? This is a more interesting list: https://wiki.postgresql.

RE: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-04-07 Thread Tsunakawa, Takayuki
From: Alvaro Herrera [mailto:alvhe...@2ndquadrant.com] > "vacuum_truncate" gets my vote too. +1 From: 'Andres Freund' [mailto:and...@anarazel.de] > Personally I think the name just needs some committer to make a > call. This largely is going to be used after encountering too many > cancellations

Re: Ordered Partitioned Table Scans

2019-04-07 Thread Amit Langote
On 2019/04/06 18:06, Julien Rouhaud wrote: > On Sat, Apr 6, 2019 at 2:45 AM David Rowley > wrote: >> >> On Sat, 6 Apr 2019 at 12:26, Tom Lane wrote: >>> Pushed with some hacking, mostly trying to improve the comments. >> >> Great! Many thanks for improving those and pushing it. >> >> Many thanks

Re: lazy_scan_heap() forgets to mark buffer dirty when setting all frozen?

2019-04-07 Thread Andres Freund
Hi, On 2019-04-08 00:48:20 -0400, Alvaro Herrera wrote: > On 2019-Apr-07, Andres Freund wrote: > > > lazy_scan_heap() contains the following block: > > > > /* > > * If the all-visible page is turned out to be all-frozen but > > not > > * marked, we should s

Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-04-07 Thread Alvaro Herrera
On 2019-Apr-08, Tom Lane wrote: > The closest match to that name, probably, is just "vacuum_truncate" --- > which was proposed at the beginning of March, but apparently also > rejected, because there's no subsequent reference. "vacuum_truncate" gets my vote too. -- Álvaro Herrera

Re: lazy_scan_heap() forgets to mark buffer dirty when setting all frozen?

2019-04-07 Thread Alvaro Herrera
On 2019-Apr-07, Andres Freund wrote: > lazy_scan_heap() contains the following block: > > /* >* If the all-visible page is turned out to be all-frozen but > not >* marked, we should so mark it. Note that all_frozen is only > valid >

Re: selecting from partitions and constraint exclusion

2019-04-07 Thread Amit Langote
Hi Thibaut, On 2019/04/06 1:12, Thibaut wrote: > Le 25/03/2019 à 01:31, Amit Langote a écrit : >> On 2019/03/22 17:17, Amit Langote wrote: >>> I'll add this to July fest to avoid forgetting about this. >> I'd forgotten to do this, but done today. :) >> >> Thanks, >> Amit > > Hello Amit, > > Just

Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-04-07 Thread Michael Paquier
On Mon, Apr 08, 2019 at 03:56:52AM +, Tsunakawa, Takayuki wrote: > Consensus on the name seems to use truncate rather than shrink (a > few poople kindly said they like shrink, and I'm OK with either > name.) And there's no dispute on the behavior. Do you see any > other point? I personally m

Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-04-07 Thread 'Andres Freund'
Hi, On 2019-04-08 00:38:44 -0400, Tom Lane wrote: > "Tsunakawa, Takayuki" writes: > > From: Tom Lane [mailto:t...@sss.pgh.pa.us] > >> And, as far as I can see from a quick review of the thread, > >> we don't really have consensus on the names and behaviors. Personally I think the name just needs

Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-04-07 Thread Tom Lane
"Tsunakawa, Takayuki" writes: > From: Tom Lane [mailto:t...@sss.pgh.pa.us] >> And, as far as I can see from a quick review of the thread, >> we don't really have consensus on the names and behaviors. > Consensus on the name seems to use truncate rather than shrink (a few poople > kindly said the

Re: Back-branch bugs with fully-prunable UPDATEs

2019-04-07 Thread Amit Langote
On 2019/04/08 1:57, Tom Lane wrote: > Amit Langote writes: >> On Sun, Apr 7, 2019 at 5:28 AM Tom Lane wrote: >>> This test script works fine in HEAD: >>> In v11, it suffers an assertion failure in ExecSetupPartitionTupleRouting. >>> In v10, it doesn't crash, but we do get >>> WARNING: relcache r

RE: Timeout parameters

2019-04-07 Thread Nagaura, Ryohei
Hi, Michael-san. > From: Michael Paquier [mailto:mich...@paquier.xyz] > I have just committed the GUC and libpq portion for TCP_USER_TIMEOUT after a > last lookup, and I have cleaned up a couple of places. It is a bit > disappointing > to see the option supported on Linux, but not on Windows, So

Re: ToDo: show size of partitioned table

2019-04-07 Thread Pavel Stehule
ne 7. 4. 2019 v 21:13 odesílatel Alvaro Herrera napsal: > On 2019-Apr-07, Pavel Stehule wrote: > > > ne 7. 4. 2019 v 20:27 odesílatel Alvaro Herrera < > alvhe...@2ndquadrant.com> > > napsal: > > > > In order for this to display sanely, I added the "Parent name" column > if > > > either the "n" fl

RE: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-04-07 Thread Tsunakawa, Takayuki
From: Tom Lane [mailto:t...@sss.pgh.pa.us] > And, as far as I can see from a quick review of the thread, > we don't really have consensus on the names and behaviors. Consensus on the name seems to use truncate rather than shrink (a few poople kindly said they like shrink, and I'm OK with either n

Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-04-07 Thread Tom Lane
"Tsunakawa, Takayuki" writes: > From: Andres Freund [mailto:and...@anarazel.de] >> I hope you realize feature freeze is in a few hours... > Ouch! Could you take care of committing the patch, please? I wouldn't be > able to express the sadness and tiredness just in case this is pushed to 13 >

lazy_scan_heap() forgets to mark buffer dirty when setting all frozen?

2019-04-07 Thread Andres Freund
Hi, lazy_scan_heap() contains the following block: /* * If the all-visible page is turned out to be all-frozen but not * marked, we should so mark it. Note that all_frozen is only valid * if all_visible is true, so we must chec

RE: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-07 Thread Tsunakawa, Takayuki
From: David Rowley [mailto:david.row...@2ndquadrant.com] > It would be good to get your view on the > shrink_bloated_locallocktable_v3.patch I worked on last night. I was > unable to measure any overhead to solving the problem that way. Thanks, it looks super simple and good. I understood the ide

Re: Transaction commits VS Transaction commits (with parallel) VS query mean time

2019-04-07 Thread Amit Kapila
On Mon, Apr 8, 2019 at 7:54 AM Jamison, Kirk wrote: > > On Monday, April 8, 2019 9:04 AM (GMT+9), Haribabu Kommi wrote: > > >On Thu, Apr 4, 2019 at 3:29 PM Amit Kapila wrote: > > > The patch looks good to me. I have changed the commit message and ran > > the pgindent in the attached patch. Can

Re: Back-branch bugs with fully-prunable UPDATEs

2019-04-07 Thread Etsuro Fujita
(2019/04/07 16:54), Amit Langote wrote: On Sun, Apr 7, 2019 at 5:28 AM Tom Lane wrote: This seems to be related to what Amit Langote complained of in https://www.postgresql.org/message-id/21e7eaa4-0d4d-20c2-a1f7-c7e96f4ce...@lab.ntt.co.jp but since there's no foreign tables involved at all, e

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-07 Thread David Rowley
On Mon, 8 Apr 2019 at 14:54, Tsunakawa, Takayuki wrote: > > From: 'Andres Freund' [mailto:and...@anarazel.de] > > Did you see that people measured slowdowns? > > Yeah, 0.5% decrease with pgbench -M prepared -S (select-only), which feels > like a somewhat extreme test case. And that might be with

RE: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-07 Thread Tsunakawa, Takayuki
From: 'Andres Freund' [mailto:and...@anarazel.de] > On 2019-04-08 02:28:12 +, Tsunakawa, Takayuki wrote: > > I think the linked list of LOCALLOCK approach is natural, simple, and > > good. > > Did you see that people measured slowdowns? Yeah, 0.5% decrease with pgbench -M prepared -S (select-

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-07 Thread 'Andres Freund'
Hi, On 2019-04-08 02:28:12 +, Tsunakawa, Takayuki wrote: > I think the linked list of LOCALLOCK approach is natural, simple, and > good. Did you see that people measured slowdowns? Greetings, Andres Freund

RE: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-07 Thread Tsunakawa, Takayuki
From: Tom Lane [mailto:t...@sss.pgh.pa.us] > On the whole I don't think there's an adequate case for committing > this patch. From: Andres Freund [mailto:and...@anarazel.de] > On 2019-04-05 23:03:11 -0400, Tom Lane wrote: > > If I reduce the number of partitions in Amit's example from 8192 > > to

RE: Transaction commits VS Transaction commits (with parallel) VS query mean time

2019-04-07 Thread Jamison, Kirk
On Monday, April 8, 2019 9:04 AM (GMT+9), Haribabu Kommi wrote: >On Thu, Apr 4, 2019 at 3:29 PM Amit Kapila >mailto:amit.kapil...@gmail.com>> wrote: >On Wed, Apr 3, 2019 at 10:45 AM Haribabu Kommi >mailto:kommi.harib...@gmail.com>> wrote: >> >> Thanks for the review. >> >> While changing the app

RE: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-04-07 Thread Tsunakawa, Takayuki
Hi Andres, Fujii-san, any committer, From: Andres Freund [mailto:and...@anarazel.de] > On 2019-04-08 09:52:27 +0900, Fujii Masao wrote: > > I'm thinking to commit this patch at first. We can change the term > > and add the support of "TRUNCATE" option for VACUUM command later. > > I hope you rea

Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-04-07 Thread Andres Freund
Hi, On 2019-04-08 09:52:27 +0900, Fujii Masao wrote: > I'm thinking to commit this patch at first. We can change the term > and add the support of "TRUNCATE" option for VACUUM command later. I hope you realize feature freeze is in a few hours... Greetings, Andres Freund

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

2019-04-07 Thread Andres Freund
Hi, On 2019-04-04 21:04:49 -0700, Andres Freund wrote: > On 2019-04-04 23:57:58 -0400, Tom Lane wrote: > > Andres Freund writes: > > > I think the right approach would be to do all of this in heap_insert and > > > heap_multi_insert. Whenever starting to work on a page, if INSERT_FROZEN > > > is s

Re: Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-04-07 Thread Masahiko Sawada
On Mon, Apr 8, 2019 at 9:52 AM Fujii Masao wrote: > > On Sat, Apr 6, 2019 at 2:04 AM Robert Haas wrote: > > > > On Thu, Apr 4, 2019 at 9:19 PM Masahiko Sawada > > wrote: > > > As INDEX_CLEANUP option has been added by commit a96c41f, the new > > > option for this feature could also accept zero

Re: Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-04-07 Thread Fujii Masao
On Sat, Apr 6, 2019 at 2:04 AM Robert Haas wrote: > > On Thu, Apr 4, 2019 at 9:19 PM Masahiko Sawada wrote: > > As INDEX_CLEANUP option has been added by commit a96c41f, the new > > option for this feature could also accept zero or one boolean > > argument, that is SHRINK_TABLE [true|false] and t

Cleanup/remove/update references to OID column

2019-04-07 Thread Justin Pryzby
Cleanup/remove/update references to OID column... ..in wake of 578b229718e8f. See also 93507e67c9ca54026019ebec3026de35d30370f9 1464755fc490a9911214817fe83077a3689250ab --- doc/src/sgml/ddl.sgml | 9 - doc/src/sgml/ref/insert.sgml | 12 +--- doc/src/sgml/ref/psql-ref.

Re: clean up pg_checksums.sgml

2019-04-07 Thread Justin Pryzby
On Sat, Mar 30, 2019 at 10:51:23AM +0900, Michael Paquier wrote: > On Fri, Mar 29, 2019 at 09:32:10AM -0500, Justin Pryzby wrote: > > PFA patch with minor improvements to documentation. > > Patch does not apply, and I have reworded the last paragraph about > failures while operating. Sorry, the p

RE: Timeout parameters

2019-04-07 Thread Tsunakawa, Takayuki
From: Michael Paquier [mailto:mich...@paquier.xyz] > I have just committed the GUC and libpq portion for TCP_USER_TIMEOUT after > a last lookup, and I have cleaned up a couple of places. Thank you for further cleanup and committing. > For the socket_timeout stuff, its way of solving the problem

Re: Transaction commits VS Transaction commits (with parallel) VS query mean time

2019-04-07 Thread Haribabu Kommi
On Thu, Apr 4, 2019 at 3:29 PM Amit Kapila wrote: > On Wed, Apr 3, 2019 at 10:45 AM Haribabu Kommi > wrote: > > > > Thanks for the review. > > > > While changing the approach to use the is_parallel_worker_flag, I thought > > that the rest of the stats are also not required to be updated and also

Re: Emacs vs pg_indent's weird indentation for function declarations

2019-04-07 Thread Thomas Munro
On Mon, Jan 28, 2019 at 9:48 PM Thomas Munro wrote: > On Mon, Jan 28, 2019 at 8:08 PM Michael Paquier wrote: > > If you could get pgindent smarter in this area, it would be really > > nice.. > > Ah, it's not indent doing it, it's pgindent's post_indent subroutine > trying to correct the effects o

Re: ToDo: show size of partitioned table

2019-04-07 Thread Justin Pryzby
PFA patch intended to have been previously attached... Hmm..maybe it should say |RELATION'S partitions is also displayed, along with the RELATION'S |description.

Re: ToDo: show size of partitioned table

2019-04-07 Thread Justin Pryzby
On Sun, Apr 07, 2019 at 03:13:36PM -0400, Alvaro Herrera wrote: > On 2019-Apr-07, Pavel Stehule wrote: > > > ne 7. 4. 2019 v 20:27 odesílatel Alvaro Herrera > > napsal: > > > > In order for this to display sanely, I added the "Parent name" column if > > > either the "n" flag is shown or a patter

What is the correct behaviour for a wCTE UPDATE followed by a DELETE?

2019-04-07 Thread Andres Freund
Hi, While fixing the report at https://postgr.es/m/19321.1554567...@sss.pgh.pa.us I noticed that our behaviour for deleting (or updating albeit less drastically) a row previously modified in the same query isn't particularly useful: DROP TABLE IF EXISTS blarg; CREATE TABLE blarg(data text, count

Re: [GSoC 2019] Proposal: Develop Performance Farm Database and Website

2019-04-07 Thread Ila B.
Thank you so much for your answer, it provided a clearer understanding of the task and it was really useful to complete my proposal which I’ve now submitted. I really hope to keep on working with Postgres. Best of luck to all GSoC students :) Ilaria > On 4 Apr 2019, at 18:59, Mark Wong wrote:

Re: ToDo: show size of partitioned table

2019-04-07 Thread Alvaro Herrera
On 2019-Apr-07, Pavel Stehule wrote: > ne 7. 4. 2019 v 20:27 odesílatel Alvaro Herrera > napsal: > > In order for this to display sanely, I added the "Parent name" column if > > either the "n" flag is shown or a pattern is given (previously it only > > appeared in the former case). > > I am thi

Re: ToDo: show size of partitioned table

2019-04-07 Thread Pavel Stehule
ne 7. 4. 2019 v 20:27 odesílatel Alvaro Herrera napsal: > On 2019-Apr-07, Pavel Stehule wrote: > > > ne 7. 4. 2019 v 19:16 odesílatel Alvaro Herrera < > alvhe...@2ndquadrant.com> > > napsal: > > > > > Here. > > > > + 1 > > BTW I'm not sure if you noticed, but I removed the error message "no > par

Re: ToDo: show size of partitioned table

2019-04-07 Thread Alvaro Herrera
On 2019-Apr-07, Pavel Stehule wrote: > ne 7. 4. 2019 v 19:16 odesílatel Alvaro Herrera > napsal: > > > Here. > > + 1 BTW I'm not sure if you noticed, but I removed the error message "no partitioned relations found" when the result was empty. This was mimicking \d behavior (which was justified

Re: change password_encryption default to scram-sha-256?

2019-04-07 Thread David Fetter
On Sun, Apr 07, 2019 at 12:59:05PM -0400, Tom Lane wrote: > Peter Eisentraut writes: > > Should we change the default of the password_encryption setting to > > 'scram-sha-256' in PG12? > > I thought we were going to wait a bit longer --- that just got added > last year, no? What do we know about

Re: Contribution to Perldoc for TestLib module in Postgres

2019-04-07 Thread Ramanarayana
Hi, Please find the updated patch. Added to the commitfest as well Regards, Ram. v2_perldoc_testlib.patch.patch Description: Binary data

Re: ToDo: show size of partitioned table

2019-04-07 Thread Pavel Stehule
ne 7. 4. 2019 v 19:16 odesílatel Alvaro Herrera napsal: > Here. > + 1 Pavel > > -- > Álvaro Herrerahttps://www.2ndQuadrant.com/ > PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services >

Re: ToDo: show size of partitioned table

2019-04-07 Thread Alvaro Herrera
Here. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services commit 39766894a30d2cbfea7904cb04e9bb0ed398b76c Author: Alvaro Herrera AuthorDate: Sun Apr 7 07:59:12 2019 -0400 CommitDate: Sun Apr 7 12:51:13 2019 -0400

Re: change password_encryption default to scram-sha-256?

2019-04-07 Thread Tom Lane
Peter Eisentraut writes: > Should we change the default of the password_encryption setting to > 'scram-sha-256' in PG12? I thought we were going to wait a bit longer --- that just got added last year, no? What do we know about the state of support in client libraries? re

Re: Back-branch bugs with fully-prunable UPDATEs

2019-04-07 Thread Tom Lane
Amit Langote writes: > On Sun, Apr 7, 2019 at 5:28 AM Tom Lane wrote: >> This test script works fine in HEAD: >> In v11, it suffers an assertion failure in ExecSetupPartitionTupleRouting. >> In v10, it doesn't crash, but we do get >> WARNING: relcache reference leak: relation "parttbl" not close

Trailing whitespaces in various documentations

2019-04-07 Thread Julien Rouhaud
Hi, While working on unrelated documentation change, I noticed some trailing whitespaces in various documentation files. PFA a simple patch to get rid of them (I didn't removed the one corresponding to psql output), if that helps. diff --git a/doc/src/sgml/client-auth.sgml b/doc/src/sgml/client-a

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-07 Thread David Rowley
On Mon, 8 Apr 2019 at 04:09, Tom Lane wrote: > Um ... I don't see where you're destroying the old hash? In CreateLocalLockHash. > Also, I entirely dislike wiring in assumptions about hash_seq_search's > private state structure here. I think it's worth having an explicit > entry point in dynahas

Re: Checksum errors in pg_stat_database

2019-04-07 Thread Julien Rouhaud
Thanks for looking it it! On Sun, Apr 7, 2019 at 4:36 PM Magnus Hagander wrote: > > I'm not sure I like the idea of using "" as the database > name. It's not very likely that somebody would be using that as a name for > their database, but i's not impossible. But it also just looks strrange. >

Re: pgbench - add minimal stats on initialization

2019-04-07 Thread Fabien COELHO
done in 0.68 s (drop 0.06 s, create table 0.02 s, generate 0.34 s, vacuum 0.13 s, primary keys 0.13 s). See the durations on the last line. It's even better with working TAP tests. -- Fabien.diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c index 99529de52a..76a5b87fe8 100

Re: ToDo: show size of partitioned table

2019-04-07 Thread Pavel Stehule
ne 7. 4. 2019 v 18:07 odesílatel Alvaro Herrera napsal: > On 2019-Apr-07, Pavel Stehule wrote: > > > ne 7. 4. 2019 v 17:27 odesílatel Justin Pryzby > > napsal: > > > > I think there's an issue with showing indices. You said that \dP > should be > > > same as \dPti, no? Right now, indices are n

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-07 Thread Tom Lane
David Rowley writes: > Okay. Here's another version with all the average locks code removed > that only recreates the table when it's completely empty. Um ... I don't see where you're destroying the old hash? Also, I entirely dislike wiring in assumptions about hash_seq_search's private state s

Re: query logging of prepared statements

2019-04-07 Thread Alvaro Herrera
On 2019-Apr-07, Justin Pryzby wrote: > [...] Since I've been using log_statement=all, and not > log_min_duration_statement, I don't have a strong opinion about it. Ah, so your plan for this in production is to use the sample-based logging facilities, I see! Did you get Adrien a beer already? --

Re: ToDo: show size of partitioned table

2019-04-07 Thread Alvaro Herrera
On 2019-Apr-07, Pavel Stehule wrote: > ne 7. 4. 2019 v 17:27 odesílatel Justin Pryzby > napsal: > > I think there's an issue with showing indices. You said that \dP should be > > same as \dPti, no? Right now, indices are not shown in \dP, unless a > > pattern is given. I see you add that beha

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-07 Thread David Rowley
On Mon, 8 Apr 2019 at 03:47, Andres Freund wrote: > Could you benchmark your adversarial case? Which case? I imagine the worst case for v2 is a query that just constantly asks for over 16 locks. Perhaps a prepared plan, so not to add planner overhead. -- David Rowley http://

Re: ToDo: show size of partitioned table

2019-04-07 Thread Pavel Stehule
ne 7. 4. 2019 v 14:15 odesílatel Alvaro Herrera napsal: > So how about the attached version? > +1 Pavel > > -- > Álvaro Herrerahttps://www.2ndQuadrant.com/ > PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services >

Re: ToDo: show size of partitioned table

2019-04-07 Thread Pavel Stehule
ne 7. 4. 2019 v 17:27 odesílatel Justin Pryzby napsal: > On Sun, Apr 07, 2019 at 08:15:06AM -0400, Alvaro Herrera wrote: > > So how about the attached version? > > +1 > > I found a few issues. > > \dP+ didn't work. Fix attached. > > +static const SchemaQuery Query_for_list_of_partitioned_relatio

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-07 Thread Andres Freund
Hi, On 2019-04-08 03:40:52 +1200, David Rowley wrote: > On Mon, 8 Apr 2019 at 03:20, Tom Lane wrote: > > > > David Rowley writes: > > > The reason I thought it was a good idea to track some history there > > > was to stop the lock table constantly being shrunk back to the default > > > size ever

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-07 Thread David Rowley
On Mon, 8 Apr 2019 at 03:20, Tom Lane wrote: > > David Rowley writes: > > The reason I thought it was a good idea to track some history there > > was to stop the lock table constantly being shrunk back to the default > > size every time a simple single table query was executed. > > I think that's

Re: query logging of prepared statements

2019-04-07 Thread Justin Pryzby
Hi, Thanks both for thinking about this. On Fri, Apr 05, 2019 at 06:16:38PM -0700, Andres Freund wrote: > On 2019-04-04 16:01:26 -0300, Alvaro Herrera wrote: > > Also, if you parse once and bind/execute many times, IMO the statement > > should be logged exactly once. I think you could that with

Re: ToDo: show size of partitioned table

2019-04-07 Thread Justin Pryzby
On Sun, Apr 07, 2019 at 08:15:06AM -0400, Alvaro Herrera wrote: > So how about the attached version? +1 I found a few issues. \dP+ didn't work. Fix attached. +static const SchemaQuery Query_for_list_of_partitioned_relations = {

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-07 Thread Tom Lane
David Rowley writes: > On Mon, 8 Apr 2019 at 02:59, Tom Lane wrote: >> We *should* be using hash_get_num_entries(), but only to verify >> that the table is empty before resetting it. The additional bit >> that is needed is to see whether the number of buckets is large >> enough to justify callin

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-07 Thread David Rowley
On Mon, 8 Apr 2019 at 02:59, Tom Lane wrote: > > David Rowley writes: > > hash_get_num_entries() looks cheap enough to me. Can you explain why > > you think that's too expensive? > > What I objected to cost-wise was counting the number of lock > acquisitions/releases, which seems entirely beside

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-07 Thread Tom Lane
David Rowley writes: > On Mon, 8 Apr 2019 at 02:20, Tom Lane wrote: >> I like the concept ... but the particular implementation, not so much. >> It seems way overcomplicated. In the first place, why should we >> add code to copy entries? Just don't do it except when the table >> is empty. In t

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-07 Thread David Rowley
On Mon, 8 Apr 2019 at 02:36, David Rowley wrote: > > LockMethodLocalHash is special in that it predictably goes to empty > > at the end of every transaction, so that de-bloating at that point > > is a workable strategy. I think we'd probably need something more > > robust if we were trying to fix

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-07 Thread David Rowley
On Mon, 8 Apr 2019 at 02:20, Tom Lane wrote: > I like the concept ... but the particular implementation, not so much. > It seems way overcomplicated. In the first place, why should we > add code to copy entries? Just don't do it except when the table > is empty. In the second, I think we could

Re: Checksum errors in pg_stat_database

2019-04-07 Thread Magnus Hagander
On Thu, Apr 4, 2019 at 2:52 PM Julien Rouhaud wrote: > On Thu, Apr 4, 2019 at 1:25 PM Magnus Hagander > wrote: > > > > On Thu, Apr 4, 2019 at 10:47 AM Julien Rouhaud > wrote: > >> > >> Actually we do track counters for shared relations (see > >> pgstat_report_stat), we just don't expose them in

Re: pg_rewind vs superuser

2019-04-07 Thread Magnus Hagander
On Fri, Apr 5, 2019 at 1:05 PM Michael Paquier wrote: > On Fri, Apr 05, 2019 at 10:39:26AM +0200, Michael Banck wrote: > > Ok, so the problem is that that checkpoint might be still ongoing when > > you quickly issue a pg_rewind from the other side? > > The end-of-recovery checkpoint may not have

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-07 Thread Tom Lane
David Rowley writes: > On Sat, 6 Apr 2019 at 16:03, Tom Lane wrote: >> My own thought about how to improve this situation was just to destroy >> and recreate LockMethodLocalHash at transaction end (or start) >> if its size exceeded $some-value. Leaving it permanently bloated seems >> like possib

Re: [PATCH] Implement uuid_version()

2019-04-07 Thread David Fetter
On Sat, Apr 06, 2019 at 12:35:47PM -0400, Tom Lane wrote: > Jose Luis Tallon writes: > >     While working on an application, the need arose to be able > > efficiently differentiate v4/v5 UUIDs (for use in partial indexes, among > > others) > > ... so please find attached a trivial patch which

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-07 Thread David Rowley
On Sat, 6 Apr 2019 at 16:03, Tom Lane wrote: > I'd also point out that this is hardly the only place where we've > seen hash_seq_search on nearly-empty hash tables become a bottleneck. > So I'm not thrilled about attacking that with one-table-at-time patches. > I'd rather see us do something to le

Re: [PATCH] Implement uuid_version()

2019-04-07 Thread Jose Luis Tallon
On 6/4/19 18:35, Tom Lane wrote: Jose Luis Tallon writes:     While working on an application, the need arose to be able efficiently differentiate v4/v5 UUIDs (for use in partial indexes, among others) ... so please find attached a trivial patch which adds the functionality. No particular ob

Re: Fix foreign key constraint check for partitioned tables

2019-04-07 Thread Alvaro Herrera
On 2019-Apr-06, Tom Lane wrote: > Hadi Moshayedi writes: > > This patch also changed the output of some of tests, i.e. previously > > foreign key constraint failures errored on the partitioned table itself, > > but now it shows the child table's name in the error message. I hope it is > > ok. >

Re: ToDo: show size of partitioned table

2019-04-07 Thread Alvaro Herrera
So how about the attached version? -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml index 2bc8bbc2a74..3587abce8b2 100644 --- a/doc/src/s

Re: Augment every test postgresql.conf

2019-04-07 Thread Andrew Dunstan
On Sun, Apr 7, 2019 at 2:41 AM Noah Misch wrote: > > On Sun, Dec 30, 2018 at 10:32:31AM -0500, Andrew Dunstan wrote: > > On 12/30/18 12:53 AM, Noah Misch wrote: > > > 2. stats_temp_directory is incompatible with TAP suites that start more > > > than > > >one node simultaneously. > > > The obv

Re: monitoring CREATE INDEX [CONCURRENTLY]

2019-04-07 Thread Peter Eisentraut
On 2019-04-06 06:40, Alvaro Herrera wrote: > On 2019-Apr-05, Peter Eisentraut wrote: > >> I've reworded the phases a bit. There was a bit of a mixup of waiting >> for snapshots and waiting for lockers. Perhaps not so important from a >> user's perspective, but at least now it's more consistent w

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-07 Thread Peter Eisentraut
On 2019-04-06 05:03, Tom Lane wrote: > Trying a standard pgbench test case (pgbench -M prepared -S with > one client and an -s 10 database), it seems that the patch is about > 0.5% slower than HEAD. Again, that's below the noise threshold, > but it's not promising for the net effects of this patch

change password_encryption default to scram-sha-256?

2019-04-07 Thread Peter Eisentraut
Should we change the default of the password_encryption setting to 'scram-sha-256' in PG12? -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Back-branch bugs with fully-prunable UPDATEs

2019-04-07 Thread Amit Langote
On Sun, Apr 7, 2019 at 5:28 AM Tom Lane wrote: > > This test script works fine in HEAD: > > drop table if exists parttbl cascade; > CREATE TABLE parttbl (a int, b int) PARTITION BY LIST (a); > CREATE TABLE parttbl_1 PARTITION OF parttbl FOR VALUES IN (NULL,500,501,502); > UPDATE parttbl SET a = NU