Re: Additional size of hash table is alway zero for hash aggregates

2020-03-13 Thread Pengzhou Tang
Thanks, Andres Freund and Andres Gierth. To be related, can I invite you to help to review the parallel grouping sets patches? It will be very great to hear some comments from you since you contributed most of the codes for grouping sets. the thread is https://www.postgresql.org/message-id/CAG4re

Re: RETURNING does not explain evaluation context for subqueries

2020-03-13 Thread Bruce Momjian
On Wed, Feb 5, 2020 at 04:32:45PM +, PG Doc comments form wrote: > The following documentation comment has been logged on the website: > > Page: https://www.postgresql.org/docs/11/dml-returning.html > Description: > > In the docs explaining RETURNING > https://www.postgresql.org/docs/11/dml-

Re: Additional size of hash table is alway zero for hash aggregates

2020-03-13 Thread Pengzhou Tang
On Fri, Mar 13, 2020 at 8:34 AM Andrew Gierth wrote: > > "Justin" == Justin Pryzby writes: > > > On Thu, Mar 12, 2020 at 12:16:26PM -0700, Andres Freund wrote: > >> Indeed, that's incorrect. Causes the number of buckets for the > >> hashtable to be set higher - the size is just used for t

Re: Additional size of hash table is alway zero for hash aggregates

2020-03-13 Thread Pengzhou Tang
> > On 2020-03-12 16:35:15 +0800, Pengzhou Tang wrote: > > When reading the grouping sets codes, I find that the additional size of > > the hash table for hash aggregates is always zero, this seems to be > > incorrect to me, attached a patch to fix it, please help to check. > > Indeed, that's incor

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-03-13 Thread James Coleman
On Friday, March 13, 2020, Tomas Vondra wrote: > On Fri, Mar 13, 2020 at 04:31:16PM -0400, James Coleman wrote: > >> On Fri, Mar 13, 2020 at 2:23 PM James Coleman wrote: >> >>> >>> On Tue, Mar 10, 2020 at 10:44 PM Tomas Vondra >>> wrote: >>> > 3) Most of the execution plans look reasonable, exc

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-03-13 Thread Tomas Vondra
On Fri, Mar 13, 2020 at 04:31:16PM -0400, James Coleman wrote: On Fri, Mar 13, 2020 at 2:23 PM James Coleman wrote: On Tue, Mar 10, 2020 at 10:44 PM Tomas Vondra wrote: > 3) Most of the execution plans look reasonable, except that some of the > plans look like this: > > >

Re: Berserk Autovacuum (let's save next Mandrill)

2020-03-13 Thread Justin Pryzby
On Fri, Mar 13, 2020 at 02:38:51PM -0700, Andres Freund wrote: > > |One disadvantage of decreasing vacuum_freeze_min_age is that it might cause > > |VACUUM to do useless work: freezing a row version is a waste of time if > > the row > > |is modified soon thereafter (causing it to acquire a new XID

Re: proposal: new polymorphic types - commontype and commontypearray

2020-03-13 Thread Tom Lane
Pavel Stehule writes: > [ anycompatible-types-20191127.patch ] I'm starting to review this patch seriously. I've found some bugs and things I didn't like, and the documentation certainly needs work, but I think I can get it to a committable state before too much longer. What I want to talk abou

Re: Berserk Autovacuum (let's save next Mandrill)

2020-03-13 Thread Laurenz Albe
On Fri, 2020-03-13 at 13:44 -0500, Justin Pryzby wrote: > Possible it would be better to run VACUUM *without* freeze_min_age=0 ? (I get > confused and have to spend 20min re-reading the vacuum GUC docs every time I > deal with this stuff, so maybe I'm off). > > As I understand, the initial motiva

Re: Berserk Autovacuum (let's save next Mandrill)

2020-03-13 Thread Andres Freund
Hi, On 2020-03-13 13:44:42 -0500, Justin Pryzby wrote: > As I understand, the initial motivation of this patch was to avoid disruptive > anti-wraparound vacuums on insert-only table. But if vacuum were triggered at > all, it would freeze the oldest tuples, which is all that's needed; especially >

Re: backend type in log_line_prefix?

2020-03-13 Thread Peter Eisentraut
On 2020-03-10 19:07, Alvaro Herrera wrote: I like these patches; the first two are nice cleanup. My only gripe is that pgstat_get_backend_desc() is not really a pgstat function; I think it should have a different name with a prototype in miscadmin.h (next to the enum's new location, which I woul

Re: backend type in log_line_prefix?

2020-03-13 Thread Peter Eisentraut
On 2020-03-11 19:53, Justin Pryzby wrote: Can I suggest: $ git diff diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c index 3a6f7f9456..56e0a1437e 100644 --- a/src/backend/utils/error/elog.c +++ b/src/backend/utils/error/elog.c @@ -2945,7 +2945,7 @@ write_csvlog(ErrorD

Re: WIP: WAL prefetch (another approach)

2020-03-13 Thread Alvaro Herrera
I tried my luck at a quick read of this patchset. I didn't manage to go over 0005 though, but I agree with Tomas that having this be configurable in terms of bytes of WAL is not very user-friendly. First of all, let me join the crowd chanting that this is badly needed; I don't need to repeat what

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-03-13 Thread James Coleman
On Fri, Mar 13, 2020 at 2:23 PM James Coleman wrote: > > On Tue, Mar 10, 2020 at 10:44 PM Tomas Vondra > wrote: > > 3) Most of the execution plans look reasonable, except that some of the > > plans look like this: > > > > > > QUERY PLAN > >---

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-03-13 Thread Tom Lane
Alvaro Herrera writes: > Also, I wonder if it would be better to modify our policies so that we > update typedefs.list more frequently. Some people include additions > with their commits, but it's far from SOP. Perhaps. My own workflow includes pulling down a fresh typedefs.list from the buildf

Re: database stuck in __epoll_wait_nocancel(). Are infinite timeouts safe?

2020-03-13 Thread Merlin Moncure
On Fri, Mar 13, 2020 at 2:28 PM Andres Freund wrote: > > Hi, > > On March 13, 2020 12:08:32 PM PDT, Merlin Moncure wrote: > >I have 5 servers in a testing environment that are comprise a data > >warehousing cluster. They will typically get each get exactly the > >same query at approximately the

Re: Re: Optimize crash recovery

2020-03-13 Thread Thomas Munro
On Sat, Mar 14, 2020 at 5:31 AM Alvaro Herrera wrote: > On 2020-Mar-14, Thunder wrote: > > For example, if page lsn in storage is 0x9 and start to replay from > > 0x1. > > If 0x1 is full-page xlog record, then we can ignore to replay xlog > > between 0x1~0x9 for this page. >

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-03-13 Thread Alvaro Herrera
On 2020-Mar-13, Tom Lane wrote: > Alvaro Herrera writes: > > ... You can specify a filelist to pgindent, also. What I do is super > > low-tech: do a "git diff origin/master", copy the filelist, and then > > ^V^E to paste that list into a command line to run pgindent (editing to > > remove the ch

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-03-13 Thread Tom Lane
Alvaro Herrera writes: > ... You can specify a filelist to pgindent, also. What I do is super > low-tech: do a "git diff origin/master", copy the filelist, and then > ^V^E to paste that list into a command line to run pgindent (editing to > remove the change histogram and irrelevant files). I sh

Re: database stuck in __epoll_wait_nocancel(). Are infinite timeouts safe?

2020-03-13 Thread Andres Freund
Hi, On March 13, 2020 12:08:32 PM PDT, Merlin Moncure wrote: >I have 5 servers in a testing environment that are comprise a data >warehousing cluster. They will typically get each get exactly the >same query at approximately the same time. Yesterday, around 1pm, 3 >of the five got stuck on th

Re: Extracting only the columns needed for a query

2020-03-13 Thread Dmitry Dolgov
> On Tue, Feb 18, 2020 at 03:26:16PM -0800, Melanie Plageman wrote: > > > > I believe it would be beneficial to add this potential API extension > > patch into > > > the thread (as an example of an interface defining how scanCols could be > > used) > > > and review them together. > > > > As for inc

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-03-13 Thread Alvaro Herrera
On 2020-Mar-13, James Coleman wrote: > > I don't propose to commit 0003 of course, since it's not our policy; > > that's just to allow running pgindent sanely, which gives you 0004 > > (though my local pgindent has an unrelated fix). And after that you > > notice the issue that 0005 fixes. > > I

database stuck in __epoll_wait_nocancel(). Are infinite timeouts safe?

2020-03-13 Thread Merlin Moncure
I have 5 servers in a testing environment that are comprise a data warehousing cluster. They will typically get each get exactly the same query at approximately the same time. Yesterday, around 1pm, 3 of the five got stuck on the same query. Each of them yields similar stack traces. This happ

Re: proposal: schema variables

2020-03-13 Thread Pavel Stehule
with explicit dest receiver setting. This design doesn't need to introduce new executor and planner nodes (like ModifyTable and ModifyTablePath). Regards Pavel > Regards > > Pavel > > >> Regards >> >> Pavel >> > schema-variables-20200313.patch.gz Description: application/gzip

Re: Berserk Autovacuum (let's save next Mandrill)

2020-03-13 Thread Justin Pryzby
On Tue, Mar 10, 2020 at 01:53:42PM +1300, David Rowley wrote: > 2. Perhaps the documentation in maintenance.sgml should mention that > the table will be vacuumed with the equivalent of having > vacuum_freeze_min_age = 0, instead of: > > "Such a vacuum will aggressively freeze tuples." > > aggress

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-03-13 Thread James Coleman
On Tue, Mar 10, 2020 at 10:44 PM Tomas Vondra wrote: > 3) Most of the execution plans look reasonable, except that some of the > plans look like this: > > > QUERY PLAN >- > Limit > -> GroupAggregate >

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-03-13 Thread Tomas Vondra
On Fri, Mar 13, 2020 at 01:16:44PM -0400, James Coleman wrote: On Tue, Mar 10, 2020 at 10:44 PM Tomas Vondra wrote: ... Now, a couple comments about parts 0001 - 0003 of the patch ... 1) I see a bunch of failures in the regression test, due to minor differences in the explain output. All the d

Re: explain HashAggregate to report bucket and memory stats

2020-03-13 Thread Andres Freund
Hi, On 2020-03-13 10:53:17 -0700, Jeff Davis wrote: > On Fri, 2020-03-13 at 10:27 -0700, Andres Freund wrote: > > On 2020-03-13 10:15:46 -0700, Jeff Davis wrote: > > > Also, is there a reason you report two different memory values > > > (hashtable and tuples)? I don't object, but it seems like a l

Re: explain HashAggregate to report bucket and memory stats

2020-03-13 Thread Jeff Davis
On Fri, 2020-03-13 at 10:27 -0700, Andres Freund wrote: > On 2020-03-13 10:15:46 -0700, Jeff Davis wrote: > > Also, is there a reason you report two different memory values > > (hashtable and tuples)? I don't object, but it seems like a little > > too > > much detail. > > Seems useful to me - the

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-03-13 Thread Andres Freund
Hi, On 2020-03-13 13:36:44 -0400, Tom Lane wrote: > James Coleman writes: > > On Tue, Mar 10, 2020 at 10:44 PM Tomas Vondra > > wrote: > >> 1) I see a bunch of failures in the regression test, due to minor > >> differences in the explain output. All the differences are about minor > >> changes i

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-03-13 Thread Tom Lane
James Coleman writes: > On Tue, Mar 10, 2020 at 10:44 PM Tomas Vondra > wrote: >> 1) I see a bunch of failures in the regression test, due to minor >> differences in the explain output. All the differences are about minor >> changes in memory usage, like this: >> >> - "Sort Space U

Re: explain HashAggregate to report bucket and memory stats

2020-03-13 Thread Andres Freund
Hi, On 2020-03-13 10:15:46 -0700, Jeff Davis wrote: > Also, is there a reason you report two different memory values > (hashtable and tuples)? I don't object, but it seems like a little too > much detail. Seems useful to me - the hashtable is pre-allocated based on estimates, whereas the tuples a

Re: explain HashAggregate to report bucket and memory stats

2020-03-13 Thread Jeff Davis
+ /* hashtable->entrysize includes additionalsize */ + hashtable->instrument.space_peak_hash = Max( + hashtable->instrument.space_peak_hash, + hashtable->hashtab->size * sizeof(TupleHashEntryData)); + + hashtabl

Re: shared-memory based stats collector

2020-03-13 Thread Andres Freund
Hi, On 2020-03-13 16:34:50 +0900, Kyotaro Horiguchi wrote: > Thank you very much!! > > At Thu, 12 Mar 2020 20:13:24 -0700, Andres Freund wrote > in > > Hi, > > > > Thomas, could you look at the first two patches here, and my review > > questions? > > > > > > General comments about this seri

Re: range_agg

2020-03-13 Thread Tom Lane
Paul A Jungwirth writes: > On Wed, Mar 11, 2020 at 4:39 PM Paul A Jungwirth > wrote: >> Oh, my last email left out the most important part. :-) Is this >> failure online somewhere so I can take a look at it and fix it? Look for your patch(es) at http://commitfest.cputube.org Right now it's not

Re: Add an optional timeout clause to isolationtester step.

2020-03-13 Thread Tom Lane
Julien Rouhaud writes: > It seems that for all the possibly interesting cases, what we want to wait on > is an heavyweight lock, which is already what isolationtester detects. Maybe > we could simply implement something like > step "" [ WAIT UNTIL BLOCKED ] { } > without any change to the bloc

Re: Additional improvements to extended statistics

2020-03-13 Thread Dean Rasheed
On Mon, 9 Mar 2020 at 00:06, Tomas Vondra wrote: > > On Mon, Mar 09, 2020 at 01:01:57AM +0100, Tomas Vondra wrote: > > > >Attaches is an updated patch series > >with parts 0002 and 0003 adding tests demonstrating the issue and then > >fixing it (both shall be merged to 0001). > > > > One day I won

Re: backup manifests

2020-03-13 Thread Robert Haas
On Fri, Mar 13, 2020 at 9:53 AM tushar wrote: > run pg_validatebackup - > > [centos@tushar-ldap-docker bin]$ ./pg_validatebackup data1 > pg_validatebackup: error: could not open file "pg_hba.conf": Permission denied > > run pg_validatebackup with switch -s > > [centos@tushar-ldap-docker bin]$ ./p

Re: range_agg

2020-03-13 Thread Paul A Jungwirth
On Wed, Mar 11, 2020 at 4:39 PM Paul A Jungwirth wrote: > > On Sat, Mar 7, 2020 at 12:20 PM Tom Lane wrote: > > Alvaro Herrera writes: > > > [ v11 patches ] > > The cfbot isn't too happy with this; it's getting differently-ordered > > results than you apparently did for the list of owned objects

Re: range_agg

2020-03-13 Thread Paul A Jungwirth
On Thu, Mar 12, 2020 at 5:38 AM Alvaro Herrera wrote: > ... thinking about gist+spgist, I think they could be written > identically to those for ranges, using the lowest (first) lower bound > and the higher (last) upper bound. > > ... thinking about selectivity, I think the way to write that is to

Re: Re: Optimize crash recovery

2020-03-13 Thread Alvaro Herrera
On 2020-Mar-14, Thunder wrote: > For example, if page lsn in storage is 0x9 and start to replay from > 0x1. > If 0x1 is full-page xlog record, then we can ignore to replay xlog > between 0x1~0x9 for this page. > > > Is there any correct issue if the page exists in the buffe

Re: Add an optional timeout clause to isolationtester step.

2020-03-13 Thread Julien Rouhaud
On Fri, Mar 13, 2020 at 10:12:20AM -0400, Tom Lane wrote: > Julien Rouhaud writes: > > > I'm not familiar with those test so I'm probably missing something, but > > looks > > like all isolation tests that setup a timeout are doing so to test server > > side > > features (deadlock detection, stat

Re: BEFORE ROW triggers for partitioned tables

2020-03-13 Thread Alvaro Herrera
On 2020-Mar-11, Ashutosh Bapat wrote: > On Thu, Feb 27, 2020 at 10:22 PM Alvaro Herrera > wrote: > > * The new function I added, ReportTriggerPartkeyChange(), contains one > > serious bug (namely: it doesn't map attribute numbers properly if > > partitions are differently defined). > > IIUC the

Re:Re: Optimize crash recovery

2020-03-13 Thread Thunder
For example, if page lsn in storage is 0x9 and start to replay from 0x1. If 0x1 is full-page xlog record, then we can ignore to replay xlog between 0x1~0x9 for this page. Is there any correct issue if the page exists in the buffer pool and ignore to replay for full-page or i

Re: [PATCH] Use PKG_CHECK_MODULES to detect the libxml2 library

2020-03-13 Thread Tom Lane
Daniel Gustafsson writes: >> On 12 Mar 2020, at 17:39, Tom Lane wrote: >> I'd originally thought that we might back-patch this, but I'm now of >> the opinion that we probably should not. If pkg-config is present, >> this can change the default behavior about where we get libxml from, >> which se

Re: PATCH: add support for IN and @> in functional-dependency statistics use

2020-03-13 Thread Tomas Vondra
On Fri, Mar 13, 2020 at 08:42:49AM +, Dean Rasheed wrote: On Thu, 12 Mar 2020 at 17:30, Tomas Vondra wrote: I'm sorry, but I don't see how we could do this for arbitrary clauses. I think we could do that for clauses that have equality semantics and reference column values as a whole. So I

Re: [PATCH] Use PKG_CHECK_MODULES to detect the libxml2 library

2020-03-13 Thread Tom Lane
Hugh McMaster writes: > On Fri, 13 Mar 2020 at 03:39, Tom Lane wrote: >> That won't win us any friends, so the attached revision >> doesn't call PKG_CHECK_MODULES unless we found pkg-config. > Did you mean to terminate configure if pkg-config cannot find > libxml-2.0 or the library is too old? Yo

Re: Optimize crash recovery

2020-03-13 Thread Alvaro Herrera
On 2020-Mar-13, Thunder wrote: > Hello hackers: > > > During crash recovery, we compare most of the lsn of xlog record with page > lsn to determine if the record has already been replayed. > The exceptions are full-page and init-page xlog records. > It's restored if the xlog record includes a f

Optimize crash recovery

2020-03-13 Thread Thunder
Hello hackers: During crash recovery, we compare most of the lsn of xlog record with page lsn to determine if the record has already been replayed. The exceptions are full-page and init-page xlog records. It's restored if the xlog record includes a full-page image of the page. And it initializes

Re: Refactor compile-time assertion checks for C/C++

2020-03-13 Thread Tom Lane
Michael Paquier writes: > Hmm. v3 actually broke the C++ fallback of StaticAssertExpr() and > StaticAssertStmt() (v1 did not), a simple fix being something like > the attached. The buildfarm seems happy, so why do you think it's broken? If we do need to change it, I'd be inclined to just use th

Re: make check crashes on POWER8 machine

2020-03-13 Thread Tom Lane
Victor Wagner writes: > Justin Pryzby wrote: >> On Fri, Mar 13, 2020 at 10:29:13AM +0300, Victor Wagner wrote: >>> I've encountered a problem with Postgres on PowerPC machine. >> Is it related to >> https://www.postgresql.org/message-id/20032.1570808731%40sss.pgh.pa.us >> https://bugzilla.kernel

Re: allow online change primary_conninfo

2020-03-13 Thread Alvaro Herrera
On 2020-Jan-22, Michael Paquier wrote: > On Tue, Jan 21, 2020 at 06:03:18PM +0300, Sergei Kornilov wrote: > > PS: also, I surprised why it's ok for wal_receiver_create_temp_slot > > to be PGC_SIGHUP and ignore change of this setting until walreceiver > > will reconnect by unrelated reason. I means

Re: allow online change primary_conninfo

2020-03-13 Thread Alvaro Herrera
On 2020-Mar-13, Alvaro Herrera wrote: > What are the downsides of changing wal_receiver_create_temp_slot to > PGC_POSTMASTER? It seems pretty nasty to requires a full server > restart. Maybe we can signal all walreceivers at that point so that > they restart with the correct setting? That's muc

Re: [proposal] de-TOAST'ing using a iterator

2020-03-13 Thread Alvaro Herrera
On 2020-Jan-12, John Naylor wrote: > > I took a brief look at v11 to see if there's anything I can do to help > it move forward. I'm not yet sure how it would look code-wise to > implement Alvaro and Tomas's comments upthread, but I'm pretty sure > this part means the iterator-related structs are

Re: PATCH: add support for IN and @> in functional-dependency statistics use

2020-03-13 Thread Bruce Momjian
On Fri, Mar 13, 2020 at 08:42:49AM +, Dean Rasheed wrote: > On Thu, 12 Mar 2020 at 17:30, Tomas Vondra > wrote: > > > > I'm sorry, but I don't see how we could do this for arbitrary clauses. I > > think we could do that for clauses that have equality semantics and > > reference column values

Re: [Proposal] Global temporary tables

2020-03-13 Thread Prabhat Sahu
Hi Wenjing, Please check the below combination of GTT with Primary and Foreign key relations, with the ERROR message. *Case1:*postgres=# CREATE GLOBAL TEMPORARY TABLE gtt1(c1 serial PRIMARY KEY, c2 VARCHAR (50) UNIQUE NOT NULL) ON COMMIT *DELETE* ROWS; CREATE TABLE postgres=# CREATE GLOBAL TEMP

Re: Add an optional timeout clause to isolationtester step.

2020-03-13 Thread Tom Lane
Julien Rouhaud writes: > On Wed, Mar 11, 2020 at 05:52:54PM -0300, Alvaro Herrera wrote: >> On 2020-Mar-11, Tom Lane wrote: >>> I'd like to see an attempt to rewrite some of the existing >>> timeout-dependent test cases to use this facility instead of >>> long timeouts. >> +1. Those long timeout

Re: bitmaps and correlation

2020-03-13 Thread Justin Pryzby
There were no comments last month, so rebased, fixed tests, and kicked to next CF. -- Justin >From e754a93aff10cb435f5ecef923a810b9edc02d68 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Wed, 8 Jan 2020 19:23:51 -0600 Subject: [PATCH v5 1/2] Make more clear the computation of min/max IO.. .

Re: backup manifests

2020-03-13 Thread tushar
On 3/12/20 8:16 PM, tushar wrote: Seems like expected behavior to me. We could consider providing a more descriptive error message, but there's now way for it to work. Right , Error message need to be more user friendly . One scenario which i feel - should error out  even if  -s option is s

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2020-03-13 Thread Dilip Kumar
On Fri, Mar 13, 2020 at 3:39 PM Amit Kapila wrote: > > On Fri, Mar 13, 2020 at 8:37 AM Dilip Kumar wrote: > > > > On Thu, Mar 12, 2020 at 5:28 PM Amit Kapila wrote: > > > > > > On Thu, Mar 12, 2020 at 11:15 AM Dilip Kumar > > > wrote: > > > > > > > > I have fixed this in the attached patch set

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2020-03-13 Thread Dilip Kumar
On Fri, Mar 13, 2020 at 11:16 AM Dilip Kumar wrote: > > On Fri, Mar 13, 2020 at 11:08 AM Amit Kapila wrote: > > > > On Thu, Mar 12, 2020 at 3:04 PM Dilip Kumar wrote: > > > > > > On Wed, Mar 11, 2020 at 2:36 PM Amit Kapila > > > wrote: > > > > > > > > > > > > If we have no other choice, then I

Re: make check crashes on POWER8 machine

2020-03-13 Thread Victor Wagner
On Fri, 13 Mar 2020 07:43:59 -0500 Justin Pryzby wrote: > On Fri, Mar 13, 2020 at 10:29:13AM +0300, Victor Wagner wrote: > > Hi, > > > > I've encountered a problem with Postgres on PowerPC machine. > > Sometimes > > Is it related to > https://www.postgresql.org/message-id/20032.1570808731%40s

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-03-13 Thread Justin Pryzby
@cfbot: rebased onto 085b6b6679e73b9b386f209b4d625c7bc60597c0 The merge conflict presents another opportunity to solicit comments on the new approach. Rather than making "recurse into tmpdir" the end goal: - add a function to show metadata of an arbitrary dir; - add isdir arguments to pg_ls_

Re: make check crashes on POWER8 machine

2020-03-13 Thread Justin Pryzby
On Fri, Mar 13, 2020 at 10:29:13AM +0300, Victor Wagner wrote: > Hi, > > I've encountered a problem with Postgres on PowerPC machine. Sometimes Is it related to https://www.postgresql.org/message-id/20032.1570808731%40sss.pgh.pa.us https://bugzilla.kernel.org/show_bug.cgi?id=205183 (My initial r

Re: [Proposal] Global temporary tables

2020-03-13 Thread tushar
On 3/9/20 10:01 PM, 曾文旌(义从) wrote: Fixed in global_temporary_table_v18-pg13.patch. Thanks Wenjing. I am getting this error  "ERROR:  could not open file "base/13589/t3_16440": No such file or directory" if max_active_global_temporary_table set to 0 Please refer this scenario - postgres=#

Re: Restore replication settings when modifying a field type

2020-03-13 Thread Peter Eisentraut
On 2020-03-10 14:16, Euler Taveira wrote: On Thu, 5 Mar 2020 at 09:45, Peter Eisentraut > wrote: On 2020-02-11 00:38, Quan Zongliang wrote: > new patch attached. I didn't like so much how the updating of the replica identity was hacked

Re: [PATCH] Use PKG_CHECK_MODULES to detect the libxml2 library

2020-03-13 Thread Hugh McMaster
On Fri, 13 Mar 2020 at 03:39, Tom Lane wrote: > I poked at this issue a bit more and realized that Hugh's patch > flat-out breaks building --with-libxml in environments without > pkg-config, because PKG_CHECK_MODULES just gives up and dies > if there's no pkg-config (as we'd already found out in co

Re: Berserk Autovacuum (let's save next Mandrill)

2020-03-13 Thread Laurenz Albe
On Fri, 2020-03-13 at 07:00 -0500, Justin Pryzby wrote: > > 2. The new feature can be completely disabled. This might be very > > useful for people who suffer from auto-vacuum starvation. > > > Yes, but in particular so it can be completely disabled easily. > > How is it disabled ? By setting sc

Re: Berserk Autovacuum (let's save next Mandrill)

2020-03-13 Thread Laurenz Albe
On Fri, 2020-03-13 at 12:05 +0300, Darafei "Komяpa" Praliaskouski wrote: > 1. introduce no new parameters and trigger autovacuum if the number > > of inserts exceeds the regular vacuum threshold. > > > > 2. introduce the new parameters with high base threshold and zero scale > > factor. > >

Re: Berserk Autovacuum (let's save next Mandrill)

2020-03-13 Thread Justin Pryzby
On Wed, Mar 11, 2020 at 10:32:47AM +1300, David Rowley wrote: > 2. The new feature can be completely disabled. This might be very > useful for people who suffer from auto-vacuum starvation. On Thu, Mar 12, 2020 at 08:28:05PM +1300, David Rowley wrote: > Yes, but in particular so it can be complete

Re: Refactor compile-time assertion checks for C/C++

2020-03-13 Thread Michael Paquier
On Fri, Mar 13, 2020 at 03:12:34PM +0900, Michael Paquier wrote: > On Thu, Mar 12, 2020 at 09:43:54AM -0400, Tom Lane wrote: >> I don't feel a need to expend a whole lot of sweat there. The existing >> text is fine, it just bugged me that the code deals with three cases >> while the comment block

Re: truncating timestamps on arbitrary intervals

2020-03-13 Thread Isaac Morland
On Fri, 13 Mar 2020 at 03:13, John Naylor wrote: > On Wed, Feb 26, 2020 at 11:36 PM Tom Lane wrote: > > > > * In general, binning involves both an origin and a stride. When > > working with plain numbers it's almost always OK to set the origin > > to zero, but it's less clear to me whether that

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2020-03-13 Thread Dilip Kumar
On Fri, Mar 13, 2020 at 2:32 PM Kuntal Ghosh wrote: > > On Fri, Mar 13, 2020 at 8:29 AM Amit Kapila wrote: > > > > On Thu, Mar 12, 2020 at 7:50 PM Kuntal Ghosh > > wrote: > > > I think moving them inside a macro is a good idea. Also, I think we > > > should move all the Assert related code insi

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2020-03-13 Thread Amit Kapila
On Fri, Mar 13, 2020 at 8:37 AM Dilip Kumar wrote: > > On Thu, Mar 12, 2020 at 5:28 PM Amit Kapila wrote: > > > > On Thu, Mar 12, 2020 at 11:15 AM Dilip Kumar wrote: > > > > > > I have fixed this in the attached patch set. > > > > > > > I have modified your > > v4-0003-Conflict-Extension-Page-lo

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2020-03-13 Thread Kuntal Ghosh
On Fri, Mar 13, 2020 at 8:42 AM Dilip Kumar wrote: > > On Thu, Mar 12, 2020 at 7:50 PM Kuntal Ghosh > > wrote: > > > > > + /* > > > + * The relation extension or page lock can never participate in actual > > > + * deadlock cycle. See Asserts in LockAcquireExtended. So, there is > > > + * no ad

Re: Berserk Autovacuum (let's save next Mandrill)

2020-03-13 Thread Komяpa
On Fri, Mar 13, 2020 at 3:19 AM Laurenz Albe wrote: > > On Fri, 2020-03-13 at 09:10 +1300, David Rowley wrote: > > So you're suggesting we drive the insert-vacuums from existing > > scale_factor and threshold? What about the 1 billion row table > > example above? > > I am still not 100% certain i

Re: Add an optional timeout clause to isolationtester step.

2020-03-13 Thread Julien Rouhaud
On Wed, Mar 11, 2020 at 05:52:54PM -0300, Alvaro Herrera wrote: > On 2020-Mar-11, Tom Lane wrote: > > > We could re-use Julien's ideas about the isolation spec syntax by > > making it be, roughly, > > > > step "" { } [ blocked if "" "" ] > > > > and then those items would need to be passed as para

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2020-03-13 Thread Kuntal Ghosh
On Fri, Mar 13, 2020 at 8:29 AM Amit Kapila wrote: > > On Thu, Mar 12, 2020 at 7:50 PM Kuntal Ghosh > wrote: > > I think moving them inside a macro is a good idea. Also, I think we > > should move all the Assert related code inside some debugging macro > > similar to this: > > #ifdef LOCK_DEBUG

Re: PATCH: add support for IN and @> in functional-dependency statistics use

2020-03-13 Thread Dean Rasheed
On Thu, 12 Mar 2020 at 17:30, Tomas Vondra wrote: > > I'm sorry, but I don't see how we could do this for arbitrary clauses. I > think we could do that for clauses that have equality semantics and > reference column values as a whole. So I think it's possible to do this > for IN clauses (which is

Re: shared-memory based stats collector

2020-03-13 Thread Kyotaro Horiguchi
Thank you very much!! At Thu, 12 Mar 2020 20:13:24 -0700, Andres Freund wrote in > Hi, > > Thomas, could you look at the first two patches here, and my review > questions? > > > General comments about this series: > - A lot of the review comments feel like I've written them before, a > year

Re: BEFORE ROW triggers for partitioned tables

2020-03-13 Thread Peter Eisentraut
On 2020-03-12 05:17, Ashutosh Bapat wrote: On Wed, Mar 11, 2020 at 8:53 PM Ashutosh Bapat wrote: Will it be easier to subject the new tuple to the partition level constraints themselves and report if those are violated. See RelationGetPartitionQual() for getting partition constraints. This func

Re: Reducing WaitEventSet syscall churn

2020-03-13 Thread Kyotaro Horiguchi
Hello. At Tue, 10 Mar 2020 08:19:24 +0900 (JST), Kyotaro Horiguchi wrote in me> I'l continue reviewing in later mail. me> me> > 0005: "libpq: Add PQsocketChangeCount to advertise socket changes." me> At Thu, 27 Feb 2020 12:17:45 +1300, Thomas Munro wrote in > On Sat, Feb 8, 2020 at 10

Re: truncating timestamps on arbitrary intervals

2020-03-13 Thread John Naylor
On Wed, Feb 26, 2020 at 11:36 PM Tom Lane wrote: > > * In general, binning involves both an origin and a stride. When > working with plain numbers it's almost always OK to set the origin > to zero, but it's less clear to me whether that's all right for > timestamps. Do we need another optional a

Re: [PATCH] Skip llvm bytecode generation if LLVM is missing

2020-03-13 Thread Andres Freund
On 2020-03-13 14:08:12 +0800, Craig Ringer wrote: > The alternative would be to detect a missing clang and emit a much more > informative error than the current one that explicitly suggests retrying > with > > make with_llvm=no > > or setting with_llvm=no in the environment. That, that, that