Re: BUG #15346: Replica fails to start after the crash

2018-08-30 Thread Michael Paquier
On Fri, Aug 31, 2018 at 02:52:06PM +0900, Kyotaro HORIGUCHI wrote: > The patch inhibits turning off updateMinRecoveryPoint on other > than the startup process running crash-recovery except at the end > of XLogNeedsFlush. Yes that's a matter of safety, as I put into the truck any modules which may

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2018-08-30 Thread Ashutosh Bapat
On Thu, Aug 30, 2018 at 2:23 PM, Dmitry Dolgov <9erthali...@gmail.com> wrote: > >> I won't be working on this actively in the next commitfest. I will be >> glad if somebody else wants to take this up. If there's nobody, >> probably we should mark this entry as "returned with feedback" in the >> nex

Re: BUG #15346: Replica fails to start after the crash

2018-08-30 Thread Kyotaro HORIGUCHI
At Thu, 30 Aug 2018 18:48:55 -0700, Michael Paquier wrote in <20180831014855.gj15...@paquier.xyz> > On Fri, Aug 31, 2018 at 09:48:46AM +0900, Kyotaro HORIGUCHI wrote: > > Please wait a bit.. I have a concern about this. > > Sure, please feel free. Thanks. I looked though the patch and related

Re: Copy function for logical replication slots

2018-08-30 Thread Masahiko Sawada
On Wed, Aug 29, 2018 at 9:39 AM, Masahiko Sawada wrote: > On Tue, Aug 28, 2018 at 10:34 PM, Michael Paquier wrote: >> On Tue, Aug 28, 2018 at 04:14:04PM +0900, Masahiko Sawada wrote: >>> I think the copying from a slot that already reserved WAL would be >>> helpful for backup cases (maybe you sug

Re: TupleTableSlot abstraction

2018-08-30 Thread Amit Khandekar
On 28 August 2018 at 22:43, Ashutosh Bapat wrote: > On Fri, Aug 24, 2018 at 6:46 AM, Andres Freund wrote: >> >>> -/* >>> - * slot_getsysattr >>> - * This function fetches a system attribute of the slot's >>> current tuple. >>> - * Unlike slot_getattr, if the slot does not con

Re: Add a semicolon to query related to search_path

2018-08-30 Thread Tatsuro Yamada
On 2018/08/31 2:28, Peter Eisentraut wrote: On 17/08/2018 05:32, Tatsuro Yamada wrote: Hi Robert, On 2018/08/17 4:32, Robert Haas wrote: On Thu, Aug 16, 2018 at 1:20 AM, Tatsuro Yamada wrote: As you can see, queries with and without a semicolon are mixed, it is hard to understand the end of

Re: BUG #15346: Replica fails to start after the crash

2018-08-30 Thread Michael Paquier
On Fri, Aug 31, 2018 at 09:48:46AM +0900, Kyotaro HORIGUCHI wrote: > Please wait a bit.. I have a concern about this. Sure, please feel free. -- Michael signature.asc Description: PGP signature

Re: BUG #15346: Replica fails to start after the crash

2018-08-30 Thread Kyotaro HORIGUCHI
At Thu, 30 Aug 2018 11:57:05 -0700, Michael Paquier wrote in <20180830185705.gf15...@paquier.xyz> > On Thu, Aug 30, 2018 at 08:31:36PM +0200, Alexander Kukushkin wrote: > > 2018-08-30 19:34 GMT+02:00 Michael Paquier : > >> I have been struggling for a couple of hours to get a deterministic test >

Re: pg_verify_checksums and -fno-strict-aliasing

2018-08-30 Thread Michael Paquier
On Thu, Aug 30, 2018 at 07:37:37PM -0400, Tom Lane wrote: > Some of these are safe, I think, because the buffers are only used as > targets for read() and write(). But some are definitely broken. Yes, I have not spent more than a couple of minutes on this issue. I noticed some of them easily tho

Re: Stored procedures and out parameters

2018-08-30 Thread Chapman Flack
On 08/30/18 15:35, Robert Haas wrote: > On Tue, Aug 28, 2018 at 6:30 AM, Peter Eisentraut > wrote: >> CALL compatible with the SQL standard. For example, if you have a >> function f1(IN a int, OUT b int), you would call it as SELECT f1(x) >> and the "b" would somehow be the return value. But a p

Re: pg_verify_checksums and -fno-strict-aliasing

2018-08-30 Thread Tom Lane
Michael Paquier writes: > On Thu, Aug 30, 2018 at 10:39:26AM -0400, Tom Lane wrote: >> (The right fix, of course, is to malloc the work buffer rather than >> put it on the stack.) > pg_upgrade/file.c is careful about that (5afcd2a), and has a comment on > the matter, as does pg_standby.c. > Now,

Re: some pg_dump query code simplification

2018-08-30 Thread Stephen Frost
Greetings, * Andrew Dunstan (andrew.duns...@2ndquadrant.com) wrote: > On 08/28/2018 06:05 PM, Tom Lane wrote: > >Dunno about the idea of running the pg_dump TAP tests against back > >branches. I find that code sufficiently unreadable that maintaining > >several more copies of it doesn't sound lik

Re: some pg_dump query code simplification

2018-08-30 Thread Stephen Frost
Greetings, * Andrew Dunstan (andrew.duns...@2ndquadrant.com) wrote: > On 08/28/2018 06:10 PM, Stephen Frost wrote: > >>Andrew has a buildfarm module that does precisely that, although > >>I'm not sure what its test dataset is --- probably the regression > >>database from each branch. I also have

psql \dC incorrectly shows casts "with inout" as "binary coercible" on 9.5.14 and 11beta3

2018-08-30 Thread jean.pierre.pelletier0
To reproduce, compare the output of \dC on two built-in casts(json to jsonb) and (xml to text) where only the the first is really "with inout". I've been using the folllowing query which (I believe) correctly shows the (json to jsonb) cast as "with inout" SELECT    CONCAT('CREATE CAST (',

Re: pg_verify_checksums and -fno-strict-aliasing

2018-08-30 Thread Andres Freund
On 2018-08-30 19:02:15 -0400, Tom Lane wrote: > Andres Freund writes: > > It certainly should be warned about. Practically I don't think it's a > > problem, because we pretty much always operate on a copy of the page > > when writing out, as otherwise concurrently set hint bits would be > > troub

Re: pg_verify_checksums and -fno-strict-aliasing

2018-08-30 Thread Tom Lane
Andres Freund writes: > On 2018-08-30 18:11:40 -0400, Tom Lane wrote: >> I suspect people will complain about the added cost of doing that. > I think the compiler will just optimize it away. Maybe. In any case, the attached version avoids any need for memcpy and is, I think, cleaner code anyhow

Re: pg_verify_checksums and -fno-strict-aliasing

2018-08-30 Thread Michael Paquier
On Thu, Aug 30, 2018 at 10:07:38PM +0200, Magnus Hagander wrote: > I wonder if your tests that pg_control has picked things up belong more in > the tests of initdb itself? For the case where checksums are disabled, moving there the check on control data makes sense. > Do you think there is value

Re: pg_verify_checksums and -fno-strict-aliasing

2018-08-30 Thread Andres Freund
On 2018-08-30 18:11:40 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2018-08-30 14:46:06 -0700, Andres Freund wrote: > >> One way to fix it would be to memcpy in/out the modified PageHeader, or > >> just do offset math and memcpy to that offset. > > > It took me a bit to reproduce the iss

Re: B-tree cache prefetches

2018-08-30 Thread Peter Geoghegan
On Thu, Aug 30, 2018 at 2:40 PM, Thomas Munro wrote: > A related topic is the cache-unfriendliness of traditional binary > searches of sorted data. Check out "Array Layouts for > Comparison-Based Searching"[1] if you haven't already. It says that > if your array fits in L2 cache, as our btree pa

Re: pg_verify_checksums and -fno-strict-aliasing

2018-08-30 Thread Tom Lane
Andres Freund writes: > On 2018-08-30 14:46:06 -0700, Andres Freund wrote: >> One way to fix it would be to memcpy in/out the modified PageHeader, or >> just do offset math and memcpy to that offset. > It took me a bit to reproduce the issue (due to sheer stupidity on my > part: no, changing the

Re: pg_verify_checksums and -fno-strict-aliasing

2018-08-30 Thread Andres Freund
On 2018-08-30 14:46:06 -0700, Andres Freund wrote: > Hi, > > On 2018-08-30 17:19:28 -0400, Tom Lane wrote: > > So, I've been fooling around trying to get it to work without > > -fno-strict-aliasing, but with little luck so far. > > The problem presumably is that pg_checksum_block() accesses the r

Re: 10.5 but not 10.4: backend startup during reindex system: could not read block 0 in file "base/16400/..": read only 0 of 8192 bytes

2018-08-30 Thread Tom Lane
Justin Pryzby writes: > Just curious, is there really any difficulty in reproducing this? Once you have the right test case, it's not hard. But it took us two months to find one ... regards, tom lane

Re: 10.5 but not 10.4: backend startup during reindex system: could not read block 0 in file "base/16400/..": read only 0 of 8192 bytes

2018-08-30 Thread Justin Pryzby
On Thu, Aug 30, 2018 at 05:30:30PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > On Wed, Aug 29, 2018 at 11:35:51AM -0400, Tom Lane wrote: > >> As far as we can tell, that bug is a dozen years old, so it's not clear > >> why you find that you can reproduce it only in 10.5. But there might be

Re: Use C99 designated initializers for some structs

2018-08-30 Thread Tom Lane
Mark Dilger writes: > I tried doing this perhaps a year ago, and there are a few files with arrays > of structs whose representations get much larger when you change the format > in this way. For instance, in guc.c: > ... > What should the general rule be for initializing arrays of structs such a

Re: pg_verify_checksums and -fno-strict-aliasing

2018-08-30 Thread Tom Lane
Magnus Hagander writes: > On Thu, Aug 30, 2018 at 4:39 PM, Tom Lane wrote: >> (The right fix, of course, is to malloc the work buffer rather than >> put it on the stack.) > So if I get you right, you're saying the attached patch should be all > that's needed? Well, that's some of what's needed

Re: pg_verify_checksums and -fno-strict-aliasing

2018-08-30 Thread Andres Freund
Hi, On 2018-08-30 17:19:28 -0400, Tom Lane wrote: > So, I've been fooling around trying to get it to work without > -fno-strict-aliasing, but with little luck so far. The problem presumably is that pg_checksum_block() accesses the relevant fields as an uint32, whereas pg_checksum_page() accesses

Re: B-tree cache prefetches

2018-08-30 Thread Thomas Munro
On Fri, Aug 31, 2018 at 5:53 AM Andrey Borodin wrote: > Hi hackers! > > I've been at the database conference and here everyone is talking about cache > prefetches. > > I've tried simple hack > > diff --git a/src/backend/access/nbtree/nbtsearch.c > b/src/backend/access/nbtree/nbtsearch.c > index

Re: Startup cost of sequential scan

2018-08-30 Thread Tom Lane
Robert Haas writes: > Whose mental model? I guess the Tom Lane mind is the canonical one > for this project, but I'm not sure that it entirely agrees with mine. Since the fact that we have a notion of startup cost at all is entirely my fault, I don't feel shy about claiming to have the authorita

Re: 10.5 but not 10.4: backend startup during reindex system: could not read block 0 in file "base/16400/..": read only 0 of 8192 bytes

2018-08-30 Thread Tom Lane
Justin Pryzby writes: > On Wed, Aug 29, 2018 at 11:35:51AM -0400, Tom Lane wrote: >> As far as we can tell, that bug is a dozen years old, so it's not clear >> why you find that you can reproduce it only in 10.5. But there might be >> some subtle timing change accounting for that. > It seems to

Re: pg_verify_checksums and -fno-strict-aliasing

2018-08-30 Thread Tom Lane
Michael Banck writes: > Could well be I'm doing something wrong, so it would be cool if somebody > could reproduce this first. In principle, it should be enough to run > 'make clean && make CLFAGS=-O2' in the src/bin/pg_verify_checksums > subdirectory in order to get a broken executable. I can re

Re: pg_verify_checksums and -fno-strict-aliasing

2018-08-30 Thread Michael Banck
Hi, Am Donnerstag, den 30.08.2018, 21:35 +0200 schrieb Magnus Hagander: > So if I get you  right, you're saying the attached patch should be all > that's needed?  I tried to do some similar changes but neither what you proposed nor what I came up with actually fixes the checksum failures, though

Re: pg_verify_checksums and -fno-strict-aliasing

2018-08-30 Thread Michael Banck
Hi, Am Donnerstag, den 30.08.2018, 22:02 +0200 schrieb Magnus Hagander: > PFA some *very* basic tests for pg_verify_checksums, which should at > least be enough to catch the kind of errors we had now in the tool > itself. I proposed something similar for pg_basebackup back then and IIRC Peter (ri

Re: 10.5 but not 10.4: backend startup during reindex system: could not read block 0 in file "base/16400/..": read only 0 of 8192 bytes

2018-08-30 Thread Andres Freund
Hi, On 2018-08-30 15:00:28 -0500, Justin Pryzby wrote: > On Wed, Aug 29, 2018 at 11:35:51AM -0400, Tom Lane wrote: > > This looks suspiciously like the issue under discussion in > > > > https://www.postgresql.org/message-id/12259.1532117714%40sss.pgh.pa.us > > > > As far as we can tell, that bug

Re: Stored procedures and out parameters

2018-08-30 Thread Dave Cramer
> > > In other words, being more like the SQL standard is probably good, but > breaking compatibility is bad. You've technically avoided a > *backward* compatibility break by deciding that functions and > procedures can work differently from each other, but that just moves > the problem around. N

Re: Use C99 designated initializers for some structs

2018-08-30 Thread Andres Freund
Hi, On 2018-08-30 13:54:41 -0300, Alvaro Herrera wrote: > On 2018-Aug-30, Mark Dilger wrote: > > > static struct config_bool ConfigureNamesBool[] = > > { > > { > > {"enable_seqscan", PGC_USERSET, QUERY_TUNING_METHOD, > > gettext_noop("Enables the planner's use of sequential

Re: pg_verify_checksums and -fno-strict-aliasing

2018-08-30 Thread Andres Freund
On 2018-08-30 10:39:26 -0400, Tom Lane wrote: > charbuf[BLCKSZ]; > PageHeader header = (PageHeader) buf; > (The right fix, of course, is to malloc the work buffer rather than > put it on the stack.) Or alternatively, for places where such allocations could be a proble

Re: pg_verify_checksums and -fno-strict-aliasing

2018-08-30 Thread Magnus Hagander
On Thu, Aug 30, 2018 at 10:02 PM, Michael Paquier wrote: > On Thu, Aug 30, 2018 at 09:35:33PM +0200, Magnus Hagander wrote: > > Should we make it a separate test in pg_verify_checksums, or should we > > piggyback on the pg_basebackup tests (which AFAICT is the only ones that > > create a cluster

Re: pg_verify_checksums and -fno-strict-aliasing

2018-08-30 Thread Michael Paquier
On Thu, Aug 30, 2018 at 09:35:33PM +0200, Magnus Hagander wrote: > Should we make it a separate test in pg_verify_checksums, or should we > piggyback on the pg_basebackup tests (which AFAICT is the only ones that > create a cluster with checksums enabled at all, and thus is the only > codepath that

Re: pg_verify_checksums and -fno-strict-aliasing

2018-08-30 Thread Magnus Hagander
On Thu, Aug 30, 2018 at 9:35 PM, Magnus Hagander wrote: > On Thu, Aug 30, 2018 at 4:39 PM, Tom Lane wrote: > >> Fabien COELHO writes: >> >> If I add -fno-strict-aliasing to $CFLAGS, the problem goes away. >> >> Is this something to worry about, or just pilot error cause I am not >> >> using the

Re: 10.5 but not 10.4: backend startup during reindex system: could not read block 0 in file "base/16400/..": read only 0 of 8192 bytes

2018-08-30 Thread Justin Pryzby
On Wed, Aug 29, 2018 at 11:35:51AM -0400, Tom Lane wrote: > Justin Pryzby writes: > > I've seen this message now a handful of times recently. It seems to happen > > overnight, during a maintenance job which reindex things, including system > > catalog indices. > > It's easy to reproduce error und

Re: Use C99 designated initializers for some structs

2018-08-30 Thread Robert Haas
On Wed, Aug 29, 2018 at 6:51 PM, Tom Lane wrote: > I agree that assuming that they're physically zeroes is OK from a > portability standpoint, because we'd have a whole lot of other issues > if they weren't. But I have a different point to make, which is that > it's fairly standard practice for u

Re: Startup cost of sequential scan

2018-08-30 Thread Robert Haas
On Thu, Aug 30, 2018 at 10:04 AM, Tom Lane wrote: > Alexander Korotkov writes: >> But I think there is another issue in sequential scan cost. We have >> zero startup cost for sequential scan. But why? > > Because it's what the mental model of startup cost says it should be. Whose mental model?

Re: Extra word in src/backend/optimizer/README

2018-08-30 Thread Magnus Hagander
On Thu, Aug 30, 2018 at 1:27 PM, Etsuro Fujita wrote: > Hi, > > Here is a small patch to remove $SUBJECT: s/has contains/contains/ > Definitely looks correct. A good first test to verify your own commit/push privileges, perhaps? -- Magnus Hagander Me: https://www.hagander.net/

Re: Stored procedures and out parameters

2018-08-30 Thread Robert Haas
On Tue, Aug 28, 2018 at 6:30 AM, Peter Eisentraut wrote: > Output parameter handling is not compatible between function calls and > procedure calls. Our implementation of output parameters in functions > is an extension of the SQL standard, and while it's been useful, it's > nonstandard, and I wo

Re: pg_verify_checksums and -fno-strict-aliasing

2018-08-30 Thread Magnus Hagander
On Thu, Aug 30, 2018 at 4:39 PM, Tom Lane wrote: > Fabien COELHO writes: > >> If I add -fno-strict-aliasing to $CFLAGS, the problem goes away. > >> Is this something to worry about, or just pilot error cause I am not > >> using the same $CFLAGS as for the rest of the build? I originally > >> not

Re: Proposal for disk quota feature

2018-08-30 Thread Pavel Stehule
2018-08-30 16:22 GMT+02:00 Chapman Flack : > On 08/30/2018 09:57 AM, Hubert Zhang wrote: > > > 2 Keep one worker process for each database. But using a parent/global > > quota worker process to manage the lifecycle of database level worker > > processes. It could handle the newly created database(

Re: Online verification of checksums

2018-08-30 Thread Magnus Hagander
On Thu, Aug 30, 2018 at 8:06 PM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 26/07/2018 13:59, Michael Banck wrote: > > I've now forward-ported this change to pg_verify_checksums, in order to > > make this application useful for online clusters, see attached patch. > > Why not

Re: BUG #15346: Replica fails to start after the crash

2018-08-30 Thread Michael Paquier
On Thu, Aug 30, 2018 at 08:31:36PM +0200, Alexander Kukushkin wrote: > 2018-08-30 19:34 GMT+02:00 Michael Paquier : >> I have been struggling for a couple of hours to get a deterministic test >> case out of my pocket, and I did not get one as you would need to get >> the bgwriter to flush a page be

Re: BUG #15346: Replica fails to start after the crash

2018-08-30 Thread Alexander Kukushkin
2018-08-30 19:34 GMT+02:00 Michael Paquier : > I have been struggling for a couple of hours to get a deterministic test > case out of my pocket, and I did not get one as you would need to get > the bgwriter to flush a page before crash recovery finishes, we could do In my case the active standby s

Re: Bug in slot.c and are replication slots ever used at Window?

2018-08-30 Thread Michael Paquier
On Thu, Aug 30, 2018 at 11:00:43AM +0300, Konstantin Knizhnik wrote: > So if "isdir" is true (and it is true in this case), it sets O_RDONLY > flag. Then fsync_fname successfully opens slot file in readonly mode > and calls fsync() which at windows is substituted with _commit() which > in turn is

Re: Online verification of checksums

2018-08-30 Thread Peter Eisentraut
On 26/07/2018 13:59, Michael Banck wrote: > I've now forward-ported this change to pg_verify_checksums, in order to > make this application useful for online clusters, see attached patch. Why not provide this functionality as a server function or command. Then you can access blocks with proper loc

Re: B-tree cache prefetches

2018-08-30 Thread Peter Geoghegan
On Thu, Aug 30, 2018 at 10:53 AM, Andrey Borodin wrote: > The idea is pretty simple - our search are cache erasing anyway, let's try to > get at least some of it by prefetching possible ways of binary search. > And it seems to me that on a simple query >> insert into x select (random()*100)::

Re: Dimension limit in contrib/cube (dump/restore hazard?)

2018-08-30 Thread Michael Paquier
On Thu, Aug 30, 2018 at 02:28:20PM +0300, Alexander Korotkov wrote: > In general looks good for me. Personally I get tired with cube.out > and cube_2.out. They are different with only few checks involving > scientific notation. But all the patches touching cube regression > tests should update b

Re: pg_verify_checksums and -fno-strict-aliasing

2018-08-30 Thread Michael Paquier
On Thu, Aug 30, 2018 at 10:39:26AM -0400, Tom Lane wrote: > I rather suspect that this hasn't been tested on anything but Intel > hardware, which is famously misalignment-tolerant. The lack of any > apparent regression test infrastructure for it isn't leaving a warm > feeling about how much the bu

B-tree cache prefetches

2018-08-30 Thread Andrey Borodin
Hi hackers! I've been at the database conference and here everyone is talking about cache prefetches. I've tried simple hack diff --git a/src/backend/access/nbtree/nbtsearch.c b/src/backend/access/nbtree/nbtsearch.c index d3700bd082..ffddf553aa 100644 --- a/src/backend/access/nbtree/nbtsearch.

Re: [HACKERS] Proposal to add work_mem option to postgres_fdw module

2018-08-30 Thread Peter Eisentraut
On 28/08/2018 05:55, Shinoda, Noriyoshi (PN Japan GCS Delivery) wrote: >>> I like the direction of your thinking, but it seems to me that this >>> would cause a problem if you want to set search_path=foo,bar. >> ... OPTIONS ( host 'remhost1', port '5433', dbname 'demodb', >> option='option1=foo',

Re: BUG #15346: Replica fails to start after the crash

2018-08-30 Thread Michael Paquier
On Thu, Aug 30, 2018 at 04:03:43PM +0200, Alexander Kukushkin wrote: > 2018-08-30 15:39 GMT+02:00 Michael Paquier : >> Does it take care of the problem? > > Yep, with the patch applied bgwriter acts as expected! Thanks for double-checking. I have been struggling for a couple of hours to get a de

Re: TupleTableSlot abstraction

2018-08-30 Thread Alvaro Herrera
Man, how I dislike patches in tarballs. 0002 says: + * shouldFree is set 'true' since a tuple stored on a disk page should not be + * pfree'd. Surely you mean 'false' :-) -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training &

Re: Add a semicolon to query related to search_path

2018-08-30 Thread Peter Eisentraut
On 17/08/2018 05:32, Tatsuro Yamada wrote: > Hi Robert, > > On 2018/08/17 4:32, Robert Haas wrote: >> On Thu, Aug 16, 2018 at 1:20 AM, Tatsuro Yamada >> wrote: >>> As you can see, queries with and without a semicolon are mixed, it is hard >>> to understand the end of each query. This is not beaut

Re: Use C99 designated initializers for some structs

2018-08-30 Thread Alvaro Herrera
On 2018-Aug-30, Mark Dilger wrote: > static struct config_bool ConfigureNamesBool[] = > { > { > {"enable_seqscan", PGC_USERSET, QUERY_TUNING_METHOD, > gettext_noop("Enables the planner's use of sequential-scan > plans."), > NULL > }, > &enable_s

Re: [PATCH] Tab completion for ALTER DATABASE … SET TABLESPACE

2018-08-30 Thread Dagfinn Ilmari Mannsåker
ilm...@ilmari.org (Dagfinn Ilmari Mannsåker) writes: > Hi hackers, > > I just noticed that psql's tab completion for ALTER TABLE … SET > TABLESPACE was treating it as any other configuration parameter and > completing with FROM DEFAULT or TO after it, instead of a list of > tablespaces. And just

Re: Hint to set owner for tablespace directory

2018-08-30 Thread Peter Eisentraut
On 23/08/2018 13:24, Maksim Milyutin wrote: > I have noticed the novice users are stuck trying to create tablespace > over a directory whose owner is not the system postgres user. They > observed the message "could not set permissions on directory ...: > permission denied". > > I want to add pa

[PATCH] Tab completion for ALTER DATABASE … SET TABLESPACE

2018-08-30 Thread Dagfinn Ilmari Mannsåker
Hi hackers, I just noticed that psql's tab completion for ALTER TABLE … SET TABLESPACE was treating it as any other configuration parameter and completing with FROM DEFAULT or TO after it, instead of a list of tablespaces. PFA a patch that fixes this. - ilmari -- "A disappointingly low fraction

Re: "Write amplification" is made worse by "getting tired" while inserting into nbtree secondary indexes (Was: Why B-Tree suffix truncation matters)

2018-08-30 Thread Peter Geoghegan
On Wed, Aug 29, 2018 at 11:28 PM, Simon Riggs wrote: > If you include heap TID as a column the suffix will be unique and cannot > benefit from suffix truncation. Right. During a page split, we must generate a new high key that's less than or equal to all items on the left side (where the new high

Re: Dimension limit in contrib/cube (dump/restore hazard?)

2018-08-30 Thread Alexander Korotkov
On Thu, Aug 30, 2018 at 4:59 PM Tom Lane wrote: > Alexander Korotkov writes: > > I'm going to check this patchset on Windows and commit if no objections. > > These error messages do not conform to our message style guidelines: > you've copied an errdetail message as primary error message, but the

Re: Use C99 designated initializers for some structs

2018-08-30 Thread Mark Dilger
> On Aug 29, 2018, at 1:51 PM, David Steele wrote: > > On 8/29/18 5:14 AM, Peter Eisentraut wrote: >> On 29/08/2018 12:13, Peter Eisentraut wrote: >>> Here is a patch to change some struct initializations to use C99-style >>> designated initializers. These are just a few particularly egregiou

Re: Startup cost of sequential scan

2018-08-30 Thread Tom Lane
Alexander Korotkov writes: > I understand that startup cost is not "time to find the first row". > But I think this example highlight not one but two issues. > 1) Row count estimates for joins are wrong. Yup. > 2) Rows are assumed to be continuous while in reality they are > discrete. Where do

Re: Startup cost of sequential scan

2018-08-30 Thread Alexander Korotkov
On Thu, Aug 30, 2018 at 5:58 PM Tom Lane wrote: > > Alexander Korotkov writes: > > On Thu, Aug 30, 2018 at 5:05 PM Tom Lane wrote: > >> Because it's what the mental model of startup cost says it should be. > > > From this model we make a conclusion that we're starting getting rows > > from seque

Re: Startup cost of sequential scan

2018-08-30 Thread Tom Lane
Andrew Gierth writes: > The model (assuming I understand it rightly) is that what we're actually > tracking is a startup cost and a per-output-row cost, but for comparison > purposes we actually store the rows and the computed total, rather than > just the per-row cost: > rows > startup_cost > to

Re: Startup cost of sequential scan

2018-08-30 Thread Andrew Gierth
> "Konstantin" == Konstantin Knizhnik writes: >> No, startup cost is not the "time to find the first row". It's >> overhead paid before you even get to start examining rows. Konstantin> But it seems to me that calculation of cost in LIMIT node Konstantin> contradicts with this statement:

Re: Startup cost of sequential scan

2018-08-30 Thread Alexander Korotkov
On Thu, Aug 30, 2018 at 6:08 PM Konstantin Knizhnik wrote: > On 30.08.2018 17:58, Tom Lane wrote: > > Alexander Korotkov writes: > >> On Thu, Aug 30, 2018 at 5:05 PM Tom Lane wrote: > >>> Because it's what the mental model of startup cost says it should be. > >> From this model we make a conclu

Re: Startup cost of sequential scan

2018-08-30 Thread Konstantin Knizhnik
On 30.08.2018 17:58, Tom Lane wrote: Alexander Korotkov writes: On Thu, Aug 30, 2018 at 5:05 PM Tom Lane wrote: Because it's what the mental model of startup cost says it should be. From this model we make a conclusion that we're starting getting rows from sequential scan sooner than fro

Re: Startup cost of sequential scan

2018-08-30 Thread Tom Lane
Alexander Korotkov writes: > On Thu, Aug 30, 2018 at 5:05 PM Tom Lane wrote: >> Because it's what the mental model of startup cost says it should be. > From this model we make a conclusion that we're starting getting rows > from sequential scan sooner than from index scan. And this conclusion >

Re: rare crash - FailedAssertion snapbuild.c Line: 580

2018-08-30 Thread Erik Rijkers
On 2018-08-30 16:44, Alvaro Herrera wrote: On 2018-Aug-30, Erik Rijkers wrote: ok, is this any use? Seems mostly good, but the Xids are not printed. Could you please do "bt full"? Also: frame 3 print *snap See the attached. # gdb --quiet -ex 'bt full' --batch /var/data1/pg_stuff/pg_

Re: pg_verify_checksums and -fno-strict-aliasing

2018-08-30 Thread Alvaro Herrera
On 2018-Aug-30, Fabien COELHO wrote: > As PostgreSQL source is expected to conform to some C standard (unsure which > one right now, possibly c89 but maybe it is beginning to switch to c99, a > young 19 years old standard), I'd suggest that the right fix is rather to > actually remove the aliasing

Re: rare crash - FailedAssertion snapbuild.c Line: 580

2018-08-30 Thread Alvaro Herrera
On 2018-Aug-30, Erik Rijkers wrote: > ok, is this any use? Seems mostly good, but the Xids are not printed. Could you please do "bt full"? Also: frame 3 print *snap Thanks, -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Traini

Re: pg_verify_checksums and -fno-strict-aliasing

2018-08-30 Thread Tom Lane
Fabien COELHO writes: >> If I add -fno-strict-aliasing to $CFLAGS, the problem goes away. >> Is this something to worry about, or just pilot error cause I am not >> using the same $CFLAGS as for the rest of the build? I originally >> noticed this problem with my external fork of pg_verify_checksum

Re: Startup cost of sequential scan

2018-08-30 Thread Alexander Korotkov
On Thu, Aug 30, 2018 at 5:05 PM Tom Lane wrote: > Alexander Korotkov writes: > > But I think there is another issue in sequential scan cost. We have > > zero startup cost for sequential scan. But why? > > Because it's what the mental model of startup cost says it should be. Right. So as I und

Re: Proposal for disk quota feature

2018-08-30 Thread Chapman Flack
On 08/30/2018 09:57 AM, Hubert Zhang wrote: > 2 Keep one worker process for each database. But using a parent/global > quota worker process to manage the lifecycle of database level worker > processes. It could handle the newly created database(avoid restart > database) and save resource when a da

Re: Startup cost of sequential scan

2018-08-30 Thread Tom Lane
Alexander Korotkov writes: > But I think there is another issue in sequential scan cost. We have > zero startup cost for sequential scan. But why? Because it's what the mental model of startup cost says it should be. Also, I do not think we can change that without a whole lot of unpleasant side

Re: BUG #15346: Replica fails to start after the crash

2018-08-30 Thread Alexander Kukushkin
Hi, 2018-08-30 15:39 GMT+02:00 Michael Paquier : > That's indeed obvious by reading the code. The bgwriter would be > started only once a consistent point has been reached, so the startup > process would have normally already updated the control file to the > consistent point. Something like th

Re: Dimension limit in contrib/cube (dump/restore hazard?)

2018-08-30 Thread Tom Lane
Alexander Korotkov writes: > I'm going to check this patchset on Windows and commit if no objections. These error messages do not conform to our message style guidelines: you've copied an errdetail message as primary error message, but the rules are different for that (no complete sentences, no i

Proposal for disk quota feature

2018-08-30 Thread Hubert Zhang
Hi all, We want to introduce disk quota feature into Postgres. *Why disk quota* *In a multi-tenant environment, there is a requirement to limit the disk quota that database/schema/table can be written or a user can consume for different organizations.* *Meanwhile, other databases such as Oracle, T

Startup cost of sequential scan

2018-08-30 Thread Alexander Korotkov
Hi! Our customer have a bad plan problem, which could be reduced to the following example. create table t1 (id int primary key, k int); create table t2 (id int); insert into t1 (select i, i from generate_series(1,100) i); insert into t2 (select 0 from generate_series(1,100)i); insert into t2

Re: BUG #15346: Replica fails to start after the crash

2018-08-30 Thread Michael Paquier
On Thu, Aug 30, 2018 at 10:55:23AM +0200, Alexander Kukushkin wrote: > Bgwriter itself never changes updateMinRecoveryPoint to true and boom, > we can get a lot of pages written to disk, but minRecoveryPoint in the > pg_control won't be updated! That's indeed obvious by reading the code. The bgwr

Re: Problem while updating a foreign table pointing to a partitioned table on foreign server

2018-08-30 Thread Etsuro Fujita
(2018/08/30 20:37), Kyotaro HORIGUCHI wrote: At Fri, 24 Aug 2018 21:45:35 +0900, Etsuro Fujita wrote in<5b7ffdef.6020...@lab.ntt.co.jp> (2018/08/21 11:01), Kyotaro HORIGUCHI wrote: At Tue, 14 Aug 2018 20:49:02 +0900, Etsuro Fujita wrote in<5b72c1ae.8010...@lab.ntt.co.jp> (2018/08/09 22:04),

Re: [HACKERS] [PATCH] kNN for SP-GiST

2018-08-30 Thread Alexander Korotkov
On Thu, Aug 30, 2018 at 12:41 PM Alexander Korotkov wrote: > Right, performance regression appears to be caused by queue memory > context allocation. I've tried to apply the same approach that we've > in GiST: allocate separate memory context for queue only at second > rescan call. And it appear

Re: rare crash - FailedAssertion snapbuild.c Line: 580

2018-08-30 Thread Erik Rijkers
On 2018-08-29 21:15, Andres Freund wrote: Hi, On 2018-08-29 17:43:17 +0200, Erik Rijkers wrote: To test postgres 11, I still regularly run series of short sessions of pgbench-over-logical-replication (basically the same thing that I used last year [1] - now in a perl incarnation). Most of the

Re: A strange GiST error message or fillfactor of GiST build

2018-08-30 Thread Andrey Borodin
Hello! > 30 авг. 2018 г., в 2:42, Kyotaro HORIGUCHI > написал(а): > > At Wed, 29 Aug 2018 10:42:59 -0300, Andrey Borodin > wrote in <6fbe12b2-4f59-4db9-bde9-62c880118...@yandex-team.ru> >> >> We are passing freespace everywhere. Also, we pass GistInsertState, and >> GistState. >> Maybe let'

Re: Catalog corruption

2018-08-30 Thread Andrew Gierth
> "Mariel" == Mariel Cherkassky writes: Mariel> Hi Andrew, Mariel> what is the name of the channel ? The name of the channel is #postgresql (including the # character) -- Andrew (irc:RhodiumToad)

Re: Reopen logfile on SIGHUP

2018-08-30 Thread Kyotaro HORIGUCHI
Hello. At Thu, 30 Aug 2018 13:42:42 +0300, Alexander Korotkov wrote in > It seems that http://commitfest.cputube.org/ runs only "make check" on > Windows. But my Postgres Pro colleagues checked that tests passed on > 32-bit and 64-bit versions of Windows Server 2008. Also I made some > minor

Re: Catalog corruption

2018-08-30 Thread Mariel Cherkassky
Hi Andrew, what is the name of the channel ? Thanks , Mariel. ‫בתאריך יום ד׳, 29 באוג׳ 2018 ב-14:31 מאת ‪Andrew Gierth‬‏ <‪ and...@tao11.riddles.org.uk‬‏>:‬ > > "Mariel" == Mariel Cherkassky writes: > > Mariel> Hi, > > Mariel> I sent already an email about this topic to pgsql-admins but I

Re: Problem while updating a foreign table pointing to a partitioned table on foreign server

2018-08-30 Thread Kyotaro HORIGUCHI
Hello. At Fri, 24 Aug 2018 21:45:35 +0900, Etsuro Fujita wrote in <5b7ffdef.6020...@lab.ntt.co.jp> > (2018/08/21 11:01), Kyotaro HORIGUCHI wrote: > > At Tue, 14 Aug 2018 20:49:02 +0900, Etsuro > > Fujita wrote > > in<5b72c1ae.8010...@lab.ntt.co.jp> > >> (2018/08/09 22:04), Etsuro Fujita wrote: >

Re: pg_verify_checksums vs windows

2018-08-30 Thread Magnus Hagander
On Thu, Aug 30, 2018 at 1:32 PM, Amit Kapila wrote: > On Wed, Aug 29, 2018 at 5:17 PM Magnus Hagander > wrote: > > > > On Wed, Aug 29, 2018 at 1:44 PM, Amit Kapila > wrote: > >> > >> On Wed, Aug 29, 2018 at 5:05 PM Magnus Hagander > wrote: > >> > > >> > On Wed, Aug 29, 2018 at 1:31 PM, Amit Ka

Re: pg_verify_checksums vs windows

2018-08-30 Thread Amit Kapila
On Wed, Aug 29, 2018 at 5:17 PM Magnus Hagander wrote: > > On Wed, Aug 29, 2018 at 1:44 PM, Amit Kapila wrote: >> >> On Wed, Aug 29, 2018 at 5:05 PM Magnus Hagander wrote: >> > >> > On Wed, Aug 29, 2018 at 1:31 PM, Amit Kapila >> > wrote: >> >> >> >> So, I think we need to open the file in bin

Re: Dimension limit in contrib/cube (dump/restore hazard?)

2018-08-30 Thread Alexander Korotkov
Hi! On Tue, Aug 28, 2018 at 10:30 PM Andrey Borodin wrote: > > 28 авг. 2018 г., в 14:18, Alexander Korotkov > > написал(а): > > > > OK, but I think cube_c_f8() and cube_c_f8_f8() also need to be > > revised. Also, I think this behavior should be covered by regression > > tests. > True. Also th

Re: pg_verify_checksums failure with hash indexes

2018-08-30 Thread Amit Kapila
On Wed, Aug 29, 2018 at 4:05 PM Dilip Kumar wrote: > > On Wed, Aug 29, 2018 at 3:39 PM, Dilip Kumar wrote: > >> SHOW block_size ; > >> block_size > >> > >> 4096 > >> > >> CREATE TABLE foo(val text); > >> INSERT INTO foo VALUES('bernd'); > >> > >> CREATE INDEX ON foo USING hash(val)

Extra word in src/backend/optimizer/README

2018-08-30 Thread Etsuro Fujita
Hi, Here is a small patch to remove $SUBJECT: s/has contains/contains/ Best regards, Etsuro Fujita diff --git a/src/backend/optimizer/README b/src/backend/optimizer/README index 0db3d36..9c852a1 100644 --- a/src/backend/optimizer/README +++ b/src/backend/optimizer/README @@ -1109,7 +1109,7 @@ of

Re: pg_verify_checksums and -fno-strict-aliasing

2018-08-30 Thread Fabien COELHO
I noticed that pg_verify_checksums computes bogus checksums if I compile it with '-O2 -Wall' but without -fno-strict-aliasing. Also I am getting a compile warning then: [...] If I add -fno-strict-aliasing to $CFLAGS, the problem goes away. Is this something to worry about, or just pilot erro

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-30 Thread Etsuro Fujita
(2018/08/29 18:40), Etsuro Fujita wrote: (2018/08/29 0:21), Jonathan S. Katz wrote: On Aug 24, 2018, at 8:38 AM, Etsuro Fujita wrote: (2018/08/24 11:47), Michael Paquier wrote: On Thu, Aug 23, 2018 at 10:00:49PM +0900, Etsuro Fujita wrote: I tried this today, but doing git behind the corporate

  1   2   >