Re: finding changed blocks using WAL scanning

2019-04-22 Thread Tomas Vondra
On Mon, Apr 22, 2019 at 07:44:45PM -0400, Bruce Momjian wrote: On Tue, Apr 23, 2019 at 01:21:27AM +0200, Tomas Vondra wrote: On Sat, Apr 20, 2019 at 04:21:52PM -0400, Robert Haas wrote: > On Sat, Apr 20, 2019 at 12:42 AM Stephen Frost wrote: > > > Oh. Well, I already explained my algorithm for

Re: memory leak checking

2019-04-22 Thread Tom Lane
Andres Freund writes: > On 2019-04-22 16:50:25 -0700, Mikhail Bautin wrote: >> What is the standard memory leak checking policy for the PostgreSQL >> codebase? I know there is some support for valgrind -- is the test suite >> being run continuously with valgrind on the build farm? > Leaks are all

Re: finding changed blocks using WAL scanning

2019-04-22 Thread Bruce Momjian
On Tue, Apr 23, 2019 at 02:13:29AM +0200, Tomas Vondra wrote: > Well, I understand that concern - all I'm saying is that makes this > useless for some use cases (that may or may not be important enough). > > However, it seems to me those files are guaranteed to be much smaller > than the WAL segme

[PATCH v1] Show whether tables are logged in \dt+

2019-04-22 Thread David Fetter
Folks, I noticed that there wasn't a bulk way to see table logged-ness in psql, so I made it part of \dt+. What say? Best, David. -- David Fetter http://fetter.org/ Phone: +1 415 235 3778 Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate >From 518efc00373

Re: bug in update tuple routing with foreign partitions

2019-04-22 Thread Amit Langote
Fujita-san, On 2019/04/22 20:00, Etsuro Fujita wrote: > (2019/04/19 14:39), Etsuro Fujita wrote: >> (2019/04/19 13:00), Amit Langote wrote: >>> I agree that we can move to fix the other issue right away as the fix you >>> outlined above seems reasonable, but I wonder if it wouldn't be better to >>

Re: finding changed blocks using WAL scanning

2019-04-22 Thread Bruce Momjian
On Mon, Apr 22, 2019 at 08:52:11PM -0400, Bruce Momjian wrote: > Well, the interesting question is whether the server will generate a > single modblock file for all WAL in pg_wal only right before we are > ready to expire some WAL, or whether modblock files will be generated > offline, perhaps inde

Re: memory leak checking

2019-04-22 Thread Andres Freund
Hi, On 2019-04-22 20:29:17 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2019-04-22 16:50:25 -0700, Mikhail Bautin wrote: > >> What is the standard memory leak checking policy for the PostgreSQL > >> codebase? I know there is some support for valgrind -- is the test suite > >> being run c

Re: memory leak checking

2019-04-22 Thread Tom Lane
Andres Freund writes: > On 2019-04-22 20:29:17 -0400, Tom Lane wrote: >> I would not call the timezone data a "leak", since it's still useful, and >> accessible from static pointers, right up to exit. A true leak for this >> purpose is memory that's allocated but not usefully accessible, and I'd

Re: change password_encryption default to scram-sha-256?

2019-04-22 Thread Jonathan S. Katz
On 4/8/19 6:10 PM, Jonathan S. Katz wrote: > On 4/8/19 4:20 PM, Alvaro Herrera wrote: >> On 2019-Apr-08, Jonathan S. Katz wrote: >> >>> On 4/8/19 4:10 PM, Alvaro Herrera wrote: >> I wonder why we have two pages https://wiki.postgresql.org/wiki/Client_Libraries https://wiki.postgresql

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread Amit Langote
On 2019/04/23 7:51, Alvaro Herrera wrote: > On 2019-Mar-06, Tom Lane wrote: >> David Rowley writes: >>> As far as I can see, the biggest fundamental difference with doing >>> things this way will be that the column order of partitions will be >>> preserved, where before it would inherit the order

Re: finding changed blocks using WAL scanning

2019-04-22 Thread Robert Haas
On Mon, Apr 22, 2019 at 7:04 PM Tomas Vondra wrote: > Some time ago there was a discussion about prefetching blocks during > recovery on a standby, and that's a great example of a use case that > benefit from this - look which blocks where modified in the next chunk > of WAL, prefetch them. But th

Re: Regression test PANICs with master-standby setup on same machine

2019-04-22 Thread Michael Paquier
On Mon, Apr 22, 2019 at 03:52:59PM +0530, Kuntal Ghosh wrote: > I accept that configuring master-standby on the same machine for this > test is not okay. But, can we avoid the PANIC somehow? Or, is this > intentional and I should not include testtablespace in this case? Well, it is a bit more than

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2019-04-22 Thread Michael Paquier
On Mon, Apr 22, 2019 at 09:19:33PM +0900, Kyotaro HORIGUCHI wrote: > The attached exercises this sequence, needing some changes in > PostgresNode.pm and RecursiveCopy.pm to allow tablespaces. +# Check for symlink -- needed only on source dir +# (note: this will fall through quietly if file

Re: clean up docs for v12

2019-04-22 Thread Michael Paquier
On Mon, Apr 22, 2019 at 12:19:55PM -0400, Alvaro Herrera wrote: > On 2019-Apr-22, Andres Freund wrote: >> I think it'd be better just to fix s/the all/that all/. > > (and s/if's/if it's/) FWIW, I have noticed that part when gathering all the pieces for what became 148266f, still the full paragrap

Symbol referencing errors

2019-04-22 Thread Li Japin
Hi, When I compile PostgreSQL-11.2 on SmartOS, I find the following errors: Undefined            first referenced  symbol                  in file per_MultiFuncCall   adminpack.o end_MultiFuncCall   adminpack.o BuildTupleFromCStrings  adminpack.o Decod

Re: Symbol referencing errors

2019-04-22 Thread Tom Lane
Li Japin writes: > When I compile PostgreSQL-11.2 on SmartOS, I find the following errors: > ... > ld: warning: symbol referencing errors Yeah, our SmartOS buildfarm members show those warnings too, eg https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=damselfly&dt=2019-04-22%2010%3A0

Re: speeding up planning with partitions

2019-04-22 Thread Amit Langote
On 2019/04/23 7:08, Tom Lane wrote: > Amit Langote writes: >> On 2019/04/02 2:34, Tom Lane wrote: >>> I'm not at all clear on what we think the interaction between >>> enable_partition_pruning and constraint_exclusion ought to be, >>> so I'm not sure what the appropriate resolution is here. Thoug

Re: Symbol referencing errors

2019-04-22 Thread Li Japin
On 4/23/19 12:09 PM, Tom Lane wrote: > AFAICT they're harmless, so my advice is just ignore them. > > If you're sufficiently annoyed by them to find the cause > and try to fix it, go ahead, but I haven't heard anyone > else worried about it. It might be that SmartOS wants > something like what w

Re: Regression test PANICs with master-standby setup on same machine

2019-04-22 Thread Kyotaro HORIGUCHI
At Tue, 23 Apr 2019 11:27:06 +0900, Michael Paquier wrote in <20190423022706.gg2...@paquier.xyz> > On Mon, Apr 22, 2019 at 03:52:59PM +0530, Kuntal Ghosh wrote: > > I accept that configuring master-standby on the same machine for this > > test is not okay. But, can we avoid the PANIC somehow? Or,

Re: Regression test PANICs with master-standby setup on same machine

2019-04-22 Thread Kyotaro HORIGUCHI
At Tue, 23 Apr 2019 13:33:39 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20190423.19.113770648.horiguchi.kyot...@lab.ntt.co.jp> > At Tue, 23 Apr 2019 11:27:06 +0900, Michael Paquier > wrote in <20190423022706.gg2...@paquier.xyz> > > On Mon, Apr 22, 2019 at 03:52:59PM +0530, Ku

Re: display of variables in EXPLAIN VERBOSE

2019-04-22 Thread Amit Langote
On 2019/04/23 0:58, Tom Lane wrote: > BTW, now that I look at this, I think the reason why I didn't make > tlist printouts pay attention to VERBOSE for this purpose is that > you don't get them at all if not verbose: > > regression=# explain select * from tt; > QUERY PLAN

Re: REINDEX INDEX results in a crash for an index of pg_class since 9.6

2019-04-22 Thread Michael Paquier
On Thu, Apr 18, 2019 at 10:14:30AM +0900, Michael Paquier wrote: > Doing a REINDEX TABLE directly on pg_class proves to work correctly, > and CONCURRENTLY is not supported for catalog tables. > > Bisecting my way through it, the first commit causing the breakage is > that: > commit: 01e386a325549b

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2019-04-22 Thread Kyotaro HORIGUCHI
At Tue, 23 Apr 2019 11:34:38 +0900, Michael Paquier wrote in <20190423023438.gh2...@paquier.xyz> > On Mon, Apr 22, 2019 at 09:19:33PM +0900, Kyotaro HORIGUCHI wrote: > > The attached exercises this sequence, needing some changes in > > PostgresNode.pm and RecursiveCopy.pm to allow tablespaces. >

Re: [PATCH v1] Show whether tables are logged in \dt+

2019-04-22 Thread Fabien COELHO
Hello David, I noticed that there wasn't a bulk way to see table logged-ness in psql, so I made it part of \dt+. Applies, compiles, works for me. ISTM That temporary-ness is not shown either. Maybe the persistence column should be shown as is? Also I'd suggest that the column should be d

Re: Symbol referencing errors

2019-04-22 Thread Andrew Gierth
> "Tom" == Tom Lane writes: >> When I compile PostgreSQL-11.2 on SmartOS, I find the following errors: >> ... >> ld: warning: symbol referencing errors Tom> Yeah, our SmartOS buildfarm members show those warnings too, eg Tom> https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2019-04-22 Thread Paul Guo
Hi Kyotaro, ignoring the MakePGDirectory() failure will fix this database create redo error, but I suspect some other kind of redo, which depends on the files under the directory (they are not copied since the directory is not created) and also cannot be covered by the invalid page mechanism, could

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread David Rowley
On Tue, 23 Apr 2019 at 13:49, Amit Langote wrote: > > On 2019/04/23 7:51, Alvaro Herrera wrote: > > To me, it sounds > > unintuitive to accept partitions that don't exactly match the order of > > the parent table; but it's been supported all along. > > You might know it already, but even though co

Re: Regression test PANICs with master-standby setup on same machine

2019-04-22 Thread Michael Paquier
On Tue, Apr 23, 2019 at 01:33:39PM +0900, Kyotaro HORIGUCHI wrote: > I think this is rahter a testing or debugging feature. This can > be apply to all paths, so the variable might be "path_prefix" or > something more generic than tablespace_chroot. > > Does it make sense? A GUC which enforces obj

Re: Regression test PANICs with master-standby setup on same machine

2019-04-22 Thread Andres Freund
Hi, On 2019-04-22 15:52:59 +0530, Kuntal Ghosh wrote: > Hello hackers, > > With a master-standby setup configured on the same machine, I'm > getting a panic in tablespace test while running make installcheck. > > 1. CREATE TABLESPACE regress_tblspacewith LOCATION 'blah'; > 2. DROP TABLESPACE reg

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread Amit Langote
On 2019/04/23 14:45, David Rowley wrote: > On Tue, 23 Apr 2019 at 13:49, Amit Langote > wrote: >> >> On 2019/04/23 7:51, Alvaro Herrera wrote: >>> To me, it sounds >>> unintuitive to accept partitions that don't exactly match the order of >>> the parent table; but it's been supported all along. >>

<    1   2