Re: non-exclusive backup cleanup is mildly broken

2019-12-13 Thread Michael Paquier
On Thu, Dec 12, 2019 at 01:52:31PM +0100, Magnus Hagander wrote: > On Thu, Dec 12, 2019 at 5:58 AM Kyotaro Horiguchi > wrote: >> The direction seems reasonable, but the patch doesn't free up the >> before_shmem_exec slot nor avoid duplicate registration of the >> callback. Actually before_shmem_ex

Re: non-exclusive backup cleanup is mildly broken

2019-12-13 Thread Magnus Hagander
On Fri, Dec 13, 2019 at 9:00 AM Michael Paquier wrote: > On Thu, Dec 12, 2019 at 01:52:31PM +0100, Magnus Hagander wrote: > > On Thu, Dec 12, 2019 at 5:58 AM Kyotaro Horiguchi < > horikyota@gmail.com> > > wrote: > > > My first reaction would be to just disallow the combination of prepared > >

Async_Notify

2019-12-13 Thread Арсен Арутюнян
Hello! I wrote my extension for PG and am trying to run Async_Notify from it. But messages come very late! Sometimes only after I execute the notify command from the shell. What am I doing wrong? How to use Async_Notify correctly.   -- Арсен Арутюнян

Re: PITR on DROP DATABASE, deleting of the database directory despite the recovery_target_time set before.

2019-12-13 Thread Jean-Christophe Arnu
Hello, Le mar. 19 nov. 2019 à 16:15, Nicolas Lutic a écrit : > > We are aware that this part is tricky and will have little effects on > normal operations, as best practices are to use xid_target or lsn_target. > > I'm working with Nicolas and we made some further testing. If we use xid target w

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-12-13 Thread Amit Kapila
On Wed, Dec 11, 2019 at 11:46 PM Robert Haas wrote: > > On Mon, Dec 2, 2019 at 3:32 AM Dilip Kumar wrote: > > I have rebased the patch set on the latest head. > > 0001 looks like a clever approach, but are you sure it doesn't hurt > performance when many small XLOG records are being inserted? I t

Re: Async_Notify

2019-12-13 Thread Pavel Stehule
Hi pá 13. 12. 2019 v 10:00 odesílatel Арсен Арутюнян napsal: > Hello! I wrote my extension for PG and am trying to run Async_Notify from > it. But messages come very late! Sometimes only after I execute the notify > command from the shell. What am I doing wrong? How to use Async_Notify > correct

[PATCH] Improve documentation of REINDEX options

2019-12-13 Thread Josef Šimánek
Hello! According to discussion at pgsql-general ( https://www.postgresql.org/message-id/flat/CAFp7QwqFYcHiARfT91rOQj%3DmFT0MWBE%2BkxEmjfQh3QmRN1UBiw%40mail.gmail.com#05b75be4fd11c0e6216f0b329c808f72) I have prepared patch to improve documentation for REINDEX. It should be more inline with another

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-12-13 Thread Amit Kapila
On Thu, Dec 12, 2019 at 9:45 AM Dilip Kumar wrote: > > On Wed, Dec 11, 2019 at 5:22 PM Amit Kapila wrote: > > > > On Mon, Dec 9, 2019 at 1:27 PM Dilip Kumar wrote: > > > > > > I have review the patch set and here are few comments/questions > > > > > > 1. > > > +static void > > > +pg_decode_strea

Unmatched test and comment in partition_join.sql regression test

2019-12-13 Thread Etsuro Fujita
I noticed this in the regression test while polishing the PWJ-enhancement patch: -- partitionwise join can not be applied for a join between list and range -- partitioned tables EXPLAIN (COSTS OFF) SELECT t1.a, t1.c, t2.b, t2.c FROM prt1_n t1 FULL JOIN prt1 t2 ON (t1.c = t2.c); The test doesn't m

RE: [Patch] Optimize dropping of relation buffers using dlist

2019-12-13 Thread k.jami...@fujitsu.com
Hi, I have updated the patch (v5). I tried to reduce the lock waiting times by using spinlock when inserting/deleting buffers in the new hash table, and exclusive lock when doing lookup for buffers to be dropped. In summary, instead of scanning the whole buffer pool in shared buffers, we just tra

Re: Re[2]: Async_Notify

2019-12-13 Thread Pavel Stehule
pá 13. 12. 2019 v 12:30 odesílatel Арсен Арутюнян napsal: > I'm trying to send a notification from the PG extension directly. > It turns out the mechanism that is used in the async file. (Async_Notify) > does not suit me since the function uses oldcontext = MemoryContextSwitchTo > (CurTransaction

Re: remove support for old Python versions

2019-12-13 Thread Peter Eisentraut
On 2019-12-09 23:32, Tom Lane wrote: * In the docs section beginning "Context managers syntax using the with keyword", could we drop that entire ? It seems like it's now not saying much more than "you can use this standard python feature", which is hardly surprising information. That section p

Re: more backtraces

2019-12-13 Thread Peter Eisentraut
On 2019-12-04 22:34, Tom Lane wrote: Andres Freund writes: It'd be bad if the addition of backtraces for SEGV/BUS suddenly made it harder to attach a debugger and getting useful results. Yeah. TBH, I'm not sure I want this, at least not in debug builds. I understand that the SEGV/BUS thing

Re[2]: Async_Notify

2019-12-13 Thread Арсен Арутюнян
I'm trying to send a notification from the PG extension directly. It turns out the mechanism that is used in the async file. (Async_Notify) does not suit me since the function uses oldcontext = MemoryContextSwitchTo (CurTransactionContext); and MemoryContextSwitchTo (oldcontext); And I did not f

Re: Remove configure --disable-float4-byval and --disable-float8-byval

2019-12-13 Thread Peter Eisentraut
On 2019-12-12 23:06, Peter Geoghegan wrote: Apparently Linux has almost no upstream resources for testing 32-bit x86, and it shows: But isn't 32-bit Windows still a thing? Or does that work differently? -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7

automating pg_config.h.win32 maintenance

2019-12-13 Thread Peter Eisentraut
Keeping pg_config.h.win32 up to date with pg_config.h.in is a gratuitous annoyance. This setup dates back to the minimal client-only Windows builds using win32.mak files, which has been removed in PG10. The MSVC build system has the power of Perl available, so we can do better. My proposal i

Re: automating pg_config.h.win32 maintenance

2019-12-13 Thread Heikki Linnakangas
On 13/12/2019 14:51, Peter Eisentraut wrote: Keeping pg_config.h.win32 up to date with pg_config.h.in is a gratuitous annoyance. Hear hear! My proposal is that we essentially emulate what config.status does in Perl code. config.status gets a list of defines discovered by configure and proces

logical replication does not fire per-column triggers

2019-12-13 Thread Peter Eisentraut
The logical replication subscription side does not fire per-column update triggers when applying an update change. (Per-table update triggers work fine.) This patch fixes that. Should be backpatched to PG10. A patch along these lines is also necessary to handle triggers involving generated

Re: error context for vacuum to include block number

2019-12-13 Thread Michael Paquier
On Thu, Dec 12, 2019 at 09:08:31PM -0600, Justin Pryzby wrote: > On Wed, Dec 11, 2019 at 08:54:25AM -0800, Andres Freund wrote: >> Hm, wonder if could be worthwhile to not use a separate struct here, but >> instead extend one of the existing structs to contain the necessary >> information. Or perha

Re: automating pg_config.h.win32 maintenance

2019-12-13 Thread Michael Paquier
On Fri, Dec 13, 2019 at 03:14:08PM +0200, Heikki Linnakangas wrote: > On 13/12/2019 14:51, Peter Eisentraut wrote: >> Keeping pg_config.h.win32 up to date with pg_config.h.in is a gratuitous >> annoyance. > > Hear hear! Youpi. > I don't think this @simple_defines is really any better than listin

Re: automating pg_config.h.win32 maintenance

2019-12-13 Thread Tom Lane
Michael Paquier writes: > On Fri, Dec 13, 2019 at 03:14:08PM +0200, Heikki Linnakangas wrote: >> On 13/12/2019 14:51, Peter Eisentraut wrote: >>> Keeping pg_config.h.win32 up to date with pg_config.h.in is a gratuitous >>> annoyance. >> Hear hear! > Youpi. +1 >> I don't think this @simple_defi

Re: automating pg_config.h.win32 maintenance

2019-12-13 Thread Peter Eisentraut
On 2019-12-13 14:44, Michael Paquier wrote: It would be nice to put a comment close to FLEXIBLE_ARRAY_MEMBER, where you use "/* */" as a way to emulate an empty value which is still defined. Or would it be cleaner to just use an empty string? That's just the way Autoconf does it. I haven't po

Re: disable only nonparallel seq scan.

2019-12-13 Thread Jeff Janes
On Tue, Dec 10, 2019 at 1:32 PM Robert Haas wrote: > On Sun, Dec 8, 2019 at 1:24 PM Jeff Janes wrote: > > Is there a way to force a meaningful parallel seq scan, or at least the > planning of one, when the planner wants a non-parallel one? > > > > Usually I can do things like with with enable_*

Re: BUG #16059: Tab-completion of filenames in COPY commands removes required quotes

2019-12-13 Thread Tom Lane
I wrote: >> Alvaro Herrera writes: >>> I don't quite understand why a readline library that doesn't have >>> rl_filename_completion_function is known to have a >>> filename_completion_function, ie. this bit >>> #ifdef HAVE_RL_FILENAME_COMPLETION_FUNCTION >>> #define filename_completion_function

Re: recovery_min_apply_delay in archive recovery causes assertion failure in latch

2019-12-13 Thread Alvaro Herrera
On 2019-Oct-19, Michael Paquier wrote: > On Thu, Oct 17, 2019 at 02:35:13PM +0900, Michael Paquier wrote: > > ArchiveRecoveryRequested will be set to true if recovery.signal or > > standby.signal are found, so it seems to me that you can make all > > those checks more simple by removing from the e

Re: BUG #16059: Tab-completion of filenames in COPY commands removes required quotes

2019-12-13 Thread Alvaro Herrera
On 2019-Dec-13, Tom Lane wrote: > I wrote: > >> Alvaro Herrera writes: > >>> #ifdef HAVE_RL_FILENAME_COMPLETION_FUNCTION > >>> #define filename_completion_function rl_filename_completion_function > >>> #else > >>> /* decl missing in some header files, but function exists anyway */ > >>> extern c

Re: Remove configure --disable-float4-byval and --disable-float8-byval

2019-12-13 Thread Robert Haas
On Fri, Dec 13, 2019 at 6:33 AM Peter Eisentraut wrote: > On 2019-12-12 23:06, Peter Geoghegan wrote: > > Apparently Linux has almost no upstream resources for testing 32-bit > > x86, and it shows: > > But isn't 32-bit Windows still a thing? Or does that work differently? Well, again, I think th

Re: Memory-Bounded Hash Aggregation

2019-12-13 Thread Tomas Vondra
On Thu, Dec 12, 2019 at 06:10:50PM -0800, Jeff Davis wrote: On Thu, 2019-11-28 at 18:46 +0100, Tomas Vondra wrote: 13) As for this: /* make sure that we don't exhaust the hash bits */ if (partition_bits + input_bits >= 32) partition_bits = 32 - input_bits; We already ran int

Re: BUG #16059: Tab-completion of filenames in COPY commands removes required quotes

2019-12-13 Thread Tom Lane
Alvaro Herrera writes: > On 2019-Dec-13, Tom Lane wrote: >> A possible further change is to switch the code over to calling >> "rl_filename_completion_function", and then invert the sense of >> this logic, like ... > +1, I think that's clearer. OK, I went ahead and pushed that change, since it s

Re: Remove configure --disable-float4-byval and --disable-float8-byval

2019-12-13 Thread Tom Lane
Robert Haas writes: > Well, again, I think the proposal here is not get rid of 32-bit > support, but to have less code that only gets regularly tested on > 32-bit machines. That seems like generally a good plan. But as to the specific idea... > If we made datums 8 bytes everywhere, we would hav

Re: BUG #16059: Tab-completion of filenames in COPY commands removes required quotes

2019-12-13 Thread Tom Lane
I wrote: > Robert Haas writes: >> On Wed, Dec 11, 2019 at 10:52 AM Tom Lane wrote: >>> I think it's Debian's problem, not ours, if that doesn't work. It is >>> not unreasonable for a package to probe existence of a library function >>> at configure time. It's up to them to make sure that the he

Re: BUG #16059: Tab-completion of filenames in COPY commands removes required quotes

2019-12-13 Thread Tom Lane
Alvaro Herrera writes: > I suggest to indicate in complete_from_files where to find the hook > functions it refers to (say "see quote_file_name, below", or something.) Done. > I tested this on libreadline 7.x (where #define > HAVE_RL_FILENAME_COMPLETION_FUNCTION 1). I noticed that if I enter a

psql's \watch is broken

2019-12-13 Thread Jeff Janes
If I do something like this: explain (analyze) select * from pgbench_accounts \watch 1 It behaves as expected. But once I break out of the loop with ctrl-C, then if I execute the same thing again it executes the command once, but shows no output and doesn't loop. It seems like some flag is gett

Re: Rearranging ALTER TABLE to avoid multi-operations bugs

2019-12-13 Thread Tom Lane
I wrote: > [ fix-alter-table-order-of-operations-1.patch ] The cfbot noticed that this failed to apply over a recent commit, so here's v2. No substantive changes. regards, tom lane diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index daa

xlog.c variable pointlessly global

2019-12-13 Thread Alvaro Herrera
commit message says it all. -- Álvaro Herrera Developer, https://www.PostgreSQL.org/ >From 7950671ea23407b1b06735268088009af73557d0 Mon Sep 17 00:00:00 2001 From: Alvaro Herrera Date: Fri, 13 Dec 2019 16:34:54 -0300 Subject: [PATCH] Demote variable from global to local r

Re: archive status ".ready" files may be created too early

2019-12-13 Thread Alvaro Herrera
On 2019-Dec-13, Kyotaro Horiguchi wrote: > At Thu, 12 Dec 2019 22:50:20 +, "Bossart, Nathan" > wrote in > > The crux of the issue seems to be that XLogWrite() does not wait for > > the entire record to be written to disk before creating the ".ready" > > file. Instead, it just waits for th

Re: error context for vacuum to include block number

2019-12-13 Thread Justin Pryzby
On Fri, Dec 13, 2019 at 10:28:50PM +0900, Michael Paquier wrote: >> v4-0001-Rename-buf-to-avoid-shadowing-buf-of-another-type.patch >> v4-0002-Remove-redundant-call-to-vacuum-progress.patch >> v4-0003-deduplication.patch >> v4-0004-vacuum-errcontext-to-show-block-being-processed.patch >> v4-0005-a

Re: Errors "failed to construct the join relation" and "failed to build any 2-way joins"

2019-12-13 Thread Tom Lane
I wrote: > Will Leinweber writes: >> On 12.1, fresh initdb the following query gives me the error >> "ERROR: failed to construct the join relation" > Eyeing the plan produced by v11, I'm suspecting some oversight in > the RTE_RESULT changes (4be058fe9); but I haven't actually bisected. Yup: it'

Re: psql's \watch is broken

2019-12-13 Thread Fabien COELHO
explain (analyze) select * from pgbench_accounts \watch 1 It behaves as expected. But once I break out of the loop with ctrl-C, then if I execute the same thing again it executes the command once, but shows no output and doesn't loop. It seems like some flag is getting set with ctrl-C, but t

Re: xlog.c variable pointlessly global

2019-12-13 Thread Daniel Gustafsson
> On 13 Dec 2019, at 21:07, Alvaro Herrera wrote: > > commit message says it all. I haven't tested it, but reading it makes perfect sense. +1. cheers ./daniel

Re: non-exclusive backup cleanup is mildly broken

2019-12-13 Thread David Steele
On 12/13/19 3:56 AM, Magnus Hagander wrote: On Fri, Dec 13, 2019 at 9:00 AM Michael Paquier I think that's a bad idea to put a restriction of this kind.  There are large consumers of 2PC, and everybody needs backups. You misunderstood me. I certainly didn't mean that people who use 2PC

Re: logical decoding bug: segfault in ReorderBufferToastReplace()

2019-12-13 Thread Andres Freund
Hi, On 2019-12-13 16:13:35 -0800, Jeremy Schneider wrote: > On 12/11/19 08:35, Andres Freund wrote: > > I think we need to see pg_waldump output for the preceding records. That > > might allow us to see why there's a toast record that's being associated > > with this table, despite there not being

Re: Why is get_actual_variable_range()'s use of SnapshotNonVacuumable safe during recovery?

2019-12-13 Thread Peter Geoghegan
On Wed, Nov 20, 2019 at 1:43 PM Peter Geoghegan wrote: > My understanding is that we can trust RecentGlobalXmin to be something > useful and current during recovery, in general, so the selfuncs.c > index-only scan (which uses SnapshotNonVacuumable + RecentGlobalXmin) > can be trusted to work just

Re: logical replication does not fire per-column triggers

2019-12-13 Thread Euler Taveira
Em sex., 13 de dez. de 2019 às 10:26, Peter Eisentraut escreveu: > > The logical replication subscription side does not fire per-column > update triggers when applying an update change. (Per-table update > triggers work fine.) This patch fixes that. Should be backpatched to PG10. > Using the re

Re: psql's \watch is broken

2019-12-13 Thread Michael Paquier
On Sat, Dec 14, 2019 at 12:09:51AM +0100, Fabien COELHO wrote: > >> explain (analyze) select * from pgbench_accounts \watch 1 >> >> It behaves as expected. But once I break out of the loop with ctrl-C, then >> if I execute the same thing again it executes the command once, but shows >> no output

Re: xlog.c variable pointlessly global

2019-12-13 Thread Michael Paquier
On Sat, Dec 14, 2019 at 12:32:30AM +0100, Daniel Gustafsson wrote: > On 13 Dec 2019, at 21:07, Alvaro Herrera wrote: >> >> commit message says it all. > > I haven't tested it, but reading it makes perfect sense. +1. +1 says it all. -- Michael signature.asc Description: PGP signature

Re: checkpointer: PANIC: could not fsync file: No such file or directory

2019-12-13 Thread Thomas Munro
On Fri, Dec 13, 2019 at 5:41 PM Thomas Munro wrote: > Here's a better version: it uses the existing fd if we have it already > in md_seg_fds, but opens and closes a transient one if not. Pushed.

Re: psql's \watch is broken

2019-12-13 Thread Jeff Janes
On Fri, Dec 13, 2019 at 9:45 PM Michael Paquier wrote: > On Sat, Dec 14, 2019 at 12:09:51AM +0100, Fabien COELHO wrote: > > > >> explain (analyze) select * from pgbench_accounts \watch 1 > >> > >> It behaves as expected. But once I break out of the loop with ctrl-C, > then > >> if I execute the

Re: checkpointer: PANIC: could not fsync file: No such file or directory

2019-12-13 Thread Thomas Munro
On Sat, Dec 14, 2019 at 4:49 PM Thomas Munro wrote: > On Fri, Dec 13, 2019 at 5:41 PM Thomas Munro wrote: > > Here's a better version: it uses the existing fd if we have it already > > in md_seg_fds, but opens and closes a transient one if not. > > Pushed. Build farm not happy... checking...

Re: checkpointer: PANIC: could not fsync file: No such file or directory

2019-12-13 Thread Thomas Munro
On Sat, Dec 14, 2019 at 5:05 PM Thomas Munro wrote: > > Pushed. > > Build farm not happy... checking... Hrmph. FileGetRawDesc() does not contain a call to FileAccess(), so this is failing on low-fd-limit systems. Looking into a way to fix that...

Re: checkpointer: PANIC: could not fsync file: No such file or directory

2019-12-13 Thread Thomas Munro
On Sat, Dec 14, 2019 at 5:32 PM Thomas Munro wrote: > On Sat, Dec 14, 2019 at 5:05 PM Thomas Munro wrote: > > > Pushed. > > > > Build farm not happy... checking... > > Hrmph. FileGetRawDesc() does not contain a call to FileAccess(), so > this is failing on low-fd-limit systems. Looking into a w

Re: logical decoding : exceeded maxAllocatedDescs for .spill files

2019-12-13 Thread Amit Kapila
On Thu, Dec 12, 2019 at 9:50 PM Amit Khandekar wrote: > > Attached is a v4 patch that also addresses your code comments so far. > I have included the test case in 006_logical_decoding.pl. I observed > that the test case just adds only about 0.5 to 1 sec time. Please > verify on your env also, and