Re: Bloom index cost model seems to be wrong

2019-09-26 Thread Michael Paquier
On Wed, Sep 25, 2019 at 05:12:26PM -0300, Alvaro Herrera wrote: > It's not clear to me what the next action should be on this patch. I > think Jeff got some feedback from Tom, but was that enough to expect a > new version to be posted? That was in February; should we now (in late > September) clo

Re: Global temporary tables

2019-09-26 Thread Konstantin Knizhnik
On 25.09.2019 23:28, Alvaro Herrera wrote: This broke recently. Can you please rebase? Rebased version of the patch is attached. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c

RE: Speed up transaction completion faster after many relations are accessed in a transaction

2019-09-26 Thread Tsunakawa, Takayuki
From: Alvaro Herrera [mailto:alvhe...@2ndquadrant.com] > On 2019-Sep-03, Tsunakawa, Takayuki wrote: > > I don't think it's rejected. It would be a pity (mottainai) to refuse > > this, because it provides significant speedup despite its simple > > modification. > > I don't necessarily disagree wit

Re: Built-in connection pooler

2019-09-26 Thread Konstantin Knizhnik
On 25.09.2019 23:14, Alvaro Herrera wrote: Travis complains that the SGML docs are broken. Please fix. Sorry. Patch with fixed SGML formating error is attached. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company diff --git a/contrib/spi

Re: [HACKERS] Cached plans and statement generalization

2019-09-26 Thread Konstantin Knizhnik
On 25.09.2019 23:06, Alvaro Herrera wrote: This patch fails the "rules" tests. Please fix. Sorry, New version of the patch with corrected expected output for rules test is attached. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company diff

Re: Refactoring of connection with password prompt loop for frontends

2019-09-26 Thread Michael Paquier
On Wed, Sep 25, 2019 at 05:47:39PM -0300, Alvaro Herrera wrote: > This patch has been absolutely overlooked by reviewers. Euler, you're > registered as a reviewer for it. Will you submit a review soon? :-) > > It does not currently apply, so if we get a rebase, that'd be good too. Here you go.

Re: Proposal for syntax to support creation of partition tables when creating parent table

2019-09-26 Thread Amit Langote
Hi Ahsan, Usama Thanks for starting work on this. On Thu, Sep 26, 2019 at 3:46 AM Ahsan Hadi wrote: > On Wed, Sep 25, 2019 at 8:53 PM Tom Lane wrote: >> As Fabien noted, there's been some related discussion about this >> area, but nobody was advocating a solution of this particular shape. > > T

pg_wal/RECOVERYHISTORY file remains after archive recovery

2019-09-26 Thread Masahiko Sawada
Hi, When we do archive recovery from the database cluster of which timeline ID is more than 2 pg_wal/RECOVERYHISTORY is remained even after archive recovery completed. The cause of this seems cbc55da556b that moved exitArchiveRecovery() to before writeTimeLineHistory(). writeTimeLineHIstory() res

Re: Add comments for a postgres program in bootstrap mode

2019-09-26 Thread Amit Langote
Hi Shiraishi-san, On Thu, Sep 26, 2019 at 3:06 PM Youki Shiraishi wrote: > > Hi, > > I have just started to read the PostgreSQL code and found a lack of comments > for a postgres backend program in bootstrap mode. > When I saw the --boot option implemented in src/backend/main/main.c at first >

Re: Minimal logical decoding on standbys

2019-09-26 Thread Amit Khandekar
On Wed, 18 Sep 2019 at 19:34, Robert Haas wrote: > I took a bit of a look at > 0004-New-TAP-test-for-logical-decoding-on-standby.patch and saw some > things I don't like in terms of general code quality: > > - Not many comments. I think each set of tests should have a block > comment at the top ex

Re: pg_wal/RECOVERYHISTORY file remains after archive recovery

2019-09-26 Thread Fujii Masao
On Thu, Sep 26, 2019 at 5:15 PM Masahiko Sawada wrote: > > Hi, > > When we do archive recovery from the database cluster of which > timeline ID is more than 2 pg_wal/RECOVERYHISTORY is remained even > after archive recovery completed. > > The cause of this seems cbc55da556b that moved exitArchiveR

Re: Add comments for a postgres program in bootstrap mode

2019-09-26 Thread Youki Shiraishi
Hello Amit, On Thu, Sep 26, 2019 at 5:38 PM Amit Langote wrote: > > Hi Shiraishi-san, > > On Thu, Sep 26, 2019 at 3:06 PM Youki Shiraishi > wrote: > > > > Hi, > > > > I have just started to read the PostgreSQL code and found a lack of > > comments for a postgres backend program in bootstrap mo

Re: Standby accepts recovery_target_timeline setting?

2019-09-26 Thread Fujii Masao
On Thu, Sep 26, 2019 at 5:22 AM David Steele wrote: > > While testing against PG12 I noticed the documentation states that > recovery targets are not valid when standby.signal is present. Or that description in the doc is not true? Other recovery target parameters seem to take effect even when st

WAL records

2019-09-26 Thread Looserof7
I have started to learn postgresql. While going through the WAL dump saw the below records, rmgr: Heap2 len (rec/tot): 60/60, tx: 0, lsn: 4F/CFF1F0F8, prev 4F/CFF1EB70, desc: CLEAN remxid 0, blkref #0: rel 1663/16385/1259 blk 1301 rmgr: Heap2 len (rec/tot): 60/

Unstable select_parallel regression output in 12rc1

2019-09-26 Thread Christoph Berg
Building the 12rc1 package on Ubuntu eoan/amd64, I got this regression diff: 12:06:27 diff -U3 /<>/build/../src/test/regress/expected/select_parallel.out /<>/build/src/bin/pg_upgrade/tmp_check/regress/results/select_parallel.out 12:06:27 --- /<>/build/../src/test/regress/expected/select_paralle

Re: Attempt to consolidate reading of XLOG page

2019-09-26 Thread Antonin Houska
Alvaro Herrera wrote: > On 2019-Sep-24, Antonin Houska wrote: > > > Alvaro Herrera wrote: > > > > If you don't have any strong dislikes for these changes, I'll push this > > > part and let you rebase the remains on top. > > > > No objections here. > > oK, pushed. Please rebase the other par

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2019-09-26 Thread Alexey Kondratov
On 01.08.2019 19:53, Alexey Kondratov wrote: On 26.07.2019 20:43, Liudmila Mantrova wrote: On a more general note, I wonder if everyone is happy with the --using-postgresql-conf option name, or we should continue searching for a narrower term. Unfortunately, I don't have any better suggestions

Re: Refactoring of connection with password prompt loop for frontends

2019-09-26 Thread Alvaro Herrera
On 2019-Sep-26, Michael Paquier wrote: > On Wed, Sep 25, 2019 at 05:47:39PM -0300, Alvaro Herrera wrote: > > This patch has been absolutely overlooked by reviewers. Euler, you're > > registered as a reviewer for it. Will you submit a review soon? :-) > > > > It does not currently apply, so if

Re: pgbench - allow to create partitioned tables

2019-09-26 Thread Alvaro Herrera
pgbench's main() is overly long already, and the new code being added seems to pollute it even more. Can we split it out into a static function that gets placed, say, just below disconnect_all() or maybe after runInitSteps? (Also, we seem to be afraid of function prototypes. Why not move the app

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-09-26 Thread Amit Kapila
On Tue, Sep 3, 2019 at 4:16 PM Tomas Vondra wrote: > > On Mon, Sep 02, 2019 at 06:06:50PM -0400, Alvaro Herrera wrote: > >In the interest of moving things forward, how far are we from making > >0001 committable? If I understand correctly, the rest of this patchset > >depends on https://commitfest

Re: Batch insert in CTAS/MatView code

2019-09-26 Thread Alvaro Herrera
On 2019-Sep-25, Asim R P wrote: > I reviewed your patch today. It looks good overall. My concern is that > the ExecFetchSlotHeapTuple call does not seem appropriate. In a generic > place such as createas.c, we should be using generic tableam API only. > However, I can also see that there is no

Re: add a MAC check for TRUNCATE

2019-09-26 Thread Yuli Khodorkovskiy
On Wed, Sep 25, 2019 at 5:57 PM Tom Lane wrote: > I don't see how the addition of a new permissions check could sanely > be back-patched unless it were to default to "allow", which seems like > an odd choice. > > regards, tom lane That makes sense. Alternatively, we coul

Re: Add FOREIGN to ALTER TABLE in pg_dump

2019-09-26 Thread Luis Carril
I don't disagree with adding FOREIGN, though. Your patch is failing the pg_dump TAP tests. Please use configure --enable-tap-tests, fix the problems, then resubmit. Fixed, I've attached a new version. Cheers Luis M Carril diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c ind

Re: dropdb --force

2019-09-26 Thread Pavel Stehule
út 24. 9. 2019 v 14:52 odesílatel Amit Kapila napsal: > On Sat, Sep 21, 2019 at 10:09 PM Pavel Stehule > wrote: > > > > Thank you for check. I am sending updated patch > > > > Alvaro has up thread suggested some alternative syntax [1] for this > patch, but I don't see any good argument to not go

Re: Auxiliary Processes and MyAuxProc

2019-09-26 Thread Alvaro Herrera
0002 seems way too large (and it doesn't currently apply). Is there something we can do to make it more manageable? I think it would be better to put your 0001 in second place rather than first, since your other patch doesn't use it AFAICS and it adds functionality that has not yet received appro

Re: Two pg_rewind patches (auto generate recovery conf and ensure clean shutdown)

2019-09-26 Thread Paul Guo
On Thu, Sep 26, 2019 at 1:48 AM Alvaro Herrera wrote: > CC Alexey for reasons that become clear below. > > On 2019-Sep-25, Paul Guo wrote: > > > > Question about 0003. If we specify --skip-clean-shutdown and the > cluter > > > was not cleanly shut down, shouldn't we error out instead of trying t

Re: Two pg_rewind patches (auto generate recovery conf and ensure clean shutdown)

2019-09-26 Thread Paul Guo
> > > Yes, -R is already used in pg_basebackup for the same functionality, so > it seems natural to use it here as well for consistency. > > I will review options naming in my own patch and update it accordingly. > Maybe -w/-W or -a/-A options will be good, since it's about WALs > retrieval from ar

Re: Auxiliary Processes and MyAuxProc

2019-09-26 Thread Mike Palmiotto
On Thu, Sep 26, 2019 at 9:49 AM Alvaro Herrera wrote: > > 0002 seems way too large (and it doesn't currently apply). Is there > something we can do to make it more manageable? Initially we were thinking of submitting one patch for the centralization work and then separate patches per backend typ

Re: dropdb --force

2019-09-26 Thread Pavel Stehule
út 24. 9. 2019 v 17:51 odesílatel vignesh C napsal: > On Tue, Sep 24, 2019 at 6:22 PM Amit Kapila > wrote: > > > > On Sat, Sep 21, 2019 at 10:09 PM Pavel Stehule > wrote: > > > > > > Thank you for check. I am sending updated patch > > > > > > Session termination in case of drop database is solv

Re: JSONPATH documentation

2019-09-26 Thread Peter Eisentraut
On 2019-09-25 16:46, Liudmila Mantrova wrote: > On 9/25/19 12:08 AM, Peter Eisentraut wrote: >> On 2019-09-23 00:03, Tom Lane wrote: >>> While we're whining about this, I find it very off-putting that >>> the jsonpath stuff was inserted in the JSON functions section >>> ahead of the actual JSON fun

Re: PostgreSQL12 and older versions of OpenSSL

2019-09-26 Thread Peter Eisentraut
On 2019-09-26 06:53, Michael Paquier wrote: > So I agree with the proposal to rely on the presence of > TLS_MAX_VERSION, and base our decision-making on that. But then there is this: commit 04cd70c6899c6b36517b2b07d7a12b2cceba1bef Author: Kurt Roeckx Date: Tue Sep 18 22:17:14 2018 Depreca

Re: Two pg_rewind patches (auto generate recovery conf and ensure clean shutdown)

2019-09-26 Thread Alvaro Herrera
> Thanks. I've updated the reset two patches and attached as v8. Great, thanks. > Note in the 2nd patch, the long option is changed as below. Both the option > and description > now seems to be more concise since we want db state as either DB_SHUTDOWNED > or > DB_SHUTDOWNED_IN_RECOVERY. > > "-

Re: Auxiliary Processes and MyAuxProc

2019-09-26 Thread Alvaro Herrera
On 2019-Sep-26, Mike Palmiotto wrote: > On Thu, Sep 26, 2019 at 9:49 AM Alvaro Herrera > wrote: > > > > 0002 seems way too large (and it doesn't currently apply). Is there > > something we can do to make it more manageable? > > Initially we were thinking of submitting one patch for the > centr

Re: Add comments for a postgres program in bootstrap mode

2019-09-26 Thread Amit Langote
Hi Shiraishi-san, On Thu, Sep 26, 2019 at 6:32 PM Youki Shiraishi wrote: > On Thu, Sep 26, 2019 at 5:38 PM Amit Langote wrote: > > On Thu, Sep 26, 2019 at 3:06 PM Youki Shiraishi > > wrote: > > > I have just started to read the PostgreSQL code and found a lack of > > > comments for a postgres

Re: Unstable select_parallel regression output in 12rc1

2019-09-26 Thread Tom Lane
Christoph Berg writes: > Building the 12rc1 package on Ubuntu eoan/amd64, I got this > regression diff: The append-order differences have been seen before, per this thread: https://www.postgresql.org/message-id/flat/CA%2BhUKG%2B0CxrKRWRMf5ymN3gm%2BBECHna2B-q1w8onKBep4HasUw%40mail.gmail.com We h

Re: Add comments for a postgres program in bootstrap mode

2019-09-26 Thread Tom Lane
Youki Shiraishi writes: > On Thu, Sep 26, 2019 at 5:38 PM Amit Langote wrote: >> * The main entry point for auxiliary processes, such as the bgwriter, >> - * walwriter, walreceiver, bootstrapper and the shared memory checker code. >> + * walwriter, walreceiver, postgres program in bootstrap mod

Re: dropdb --force

2019-09-26 Thread Pavel Stehule
st 25. 9. 2019 v 6:38 odesílatel vignesh C napsal: > On Sat, Sep 21, 2019 at 10:09 PM Pavel Stehule > wrote: > > fixed > > > > Thank you for check. I am sending updated patch > > > Thanks for fixing all the comments. > Couple of suggestions: > -DROP DATABASE [ IF EXISTS ] class="parameter">name

Re: dropdb --force

2019-09-26 Thread Pavel Stehule
st 25. 9. 2019 v 4:14 odesílatel Amit Kapila napsal: > On Tue, Sep 24, 2019 at 6:22 PM Amit Kapila > wrote: > > On Sat, Sep 21, 2019 at 10:09 PM Pavel Stehule > wrote: > > > > > > Thank you for check. I am sending updated patch > > > > > > > Alvaro has up thread suggested some alternative synta

Re: dropdb --force

2019-09-26 Thread Alvaro Herrera
On 2019-Sep-26, Pavel Stehule wrote: > Alternative is DROP DATABASE [IF EXISTS] name [ CASCADE | RESTRICT ] [ WITH > FORCE ] > > but in this case WIDTH keyword should not be optional (If I need to solve > Tom's note). Currently WITH keyword is optional every where, so I think so > using syntax wi

Re: dropdb --force

2019-09-26 Thread Pavel Stehule
čt 26. 9. 2019 v 17:35 odesílatel Alvaro Herrera napsal: > On 2019-Sep-26, Pavel Stehule wrote: > > > Alternative is DROP DATABASE [IF EXISTS] name [ CASCADE | RESTRICT ] [ > WITH > > FORCE ] > > > > but in this case WIDTH keyword should not be optional (If I need to solve > > Tom's note). Curren

Re: PostgreSQL12 and older versions of OpenSSL

2019-09-26 Thread Peter Eisentraut
Here is my proposed patch, currently completely untested. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services >From 35324dbe908a779c9d84b438cb54328cfd74e403 Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Thu, 2

Re: dropdb --force

2019-09-26 Thread Peter Eisentraut
On 2019-09-26 17:35, Alvaro Herrera wrote: > Well, you would have one of those: > > DROP DATABASE [IF EXISTS] name WITH (FORCE) > DROP DATABASE [IF EXISTS] name > > Naturally, the WITH is optional in the sense that the clause itself is > optional. (Note we don't have CASCADE/RESTRICT in DROP DAT

Re: Hooks for session start and end, take two

2019-09-26 Thread legrand legrand
Hello, Thank you for the work done on this subject. After starting to play with it, I have a question and a remark: > - previous hook calls were only called for normal backends, which was > incorrect as we define the backend so as we apply no backend-related > filtering for the hook. Does that

Re: Parallel Append subplan order instability on aye-aye

2019-09-26 Thread Tom Lane
I wrote: > Yeah. I think we've had quite enough of the stats-transmission-related > failures, and they're no longer proving anything about the original > problem. So I will go do what I proposed in mid-July and revert the > stats queries, while keeping the reltuples/relpages check. (I'd kind > o

Re: Fetching timeline during recovery

2019-09-26 Thread Jehan-Guillaume de Rorthais
On Mon, 9 Sep 2019 19:44:10 +0900 Fujii Masao wrote: > On Sat, Sep 7, 2019 at 12:06 AM Jehan-Guillaume de Rorthais > wrote: > > > > On Wed, 4 Sep 2019 00:32:03 +0900 > > Fujii Masao wrote: > > [...] > Thanks for updating the patch! Thank you for your review! Please find in attachment a new

Re: dropdb --force

2019-09-26 Thread Pavel Stehule
čt 26. 9. 2019 v 18:34 odesílatel Peter Eisentraut < peter.eisentr...@2ndquadrant.com> napsal: > On 2019-09-26 17:35, Alvaro Herrera wrote: > > Well, you would have one of those: > > > > DROP DATABASE [IF EXISTS] name WITH (FORCE) > > DROP DATABASE [IF EXISTS] name > > > > Naturally, the WITH is o

Re: Standby accepts recovery_target_timeline setting?

2019-09-26 Thread David Steele
On 9/26/19 5:55 AM, Fujii Masao wrote: > On Thu, Sep 26, 2019 at 5:22 AM David Steele wrote: >> >> While testing against PG12 I noticed the documentation states that >> recovery targets are not valid when standby.signal is present. > > Or that description in the doc is not true? Other recovery ta

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-09-26 Thread Tomas Vondra
BA, Training & Services 0001-Add-logical_work_mem-to-limit-ReorderBuffer-20190926.patch.gz Description: application/gzip 0002-Immediately-WAL-log-assignments-20190926.patch.gz Description: application/gzip 0003-Issue-individual-invalidations-with-wal_lev-20190926.patch.gz Description: application/

Re: recovery starting when backup_label exists, but not recovery.signal

2019-09-26 Thread David Steele
On 9/24/19 1:25 AM, Fujii Masao wrote: > > When backup_label exists, the startup process enters archive recovery mode > even if recovery.signal file doesn't exist. In this case, the startup process > tries to retrieve WAL files by using restore_command. Then, at the beginning > of the archive reco

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-09-26 Thread Tomas Vondra
On Thu, Sep 26, 2019 at 06:58:17PM +0530, Amit Kapila wrote: On Tue, Sep 3, 2019 at 4:16 PM Tomas Vondra wrote: On Mon, Sep 02, 2019 at 06:06:50PM -0400, Alvaro Herrera wrote: >In the interest of moving things forward, how far are we from making >0001 committable? If I understand correctly, t

Re: Allow to_date() and to_timestamp() to accept localized names

2019-09-26 Thread Juan José Santamaría Flecha
On Wed, Sep 25, 2019 at 9:57 PM Alvaro Herrera wrote: > > This patch no longer applies. Can you please rebase? Thank you for the notification. The patch rot after commit 1a950f3, a rebased version is attached. Regards, Juan José Santamaría Flecha diff --git a/doc/src/sgml/func.sgml b/doc/src/

Re: WAL records

2019-09-26 Thread Tomas Vondra
On Thu, Sep 26, 2019 at 11:14:53AM +0530, Looserof7 wrote: I have started to learn postgresql. While going through the WAL dump saw the below records, rmgr: Heap2 len (rec/tot): 60/60, tx: 0, lsn: 4F/CFF1F0F8, prev 4F/CFF1EB70, desc: CLEAN remxid 0, blkref #0: rel 1663/163

Re: WIP: BRIN multi-range indexes

2019-09-26 Thread Tomas Vondra
On Wed, Sep 25, 2019 at 05:07:48PM -0300, Alvaro Herrera wrote: This patch fails to apply (or the opclass params one, maybe). Please update. Yeah, the opclass params patches got broken by 773df883e adding enum reloptions. The breakage is somewhat extensive so I'll leave it up to Nikita to fix

Re: Shared Memory: How to use SYSV rather than MMAP ?

2019-09-26 Thread Alvaro Herrera
On 2019-Sep-10, Thomas Munro wrote: > Here's a quick rebase. It needs testing, review and (probably) > adjustment from AIX users. I'm not going to be able to do anything > with it on my own due to lack of access, though I'm happy to help get > this committed eventually. If we don't get any trac

Re: Memory Accounting

2019-09-26 Thread Tomas Vondra
On Tue, Sep 24, 2019 at 11:46:49AM -0700, Melanie Plageman wrote: On Thu, Sep 19, 2019 at 11:00 AM Jeff Davis wrote: On Wed, 2019-09-18 at 13:50 -0700, Soumyadeep Chakraborty wrote: > Hi Jeff, Hi Soumyadeep and Melanie, Thank you for the review! > max_stack_depth max level lazy

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-09-26 Thread Alvaro Herrera
On 2019-Sep-26, Tomas Vondra wrote: > Hi, > > Attached is an updated patch series, rebased on current master. It does > fix one memory accounting bug in ReorderBufferToastReplace (the code was > not properly updating the amount of memory). Cool. Can we aim to get 0001 pushed during this commitf

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-09-26 Thread Alvaro Herrera
On 2019-Sep-26, Alvaro Herrera wrote: > How certain are you about the approach to measure memory used by a > reorderbuffer transaction ... does it not cause a measurable performance > drop? I wonder if it would make more sense to use a separate contexts > per transaction and use context-level acc

Re: Online checksums patch - once again

2019-09-26 Thread Alvaro Herrera
On 2019-Aug-26, Magnus Hagander wrote: > OK, let's try this again :) > > This is work mainly based in the first version of the online checksums > patch, but based on top of Andres WIP patchset for global barriers ( > https://www.postgresql.org/message-id/20181030051643.elbxjww5jjgnjaxg%40alap3.an

Re: pg_upgrade fails with non-standard ACL

2019-09-26 Thread Bruce Momjian
On Wed, Sep 11, 2019 at 06:25:38PM -0300, Álvaro Herrera wrote: > On 2019-Aug-21, Bruce Momjian wrote: > > > > 1) How exactly should we report this incompatibility to a user? > > > I think it's fine to leave the warnings and also write some hint for the > > > user by analogy with other checks. > >

Re: pg_upgrade fails with non-standard ACL

2019-09-26 Thread Alvaro Herrera
On 2019-Sep-26, Bruce Momjian wrote: > On Wed, Sep 11, 2019 at 06:25:38PM -0300, Álvaro Herrera wrote: > > On 2019-Aug-21, Bruce Momjian wrote: > > > > > > 1) How exactly should we report this incompatibility to a user? > > > > I think it's fine to leave the warnings and also write some hint for

Re: pg_upgrade fails with non-standard ACL

2019-09-26 Thread Bruce Momjian
On Thu, Sep 26, 2019 at 05:16:19PM -0300, Alvaro Herrera wrote: > On 2019-Sep-26, Bruce Momjian wrote: > > Well, right now, pg_upgrade --check succeeds, but the upgrade fails. I > > am proposing, at a minimum, that pg_upgrade --check fails in such cases, > > Agreed, that should be a minimum fix.

Re: Minimal logical decoding on standbys

2019-09-26 Thread Robert Haas
On Thu, Sep 26, 2019 at 5:14 AM Amit Khandekar wrote: > Actually in some of the conflict-recovery testcases, I am still using > wait_for_xmins() so that we could test the xmin values back after we > drop the slots. So xmin-related testing is embedded in these recovery > tests as well. We can move

Re: Memory Accounting

2019-09-26 Thread Jeff Davis
On Thu, 2019-09-26 at 21:22 +0200, Tomas Vondra wrote: > It's worth mentioning that those bechmarks (I'm assuming we're > talking > about the numbers Rober shared in [1]) were done on patches that used > the eager accounting approach (i.e. walking all parent contexts and > updating the accounting f

Re: Standby accepts recovery_target_timeline setting?

2019-09-26 Thread Peter Eisentraut
On 2019-09-25 22:21, David Steele wrote: > While testing against PG12 I noticed the documentation states that > recovery targets are not valid when standby.signal is present. > > But surely the exception is recovery_target_timeline? My testing > confirms that this works just as in prior versions

Re: pgbench - allow to create partitioned tables

2019-09-26 Thread Fabien COELHO
Hello Alvaro, pgbench's main() is overly long already, and the new code being added seems to pollute it even more. Can we split it out into a static function that gets placed, say, just below disconnect_all() or maybe after runInitSteps? I agree that refactoring is a good idea, but I do not

Re: Standby accepts recovery_target_timeline setting?

2019-09-26 Thread David Steele
On 9/26/19 4:48 PM, Peter Eisentraut wrote: On 2019-09-25 22:21, David Steele wrote: While testing against PG12 I noticed the documentation states that recovery targets are not valid when standby.signal is present. But surely the exception is recovery_target_timeline? My testing confirms that

Re: pgbench - allow to create partitioned tables

2019-09-26 Thread Alvaro Herrera
On 2019-Sep-26, Fabien COELHO wrote: > > Hello Alvaro, > > > pgbench's main() is overly long already, and the new code being added > > seems to pollute it even more. Can we split it out into a static > > function that gets placed, say, just below disconnect_all() or maybe > > after runInitSteps

Re: FETCH FIRST clause PERCENT option

2019-09-26 Thread Ryan Lambert
On Thu, Sep 19, 2019 at 6:52 AM Surafel Temesgen wrote: > Hi Tom, > In the attached patch i include the comments given > > regards > Surafel > Patch v9 applies and passes make installcheck-world. > From: Tom Lane > Date: 2019-09-05 22:26:29 > * I didn't really study the changes in nodeLimit.c

Re: range_agg

2019-09-26 Thread Alvaro Herrera
Hello Paul, I've started to review this patch. Here's a few minor things I ran across -- mostly compiler warnings (is my compiler too ancient?). You don't have to agree with every fix -- feel free to use different fixes if you have them. Also, feel free to squash them onto whatever commit you li

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-09-26 Thread Tomas Vondra
On Thu, Sep 26, 2019 at 04:36:20PM -0300, Alvaro Herrera wrote: On 2019-Sep-26, Alvaro Herrera wrote: How certain are you about the approach to measure memory used by a reorderbuffer transaction ... does it not cause a measurable performance drop? I wonder if it would make more sense to use a

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-09-26 Thread Tomas Vondra
On Thu, Sep 26, 2019 at 04:33:59PM -0300, Alvaro Herrera wrote: On 2019-Sep-26, Tomas Vondra wrote: Hi, Attached is an updated patch series, rebased on current master. It does fix one memory accounting bug in ReorderBufferToastReplace (the code was not properly updating the amount of memory).

Re: Auxiliary Processes and MyAuxProc

2019-09-26 Thread Mike Palmiotto
On Thu, Sep 26, 2019 at 10:56 AM Alvaro Herrera wrote: > > On 2019-Sep-26, Mike Palmiotto wrote: > > > On Thu, Sep 26, 2019 at 9:49 AM Alvaro Herrera > > wrote: > > > > > > 0002 seems way too large (and it doesn't currently apply). Is there > > > something we can do to make it more manageable?

Instability of partition_prune regression test results

2019-09-26 Thread Tom Lane
Every so often the partition_prune test falls over, for example here, here, and here: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=sidewinder&dt=2019-08-15%2021%3A45%3A00 https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=anole&dt=2019-08-21%2022%3A19%3A23 https://buildfarm.postgre

Re: range test for hash index?

2019-09-26 Thread Tomas Vondra
On Wed, Sep 25, 2019 at 09:07:13AM +0530, Amit Kapila wrote: On Wed, Sep 18, 2019 at 9:30 AM Amit Kapila wrote: On Mon, Sep 16, 2019 at 11:24 PM Paul A Jungwirth wrote: > > On Mon, Sep 16, 2019 at 5:28 AM Amit Kapila wrote: > > I don't see this function on the master branch. Is this functio

Re: Optimize partial TOAST decompression

2019-09-26 Thread Tomas Vondra
On Wed, Sep 25, 2019 at 05:38:34PM -0300, Alvaro Herrera wrote: Hello, can you please update this patch? I'm not the patch author, but I've been looking at the patch recently and I have a rebased version at hand - so attached. FWIW I believe the patch is solid and in good shape, and it got st

Re: [PATCH] Race condition in logical walsender causes long postgresql shutdown delay

2019-09-26 Thread Jeff Janes
On Wed, Sep 11, 2019 at 3:52 PM Alvaro Herrera wrote: > > Reading over this code, I noticed that the detection of the catch-up > state ends up being duplicate code, so I would rework that function as > in the attached patch. > > The naming of those flags (got_SIGUSR2, got_STOPPING) is terrible, b

Re: range test for hash index?

2019-09-26 Thread Amit Kapila
On Fri, Sep 27, 2019 at 4:03 AM Tomas Vondra wrote: > > On Wed, Sep 25, 2019 at 09:07:13AM +0530, Amit Kapila wrote: > >On Wed, Sep 18, 2019 at 9:30 AM Amit Kapila wrote: > >> > >> On Mon, Sep 16, 2019 at 11:24 PM Paul A Jungwirth > >> wrote: > >> > > >> > On Mon, Sep 16, 2019 at 5:28 AM Amit Ka

Re: PostgreSQL12 and older versions of OpenSSL

2019-09-26 Thread Michael Paquier
On Thu, Sep 26, 2019 at 04:43:33PM +0200, Peter Eisentraut wrote: > On 2019-09-26 06:53, Michael Paquier wrote: > > So I agree with the proposal to rely on the presence of > > TLS_MAX_VERSION, and base our decision-making on that. > > But then there is this: > > commit 04cd70c6899c6b36517b2b07d7a

Re: FETCH FIRST clause PERCENT option

2019-09-26 Thread Kyotaro Horiguchi
At Thu, 26 Sep 2019 15:13:42 -0600, Ryan Lambert wrote in > On Thu, Sep 19, 2019 at 6:52 AM Surafel Temesgen > wrote: > > > Hi Tom, > > In the attached patch i include the comments given > > > > regards > > Surafel > > > > Patch v9 applies and passes make installcheck-world. > > > From: Tom

Re: [PATCH] Race condition in logical walsender causes long postgresql shutdown delay

2019-09-26 Thread Alvaro Herrera
On 2019-Sep-26, Jeff Janes wrote: > On Wed, Sep 11, 2019 at 3:52 PM Alvaro Herrera > wrote: > > > Reading over this code, I noticed that the detection of the catch-up > > state ends up being duplicate code, so I would rework that function as > > in the attached patch. > > > > The naming of those

Re: PostgreSQL12 and older versions of OpenSSL

2019-09-26 Thread Michael Paquier
On Thu, Sep 26, 2019 at 06:24:22PM +0200, Peter Eisentraut wrote: > Here is my proposed patch, currently completely untested. I have tested compilation of REL_12_STABLE with the top of OpenSSL 0.9.8, 1.0.0, 1.0.1, 1.0.2, 1.1.0 and 1.1.1. Our SSL tests also pass in all the setups I have tested. Y

Re: pgbench - allow to create partitioned tables

2019-09-26 Thread Amit Kapila
On Fri, Sep 27, 2019 at 2:36 AM Alvaro Herrera wrote: > On 2019-Sep-26, Fabien COELHO wrote: > > > pgbench's main() is overly long already, and the new code being added > > > seems to pollute it even more. Can we split it out into a static > > > function that gets placed, say, just below disconne

tab complete for explain SETTINGS

2019-09-26 Thread Justin Pryzby
Here's to hoping this is the worst omission in v12. Justin >From e21f58504e5006de9766fe586550b59167e00ffd Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Thu, 26 Sep 2019 21:12:26 -0500 Subject: [PATCH v1] tab completion for explain (SETTINGS) missed at ea569d64ac7174d3fe657e3e682d11053ecf186

Re: Instability of partition_prune regression test results

2019-09-26 Thread Amit Langote
On Fri, Sep 27, 2019 at 7:25 AM Tom Lane wrote: > Every so often the partition_prune test falls over, for example > here, here, and here: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=sidewinder&dt=2019-08-15%2021%3A45%3A00 > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=an

SSL tests failing for channel_binding with OpenSSL <= 1.0.1

2019-09-26 Thread Michael Paquier
Hi all, (Jeff Davis in CC) As $subject tells, any version of OpenSSL not including X509_get_signature_nid() (version <= 1.0.1) causes the SSL tests to fail. This has been introduced by d6e612f. We need to do something similar to c3d41cc for the test, as per the attached. I have tested that with

Re: Implementing Incremental View Maintenance

2019-09-26 Thread Tatsuo Ishii
> Have you had any thoughts for more than two joined tables? I am not sure what you are asking here but if you are asking if IVM supports two or more tables involved in a join, we already support it: DROP MATERIALIZED VIEW mv1; DROP MATERIALIZED VIEW DROP TABLE t1; DROP TABLE DROP TABLE t2; DROP

Keep compiler silence (clang 10, implicit conversion from 'long' to 'double' )

2019-09-26 Thread Yuya Watari
Hello, I found the problem that clang compiler introduces warnings when building PostgreSQL. Attached patch fixes it. === Compiler version === clang version 10.0.0-svn372772-1~exp1+0~20190924181208.2504~1.gbpb209ff (trunk) Older versions of clang may not generate this warning. === Warning ===

Re: Remove page-read callback from XLogReaderState.

2019-09-26 Thread Kyotaro Horiguchi
At Wed, 25 Sep 2019 15:50:32 +0900 (Tokyo Standard Time), Kyotaro Horiguchi wrote in <20190925.155032.13779064.horikyota@gmail.com> > 709d003fbd hit this. Rebased. Oops! I found a silly silent bug that it doesn't verify the first page in new segments. Moreover it didin't load the first page

Re: tab complete for explain SETTINGS

2019-09-26 Thread Tatsuro Yamada
On 2019/09/27 11:20, Justin Pryzby wrote: Here's to hoping this is the worst omission in v12. Justin Hi Justin, I share my test result of your patch. I used two commits REL_12_RC1 and Head, and got a Hunk below: #REL_12_RC1 (17822c0e4f5ab8093e78f665c9e44766ae648a44) ===

Cleanup code related to OpenSSL <= 0.9.6 in fe/be-secure-openssl.c

2019-09-26 Thread Michael Paquier
Hi all, While reviewing the area, I have bumped into the following bit in fe-secure-openssl.c and be-secure-openssl.c: -/* OpenSSL 0.96 does not support X509_V_FLAG_CRL_CHECK */ -#ifdef X509_V_FLAG_CRL_CHECK [... stuff ...] I think that this did not get removed because of the incorrect v

Re: Two pg_rewind patches (auto generate recovery conf and ensure clean shutdown)

2019-09-26 Thread Paul Guo
> > > > Note in the 2nd patch, the long option is changed as below. Both the > option > > and description > > now seems to be more concise since we want db state as either > DB_SHUTDOWNED > > or > > DB_SHUTDOWNED_IN_RECOVERY. > > > > "-s, --no-ensure-shutdowned do not auto-fix unclean shutdown"

Re: Add comments for a postgres program in bootstrap mode

2019-09-26 Thread Youki Shiraishi
Hello Tom, On Fri, Sep 27, 2019 at 12:23 AM Tom Lane wrote: > > Youki Shiraishi writes: > > On Thu, Sep 26, 2019 at 5:38 PM Amit Langote > > wrote: > >> * The main entry point for auxiliary processes, such as the bgwriter, > >> - * walwriter, walreceiver, bootstrapper and the shared memory c

Re: Control your disk usage in PG: Introduction to Disk Quota Extension

2019-09-26 Thread Haozhou Wang
Thanks Alvaro! I rebased this patch with the newest master branch. Attached the new patch disk_quota_hooks_v5.patch in the attachment. Regards, Haozhou On Thu, Sep 26, 2019 at 3:54 AM Alvaro Herrera wrote: > This patch no longer applies. Can you please rebase? > > -- > Álvaro Herrera > https://

Re: Add comments for a postgres program in bootstrap mode

2019-09-26 Thread Youki Shiraishi
Hi Amit, On Fri, Sep 27, 2019 at 12:10 AM Amit Langote wrote: > > Hi Shiraishi-san, > > On Thu, Sep 26, 2019 at 6:32 PM Youki Shiraishi > wrote: > > On Thu, Sep 26, 2019 at 5:38 PM Amit Langote > > wrote: > > > On Thu, Sep 26, 2019 at 3:06 PM Youki Shiraishi > > > wrote: > > > > I have just

Re: tab complete for explain SETTINGS

2019-09-26 Thread Michael Paquier
On Fri, Sep 27, 2019 at 12:18:17PM +0900, Tatsuro Yamada wrote: > Anyway, I tested the patch and it looks fine. :) Thanks Justin and Yamada-san. The order of the options in the list to display and in the check did not match the order of the documentation, which is the intention here, so fixed and

Re: Add comments for a postgres program in bootstrap mode

2019-09-26 Thread Michael Paquier
On Fri, Sep 27, 2019 at 12:29:08PM +0900, Youki Shiraishi wrote: > I also vote to get rid of such ambiguous stuff. > As you can see by grepping, "bootstrap-mode backend" (and something > like that) is also called in the sources as: > > - bootstrap backend > - (basic) bootstrap process > - backend

Re: dropdb --force

2019-09-26 Thread Amit Kapila
On Thu, Sep 26, 2019 at 10:04 PM Peter Eisentraut wrote: > > On 2019-09-26 17:35, Alvaro Herrera wrote: > > Well, you would have one of those: > > > > DROP DATABASE [IF EXISTS] name WITH (FORCE) > > DROP DATABASE [IF EXISTS] name > > > > Naturally, the WITH is optional in the sense that the clause

Re: Add comments for a postgres program in bootstrap mode

2019-09-26 Thread Amit Langote
On Fri, Sep 27, 2019 at 1:03 PM Michael Paquier wrote: > > On Fri, Sep 27, 2019 at 12:29:08PM +0900, Youki Shiraishi wrote: > > I also vote to get rid of such ambiguous stuff. > > As you can see by grepping, "bootstrap-mode backend" (and something > > like that) is also called in the sources as: >

Re: range test for hash index?

2019-09-26 Thread Amit Kapila
On Fri, Sep 27, 2019 at 6:02 AM Amit Kapila wrote: > > On Fri, Sep 27, 2019 at 4:03 AM Tomas Vondra > wrote: > > > > > > By "inconsistent" you mean that pre-10 versions will have different > > expected output than versions with WAL-logged hash indexes? > > > > Yes. > > > I don't see > > why that

Re: Batch insert in CTAS/MatView code

2019-09-26 Thread Paul Guo
Asim Thanks for the review. On Wed, Sep 25, 2019 at 6:39 PM Asim R P wrote: > > > > On Mon, Sep 9, 2019 at 4:02 PM Paul Guo wrote: > > > > So in theory > > we should not worry about additional tuple copy overhead now, and then I > tried the patch without setting > > multi-insert threshold as at

  1   2   >