Re: [HACKERS] pg_basebackup --progress output for batch execution

On Fri, Sep 29, 2017 at 4:00 PM, Martin Marques < martin.marq...@2ndquadrant.com> wrote: > Hi, > > Some time ago I had to work on a system where I was cloning a standby > using pg_basebackup, that didn't have screen or tmux. For that reason I > redirected the output to a file and ran it with nohup

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

On Fri, Nov 3, 2017 at 1:34 PM, Tom Lane wrote: > Jeff Janes writes: > > With this v3 patch (assuming this is the one you just committed > > as ec42a1dcb30de235b252f6d4) am now getting make check failures. > > There's a followup commit already :-( > >

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

Hi Alvaro, With this v3 patch (assuming this is the one you just committed as ec42a1dcb30de235b252f6d4) am now getting make check failures. brin_summarize_range is returning unexpected values. CentOS6, PostgreSQL 11devel on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7

[HACKERS] bgwriter_lru_maxpages range in postgresql.conf

With v10, the upper limit on bgwriter_lru_maxpages was changed from 1000 to INT_MAX / 2, but the postgresql.conf.sample was not updated. #bgwriter_lru_maxpages = 100# 0-1000 max buffers written/round I don't see any precedence for including INT_MAX-type limits in the sample config file, s

Re: [HACKERS] postgres_fdw super user checks

On Thu, Oct 5, 2017 at 6:44 AM, Robert Haas wrote: > On Wed, Oct 4, 2017 at 6:13 PM, Jeff Janes wrote: > > OK. And if you want the first one, you can wrap it in a view currently, > but > > if it were changed I don't know what you would do if you want the 2nd one >

Re: [HACKERS] postgres_fdw super user checks

On Thu, Sep 14, 2017 at 1:08 PM, Robert Haas wrote: > On Thu, Sep 14, 2017 at 2:33 PM, Jeff Janes wrote: > > I think that foreign tables ought to behave as views do, where they run > as > > the owner rather than the invoker. No one has talked me out of it, but > no > &

Re: [HACKERS] Possible SSL improvements for a newcomer to tackle

On Tue, Oct 3, 2017 at 6:44 AM, Tom Lane wrote: > Magnus Hagander writes: > > On Tue, Oct 3, 2017 at 6:33 AM, Tom Lane wrote: > >> I'm not an SSL expert, so insert appropriate grain of salt, but AIUI the > >> question is what are you going to verify against? > > > One way to do it would be to d

Re: [HACKERS] Possible SSL improvements for a newcomer to tackle

On Mon, Oct 2, 2017 at 9:33 PM, Tom Lane wrote: > > It's possible that we could adopt some policy like "if the root.crt file > exists then default to verify" ... but that seems messy and unreliable, > so I'm not sure it would really add any security. > That is what we do. If root.crt exists, we

Re: [HACKERS] 10RC1 crash testing MultiXact oddity

On Fri, Sep 22, 2017 at 1:19 PM, Robert Haas wrote: > On Fri, Sep 22, 2017 at 3:39 PM, Jeff Janes wrote: > > It turns out it is not new in pg10. I spotted in the log file only by > > accident while looking for something else. Now that I am looking for > it, I > >

Re: [HACKERS] pgbench - minor fix for meta command only scripts

On Mon, Sep 11, 2017 at 6:27 PM, Fabien COELHO wrote: > > Hello Jeff, > > Shouldn't we use pg_usleep to ensure portability? it is defined for >> front-end code. But it returns void, so the error check will have to be >> changed. >> > > Attached v3 with pg_usleep called instead. > > I didn't see

Re: [HACKERS] v10 pg_ctl compatibility

On Tue, Sep 26, 2017 at 4:31 PM, Tom Lane wrote: > Jeff Janes writes: > > I was not using -l. Instead I set logging_collector=on in > postgresql.conf, > > but I suppose that that is not sufficent? > > No, because initial stderr is still connected to whatever. > >

Re: [HACKERS] v10 pg_ctl compatibility

On Tue, Sep 26, 2017 at 3:54 PM, Tom Lane wrote: > Jeff Janes writes: > > On Tue, Sep 26, 2017 at 1:10 PM, Tom Lane wrote: > >> Really? The server should have detached itself from your terminal > >> group long before that. What platform is this? > > > Cen

Re: [HACKERS] v10 pg_ctl compatibility

On Tue, Sep 26, 2017 at 1:10 PM, Tom Lane wrote: > Jeff Janes writes: > > To add insult to injury, when v10 pg_ctl does restart a pre-10 server and > > it sits there for a long time waiting for it to start up even though it > has > > already started up, if I hit

Re: [HACKERS] v10 pg_ctl compatibility

On Tue, Sep 26, 2017 at 12:29 PM, Andres Freund wrote: > Hi, > > On 2017-09-26 11:59:42 -0700, Jeff Janes wrote: > > Should the release notes have a compatibility entry about pg_ctl restart, > > being used against a running pre-10 server, no longer being able to > de

[HACKERS] v10 pg_ctl compatibility

Should the release notes have a compatibility entry about pg_ctl restart, being used against a running pre-10 server, no longer being able to detect when startup is complete? I don't know if cross-version use of pg_ctl restart was ever officially supported, but the current behavior is rather confu

Re: [HACKERS] 10RC1 crash testing MultiXact oddity

On Fri, Sep 22, 2017 at 8:47 AM, Alvaro Herrera wrote: > Jeff Janes wrote: > > I am running some crash recovery testing against 10rc1 by injecting torn > > page writes, using a test case which generates a lot of multixact, some > > naturally by doing a lot fk updates, but

[HACKERS] 10RC1 crash testing MultiXact oddity

I am running some crash recovery testing against 10rc1 by injecting torn page writes, using a test case which generates a lot of multixact, some naturally by doing a lot fk updates, but most artificially by calling the pg_burn_multixact function from one of the attached patches. In 22 hours of run

Re: [HACKERS] SCRAM in the PG 10 release notes

On Thu, Sep 21, 2017 at 7:42 AM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 9/19/17 20:45, Peter Eisentraut wrote: > > On 9/19/17 17:55, Jeff Janes wrote: > >> I guess I'm late to the party, but I don't see why this is needed at > >&g

Re: [HACKERS] why not parallel seq scan for slow functions

On Tue, Sep 19, 2017 at 9:15 PM, Amit Kapila wrote: > On Wed, Sep 20, 2017 at 3:05 AM, Jeff Janes wrote: > > On Tue, Sep 19, 2017 at 1:17 PM, Thomas Munro > > wrote: > >> > >> On Thu, Sep 14, 2017 at 3:19 PM, Amit Kapila > >> wrote: > >> >

Re: [HACKERS] SCRAM in the PG 10 release notes

On Wed, Sep 20, 2017 at 6:42 AM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 9/19/17 21:44, Michael Paquier wrote: > >> The patch that Heikki posted seemed reasonable to me as a starting > >> point, but there probably needs to be more "how" information somewhere. > > > > I agr

Re: [HACKERS] SCRAM in the PG 10 release notes

On Tue, Sep 19, 2017 at 4:29 PM, Michael Paquier wrote: > On Wed, Sep 20, 2017 at 6:55 AM, Jeff Janes wrote: > > On Tue, Sep 19, 2017 at 1:32 PM, Heikki Linnakangas > wrote: > >> I'm not sure what exactly to do here. Where should we stick that notice? > >> W

Re: [HACKERS] SCRAM in the PG 10 release notes

On Tue, Sep 19, 2017 at 1:32 PM, Heikki Linnakangas wrote: > I'm not sure what exactly to do here. Where should we stick that notice? > We could put it in the release notes, where the bullet point about SCRAM > is, but it would be well hidden. If we want to give advice to people who > might not

Re: [HACKERS] why not parallel seq scan for slow functions

On Tue, Sep 19, 2017 at 1:17 PM, Thomas Munro wrote: > On Thu, Sep 14, 2017 at 3:19 PM, Amit Kapila > wrote: > > The attached patch fixes both the review comments as discussed above. > > This cost stuff looks unstable: > > test select_parallel ... FAILED > > ! Gather (cost=0.00..62388

Re: [HACKERS] Create replication slot in pg_basebackup if requested and not yet present

On Thu, Sep 14, 2017 at 8:23 AM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 9/12/17 16:39, Michael Banck wrote: > > We could split up the logic here and create the optional physical > > replication slot in the main connection and the temporary one in the WAL > > streamer conn

Re: [HACKERS] GnuTLS support

On Sun, Sep 17, 2017 at 2:17 PM, Andreas Karlsson wrote: > On 09/15/2017 06:55 PM, Jeff Janes wrote: > >> I can't build against gnutls-2.12.23-21.el6.x86_64 from CentOS 6.9 >> > > Thanks for testing my patch. I have fixed both these issues plus some of > the oth

Re: [HACKERS] CLUSTER command progress monitor

On Wed, Aug 30, 2017 at 7:12 PM, Tatsuro Yamada < yamada.tats...@lab.ntt.co.jp> wrote: > > The view provides the information of CLUSTER command progress details as > follows > postgres=# \d pg_stat_progress_cluster >View "pg_catalog.pg_stat_progress_cluster" >Column| T

Re: [HACKERS] GnuTLS support

On Thu, Aug 31, 2017 at 10:52 AM, Andreas Karlsson wrote: > > > > = Work left to do > > - Test code with older versions of GnuTLS > I can't build against gnutls-2.12.23-21.el6.x86_64 from CentOS 6.9 be-secure-gnutls.c: In function 'be_tls_init': be-secure-gnutls.c:168: warning: implicit declar

Re: [HACKERS] postgres_fdw super user checks

On Tue, Sep 12, 2017 at 1:13 AM, Andreas Karlsson wrote: > On 07/27/2017 09:45 PM, Jeff Janes wrote:> Here is an updated patch. This > version allows you use the password-less > >> connection if you either are the super-user directly (which is the >> existing committed

Re: [HACKERS] uninterruptible state in 10beta4

On Wed, Sep 13, 2017 at 2:41 PM, Andres Freund wrote: > Hi, > > On 2017-09-13 14:28:34 -0700, Jeff Janes wrote: > > In 10beta4 and 11dev, If I run the below it enters an uninterruptible > > state. After the insert starts, I give 15 or seconds or so until the > > memo

[HACKERS] uninterruptible state in 10beta4

In 10beta4 and 11dev, If I run the below it enters an uninterruptible state. After the insert starts, I give 15 or seconds or so until the memory usage starts to grow due to enqueued triggers checks. Then I can't interrupt it with either ctrl-C in psql or kill -15 from another terminal. I have t

Re: [HACKERS] pg_basebackup behavior on non-existent slot

On Wed, Sep 6, 2017 at 2:50 AM, Alvaro Herrera wrote: > Magnus Hagander wrote: > > On Mon, Sep 4, 2017 at 3:21 PM, Jeff Janes wrote: > > > > Should the parent process of pg_basebackup be made to respond to > SIGCHLD? > > > Or call waitpid(bgchild, &st

Re: [HACKERS] pgbench - minor fix for meta command only scripts

On Mon, Sep 11, 2017 at 1:49 AM, Fabien COELHO wrote: > > Hello Jeff, > > Ok, the problem was a little bit more trivial than I thought. > > The issue is that under a low rate there may be no transaction in > progress, however the wait procedure was relying on select's timeout. If > nothing is act

Re: [HACKERS] Automatic testing of patches in commit fest

On Sun, Sep 10, 2017 at 4:40 PM, Michael Paquier wrote: > Hi all, > > Thomas Munro has hacked up a prototype of application testing > automatically if patches submitted apply and build: > http://commitfest.cputube.org/ > > I would recommend have a look at it from time to time if you are a > patch

Re: [HACKERS] Create replication slot in pg_basebackup if requested and not yet present

On Wed, Sep 6, 2017 at 9:22 AM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 8/18/17 05:28, Michael Banck wrote: > >>> Rebased, squashed and slighly edited version attached. I've added this > >>> to the 2017-07 commitfest now as well: > >>> > >>> https://commitfest.postgresql.o

Re: [HACKERS] Fix performance of generic atomics

On Tue, Sep 5, 2017 at 11:39 AM, Jesper Pedersen wrote: > On 09/05/2017 02:24 PM, Tom Lane wrote: > >> Jesper Pedersen writes: >> >>> I have tested this patch on a 2-socket machine, but don't see any >>> performance change in the various runs. However, there is no regression >>> either in all ca

[HACKERS] pg_basebackup behavior on non-existent slot

If I tell pg_basebackup to use a non-existent slot, it immediately reports an error. And then it exits with an error, but only after streaming the entire database contents. If you are doing this interactively and are on the ball, of course, you can hit ctrl-C when you see the error message. I do

Re: [HACKERS] pgbench - minor fix for meta command only scripts

On Mon, Sep 4, 2017 at 1:56 PM, Fabien COELHO wrote: > > Hello Jeff, > > I have fixed a bug introduced in the patch by changing && by || in the (min_sec > 0 && maxsock != -1) condition which was inducing errors with multi-threads & clients... >>> > Since this commit (12788ae49e1933

Re: [HACKERS] pgbench - minor fix for meta command only scripts

On Mon, Sep 26, 2016 at 1:01 AM, Heikki Linnakangas wrote: > On 09/24/2016 12:45 PM, Fabien COELHO wrote: > >> >> Attached are some small changes to your version: >> >> I have added the sleep_until fix. >> >> I have fixed a bug introduced in the patch by changing && by || in the >> (min_sec > 0 &

Re: [HACKERS] pg_basebackup throttling doesn't throttle as promised

On Fri, Sep 1, 2017 at 1:32 PM, Jeff Janes wrote: > The "-r" option to pg_basebackup is supposed to throttle the rate of the > backup. But it only works properly if the server is mostly idle. > > Every non-trivial call to XLogFlush or XLogBackgroundFlush will wake up >

[HACKERS] pg_basebackup throttling doesn't throttle as promised

The "-r" option to pg_basebackup is supposed to throttle the rate of the backup. But it only works properly if the server is mostly idle. Every non-trivial call to XLogFlush or XLogBackgroundFlush will wake up the wal sender (the one which is not really sending wal, but base files), and the throt

Re: [HACKERS] pgbench: faster version of tpcb-like transaction

On Sat, Aug 26, 2017 at 4:28 PM, Peter Geoghegan wrote: > On Sat, Aug 26, 2017 at 3:53 PM, Jeff Janes wrote: > > I get nearly a 3 fold speed up using the new transaction, from 9184 to > 26383 > > TPS, on 8 CPU machine using scale 50 and: > > > > PGOPTIONS="

[HACKERS] pgbench: faster version of tpcb-like transaction

If all the data is in memory and you have a system with fast fsyncs (or are running with fsync off, or unlogged tables, or synchronous_commit off), then the big bottleneck in pgbench is the amount of back and forth between the pgbench program and the backend. There are 7 commands per transaction.

Re: [HACKERS] subscription worker signalling wal writer too much

On Mon, Jul 3, 2017 at 8:26 PM, Jeff Janes wrote: > On Sat, Jun 24, 2017 at 5:09 PM, Andres Freund wrote: > >> On 2017-06-15 15:06:43 -0700, Jeff Janes wrote: >> >> >> > >> > Wouldn't it >> > better, and much simpler, just to have revert

Re: [HACKERS] distinct estimate of a hard-coded VALUES list

On Wed, Aug 16, 2017 at 12:40 PM, Tom Lane wrote: > Jeff Janes writes: > > This patch still applies, and I think the argument for it is still valid. > > So I'm going to make a commit-fest entry for it. Is there additional > > evidence we should gather? > > I

Re: [HACKERS] Create replication slot in pg_basebackup if requested and not yet present

On Sun, Apr 9, 2017 at 4:22 AM, Michael Banck wrote: > Hi, > > Am Freitag, den 24.03.2017, 19:32 +0100 schrieb Michael Banck: > > On Thu, Mar 23, 2017 at 12:41:54PM +0100, Magnus Hagander wrote: > > > On Tue, Mar 21, 2017 at 8:34 PM, Robert Haas > wrote: > > > > So I tend to think that there sho

Re: [HACKERS] why not parallel seq scan for slow functions

On Wed, Jul 12, 2017 at 7:08 PM, Amit Kapila wrote: > On Wed, Jul 12, 2017 at 11:20 PM, Jeff Janes wrote: > > On Tue, Jul 11, 2017 at 10:25 PM, Amit Kapila > > wrote: > >> > >> On Wed, Jul 12, 2017 at 1:50 AM, Jeff Janes > wrote: > >> &

Re: [HACKERS] tab complete for psql pset pager values

On Wed, Jul 26, 2017 at 8:02 AM, Pavel Stehule wrote: > Hi > > attached trivial patch for missing tab complete for \pset pager setting > > Looks good to me. I've set it ready for committer. Cheers, Jeff

Re: [HACKERS] postgres_fdw super user checks

er >> > wrote: >> >> On Mon, Oct 17, 2016 at 3:33 AM, Jeff Janes >> wrote: >> >>> postgres_fdw has some checks to enforce that non-superusers must >> connect to >> >>> the foreign server with a password-based method. The reason for t

[HACKERS] tab completion for "create user mapping for"

If you have "CREATE USE" tab completion will recommend both USER and USER MAPPING FOR. But once you have the full "CREATE USER " or "CREATE USER M" it will not complete the rest of the "MAPPING FOR". Attached patch fixes that. Cheers, Jeff user_mapping_tab_complete_v1.patch Description: Binar

Re: [HACKERS] Increase Vacuum ring buffer.

On Thu, Jul 20, 2017 at 12:51 PM, Tom Lane wrote: > Robert Haas writes: > > I think that's a valid point. There are also other concerns here - > > e.g. whether instead of adopting the patch as proposed we ought to (a) > > use some smaller size, or (b) keep the size as-is but reduce the > > maxi

Re: [HACKERS] why not parallel seq scan for slow functions

On Sat, Jul 22, 2017 at 8:53 PM, Amit Kapila wrote: > On Thu, Jul 13, 2017 at 7:38 AM, Amit Kapila > wrote: > > On Wed, Jul 12, 2017 at 11:20 PM, Jeff Janes > wrote: > >> > >> > >> > >> Setting parallel_workers to 8 changes the threshold

Re: [HACKERS] Better error message for trying to drop a DB with open subscriptions?

On Thu, Jul 20, 2017 at 4:09 PM, Josh Berkus wrote: > All: > > The problem: > > postgres=# drop database bookdata; > ERROR: database "bookdata" is being accessed by other users > DETAIL: There is 1 other session using the database. > postgres=# \c bookdata > You are now connected to database "b

Re: [HACKERS] Increase Vacuum ring buffer.

On Thu, Jul 20, 2017 at 7:59 AM, Robert Haas wrote: > > > > Initially I wanted to make BAS_BULKWRITE and BAS_VACUUM ring sizes > > configurable, but after testing I don't see much gain from increasing > > ring buffer above 16MB. So I propose just 1 line change. > > I think the question for this p

Re: [HACKERS] autovacuum can't keep up, bloat just continues to rise

On Thu, Jul 20, 2017 at 6:28 AM, Stephen Frost wrote: > Greetings, > > * Sokolov Yura (y.soko...@postgrespro.ru) wrote: > > I wrote two days ago about vacuum ring buffer: > > https://www.postgresql.org/message-id/8737e9bddb82501da1134f021bf492 > 9a%40postgrespro.ru > > > > Increasing Vacuum's rin

Re: [HACKERS] SCRAM auth and Pgpool-II

On Fri, Jul 14, 2017 at 7:48 AM, Vladimir Borodin wrote: > > 14 июля 2017 г., в 1:33, Stephen Frost написал(а): > > What would be really nice for such cases is support for Kerberos and > delegated Kerberos credentials. Having pgpool support that would remove > the need to deal with passwords at

Re: [HACKERS] why not parallel seq scan for slow functions

On Tue, Jul 11, 2017 at 10:25 PM, Amit Kapila wrote: > On Wed, Jul 12, 2017 at 1:50 AM, Jeff Janes wrote: > > On Mon, Jul 10, 2017 at 9:51 PM, Dilip Kumar > wrote: > >> > >> So because of this high projection cost the seqpath and parallel path > >> bot

Re: [HACKERS] why not parallel seq scan for slow functions

On Mon, Jul 10, 2017 at 9:51 PM, Dilip Kumar wrote: > > > In below function, we always multiply the target->cost.per_tuple with > path->rows, but in case of gather it should multiply this with > subpath->rows > > apply_projection_to_path() > > > path->startup_cost += target->cost.startup - o

[HACKERS] why not parallel seq scan for slow functions

If I have a slow function which is evaluated in a simple seq scan, I do not get parallel execution, even though it would be massively useful. Unless force_parallel_mode=ON, then I get a dummy parallel plan with one worker. explain select aid,slow(abalance) from pgbench_accounts; CREATE OR REPLAC

Re: [HACKERS] pgsql 10: hash indexes testing

On Tue, Jul 4, 2017 at 3:57 AM, AP wrote: > > The data being indexed is BYTEA, (quasi)random and 64 bytes in size. > The table has over 2 billion entries. The data is not unique. There's > an average of 10 duplicates for every unique value. > What is the number of duplicates for the most common

Re: [HACKERS] subscription worker signalling wal writer too much

On Sat, Jun 24, 2017 at 5:09 PM, Andres Freund wrote: > Hi, > > On 2017-06-15 15:06:43 -0700, Jeff Janes wrote: > > > It looks like only limited consolidation was happening, the number of > kills > > invoked was more than half of the number of SetLatch. I think the r

Re: [HACKERS] Reducing pg_ctl's reaction time

On Thu, Jun 29, 2017 at 11:39 AM, Tom Lane wrote: > Jeff Janes writes: > > In the now-committed version of this, the 'pg_ctl start' returns > > successfully as soon as the server reaches a consistent state. Which is > OK, > > except that it does the same

Re: [HACKERS] Reducing pg_ctl's reaction time

On Tue, Jun 27, 2017 at 11:59 AM, Tom Lane wrote: > I wrote: > > Andres Freund writes: > >> On 2017-06-26 17:38:03 -0400, Tom Lane wrote: > >>> Hm. Take that a bit further, and we could drop the connection probes > >>> altogether --- just put the whole responsibility on the postmaster to > >>>

Re: [HACKERS] distinct estimate of a hard-coded VALUES list

On Tue, Aug 23, 2016 at 5:28 AM, Tom Lane wrote: > Tomas Vondra writes: > > On 08/22/2016 07:42 PM, Alvaro Herrera wrote: > >> Also, if we patch it this way and somebody has a slow query because of a > >> lot of duplicate values, it's easy to solve the problem by > >> de-duplicating. But with t

Re: [HACKERS] Reducing pg_ctl's reaction time

On Mon, Jun 26, 2017 at 12:15 PM, Tom Lane wrote: > Michael Paquier writes: > > On Mon, Jun 26, 2017 at 7:13 AM, Tom Lane wrote: > >> The attached proposed patch adjusts pg_ctl to check every 100msec, > >> instead of every second, for the postmaster to be done starting or > >> stopping. > > >>

[HACKERS] CREATE SUBSCRIPTION log noise

I think this should go away: ereport(NOTICE, (errmsg("created replication slot \"%s\" on publisher", slotname))); It doesn't appear to be contingent on anything other than the content of the command you just gave it. I don't th

Re: [HACKERS] subscription worker signalling wal writer too much

On Thu, Jun 15, 2017 at 3:06 PM, Jeff Janes wrote: > > This new patch is simpler than the previous one, and more effective at > speeding up replication. I assume it would speed up pgbench with > synchronous_commit turned off (or against unlogged tables) as well, but I > don

[HACKERS] CREATE SUBSCRIPTION documentation

https://www.postgresql.org/docs/devel/static/sql-createsubscription.html Has the note: See Section 26.2.5.1 for details on how to configure access control between the subscription and the publica

Re: [HACKERS] subscription worker signalling wal writer too much

On Wed, Jun 14, 2017 at 4:29 PM, Andres Freund wrote: > On 2017-06-14 16:24:27 -0700, Jeff Janes wrote: > > On Wed, Jun 14, 2017 at 3:20 PM, Andres Freund > wrote: > > > > > On 2017-06-14 15:08:49 -0700, Jeff Janes wrote: > > > > On Wed, Jun 14, 20

Re: [HACKERS] subscription worker signalling wal writer too much

On Wed, Jun 14, 2017 at 3:20 PM, Andres Freund wrote: > On 2017-06-14 15:08:49 -0700, Jeff Janes wrote: > > On Wed, Jun 14, 2017 at 11:55 AM, Jeff Janes > wrote: > > > > > If I publish a pgbench workload and subscribe to it, the subscription > > > worker is

Re: [HACKERS] subscription worker signalling wal writer too much

On Wed, Jun 14, 2017 at 11:55 AM, Jeff Janes wrote: > If I publish a pgbench workload and subscribe to it, the subscription > worker is signalling the wal writer thousands of times a second, once for > every async commit. This has a noticeable performance cost. > I've used a

[HACKERS] subscription worker signalling wal writer too much

If I publish a pgbench workload and subscribe to it, the subscription worker is signalling the wal writer thousands of times a second, once for every async commit. This has a noticeable performance cost. I don't think it is ever necessary to signal the wal writer here, unless wal writer is taking

Re: [HACKERS] logical replication: \dRp+ and "for all tables"

On Sat, Jun 10, 2017 at 7:42 AM, Tom Lane wrote: > Jeff Janes writes: > > On Fri, Jun 9, 2017 at 10:20 PM, Masahiko Sawada > > wrote: > >> On Sat, Jun 10, 2017 at 7:29 AM, Jeff Janes > wrote: > >>> That seems unfortunate. Should the "for all ta

Re: [HACKERS] Why restore_command is called for existing files in pg_xlog?

On Mon, Jun 12, 2017 at 5:25 AM, Alex Kliukin wrote: > On Fri, Jun 2, 2017, at 11:51 AM, Alexander Kukushkin wrote: > > Hello hackers, > There is one strange and awful thing I don't understand about > restore_command: it is always being called for every single WAL segment > postgres wants to appl

Re: [HACKERS] logical replication: \dRp+ and "for all tables"

On Fri, Jun 9, 2017 at 10:20 PM, Masahiko Sawada wrote: > On Sat, Jun 10, 2017 at 7:29 AM, Jeff Janes wrote: > > If I create a publication "for all tables", \dRp+ doesn't indicate it is > for > > all tables, it just gives a list of the tables. > >

[HACKERS] logical replication: \dRp+ and "for all tables"

If I create a publication "for all tables", \dRp+ doesn't indicate it is for all tables, it just gives a list of the tables. So it doesn't distinguish between a publication specified to be for all tables (which will be dynamic regarding future additions), and one which just happens to include all

[HACKERS] logical replication NOTICE "synchronized table states"

When creating a subscription, I get a NOTICE "synchronized table states" What is this supposed to be telling the end user? Is this just a debugging message not intended to be included in the released code? When I first saw this NOTICE, I thought, based on the wording, that it was telling me the

Re: [HACKERS] List of hostaddrs not supported

On Fri, Jun 9, 2017 at 11:52 AM, Heikki Linnakangas wrote: > On 06/09/2017 05:47 PM, Jeff Janes wrote: > >> Your commit to fix this part, 76b11e8a43eca4612d, is giving me compiler >> warnings: >> >> fe-connect.c: In function 'connectDBStart': >>

Re: [HACKERS] partial aggregation with internal state type

On Fri, Jun 9, 2017 at 9:06 AM, Tom Lane wrote: > Jeff Janes writes: > > The docs for creating aggregates for 9.6 and beyond say: > > "For aggregate functions whose state_data_type is internal, the > combinefunc > > must not be strict. In this case the comb

[HACKERS] partial aggregation with internal state type

The docs for creating aggregates for 9.6 and beyond say: "For aggregate functions whose state_data_type is internal, the combinefunc must not be strict. In this case the combinefunc must ensure that null states are handled correctly and that the state being returned is properly stored in the aggre

Re: [HACKERS] List of hostaddrs not supported

On Thu, Jun 8, 2017 at 1:36 AM, Heikki Linnakangas wrote: > While testing libpq and GSS the other day, I was surprised by the behavior > of the host and hostaddr libpq options, if you specify a list of hostnames. > > I did this this, and it took me quite a while to figure out what was going > on:

[HACKERS] postgres_fdw cost estimation defaults and documentation

The default value for fdw_tuple_cost is 0.01, which seems way too low. If I set up a loop-back foreign server with a large fetch_size, then tests like: select * from pgbench_accounts except select * from loopback.pgbench_accounts vs select * from pgbench_accounts except select * from pgbench_ac

Re: [HACKERS] logical replication - still unstable after all these months

On Fri, Jun 2, 2017 at 4:10 PM, Petr Jelinek wrote: > > While I was testing something for different thread I noticed that I > manage transactions incorrectly in this patch. Fixed here, I didn't test > it much yet (it takes a while as you know :) ). Not sure if it's related > to the issue you've s

Re: [HACKERS] psql: Activate pager only for height, not width

On Sun, May 28, 2017 at 10:09 PM, Brendan Jurd wrote: > Hello hackers, > > I am often frustrated by the default behaviour of the psql pager, which > will activate a pager if the output is deemed to be "too wide" for the > terminal, regardless of the number of lines output, and of the > pager_min_

Re: [HACKERS] logical replication busy-waiting on a lock

On Sat, May 27, 2017 at 6:48 AM, Petr Jelinek wrote: > > > Actually, I guess it's the pid 47457 (COPY process) who is actually > running the xid 73322726. In that case that's the same thing Masahiko > Sawada reported [1]. Related, but not the same. It would be nice if they didn't block, but if

Re: [HACKERS] logical replication - still unstable after all these months

On Sun, May 28, 2017 at 3:17 PM, Mark Kirkwood < mark.kirkw...@catalyst.net.nz> wrote: > On 28/05/17 19:01, Mark Kirkwood wrote: > > >> So running in cloud land now...so for no errors - will update. >> >> >> >> > The framework ran 600 tests last night, and I see 3 'NOK' results, i.e 3 > failed tes

[HACKERS] execGrouping.c limit on work_mem

In BuildTupleHashTable /* Limit initial table size request to not more than work_mem */ nbuckets = Min(nbuckets, (long) ((work_mem * 1024L) / entrysize)); Is this a good idea? If the caller of this code has no respect for work_mem, they are still going to blow it out of the water. Now

[HACKERS] simplehash.h typo

/* round up size to the next power of 2, that's the bucketing works */ That should probably be "that's the **way** bucketing works". Or maybe it is an idiom I don't grok. Cheers, Jeff

[HACKERS] logical replication busy-waiting on a lock

When I create a subscription in the disabled state, and then later doing "alter subscription sub enable;", on the master I sometimes get a tight loop of the deadlock detector: (log_lock_waits is on, of course) deadlock_timeout is set to 1s, so I don't know why it seems to be running several times

Re: [HACKERS] logical replication - still unstable after all these months

On Fri, May 26, 2017 at 12:27 AM, Erik Rijkers wrote: > On 2017-05-26 08:58, Simon Riggs wrote: > >> On 26 May 2017 at 07:10, Erik Rijkers wrote: >> >> - Do you agree this number of failures is far too high? >>> - Am I the only one finding so many failures? >>> >> >> What type of failure are you

Re: [HACKERS] logical replication - still unstable after all these months

On Fri, May 26, 2017 at 5:17 AM, tushar wrote: > > run second time = > ./pgbench -T 20 -c 90 -j 90 -f test.sql postgres > > check the row count on master/standby > Master= > postgres=# select count(*) from pgbench_history ; > count > > 536836 > (1 row) > > Standby = > > postgres=#

Re: [HACKERS] CREATE STATISTICS statistic_type documentation

On Thu, May 25, 2017 at 9:28 AM, Tom Lane wrote: > Jeff Janes writes: > > The docs for CREATE STATISTICS does not say what happens if the > > statistic_type clause is omitted. It should probably say that the > default > > action is to create both ndistinct and depende

[HACKERS] CREATE STATISTICS statistic_type documentation

The docs for CREATE STATISTICS does not say what happens if the statistic_type clause is omitted. It should probably say that the default action is to create both ndistinct and dependencies. Cheers, Jeff

[HACKERS] ALTER PUBLICATION materializing the list of tables

If I create a publication FOR ALL TABLES and then change my mind, the only thing I can do is drop the publication and recreate it. Since "ALTER PUBLICATION name SET TABLE" allows you to replace the entire table list, shouldn't it also let you change from the dynamic FOR ALL TABLES to a static spec

[HACKERS] ALTER PUBLICATION documentation

"The first variant of this command listed in the synopsis can change all of the publication properties specified in CREATE PUBLICATION ." That referenced first variant no longer exists. I don't if that should just be removed

[HACKERS] postgres_fdw aggregation pushdown has collation change in 10beta.

It is shipping collation-sensitive aggregates between servers which have different collations. commit 7012b132d07c2b4ea15b0b3cb1ea9f3278801d98 Author: Robert Haas Date: Fri Oct 21 09:54:29 2016 -0400 postgres_fdw: Push down aggregates to remote servers. I've attached a reproducing case.

[HACKERS] Transaction held open by autoanalyze can be a bottleneck

Autovacuum's analyze starts a transaction when it starts on a table, and holds it for the duration. This holds back the xmin horizon. On a TPC-B-like benchmark, this can be a problem. While it is autoanalyzing pgbench_accounts and pgbench_history, dead-but-for-analyze tuples accumulate rapidly in

Re: [HACKERS] logical replication deranged sender

On Tue, May 9, 2017 at 9:18 AM, Petr Jelinek wrote: > On 08/05/17 13:47, Petr Jelinek wrote: > > On 08/05/17 01:17, Jeff Janes wrote: > >> After dropping a subscription, it says it succeeded and that it dropped > >> the slot on the publisher. > >> > >>

[HACKERS] logical replication deranged sender

After dropping a subscription, it says it succeeded and that it dropped the slot on the publisher. But the publisher still has the slot, and a full-tilt process described by ps as postgres: wal sender process jjanes [local] idle in transaction Strace shows that this process is doing nothing but

Re: [HACKERS] tablesync patch broke the assumption that logical rep depends on?

On Wed, Apr 26, 2017 at 8:00 AM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 4/25/17 15:42, Petr Jelinek wrote: > >> Here is the patch doing just that. > > > > And one more revision which also checks in_use when attaching shared > > memory. This is mainly to improve the user v

Re: [HACKERS] DELETE and UPDATE with LIMIT and ORDER BY

On Mon, Apr 24, 2017 at 8:09 AM, Surafel Temesgen wrote: > the necessity of allowing limit and order by clause to be used with delete > and > update statement is discussed in the past and added to the todo list > > preveouse mailing list descissions > > http://archives.postgresql.org/pgadmin-hac

  1   2   3   4   5   6   7   8   9   10   >