Re: PATCH: pgbench - option to build using ppoll() for larger connection counts

Hello Doug, This version of the patch attempts to address the feedback for the previous submission on 28-Nov-2017 Please avoid recreating a thread, but rather respond to the previous one, I missed this post. The overall function-based implementation with limited ifdefs seems readable and

Re: Help needed in using 'on_dsm_detach' callback

On Wed, Jan 24, 2018 at 8:37 PM, Gaddam Sai Ram wrote: > Found that there is a callback for dsa detach but that function requires > segment pointer as an argument, Should be as below: > > on_dsm_detach(PROC_DSA_AREA->segment_maps[0].segment, detach_func); > ... > But i couldn't access that segment

Re: non-bulk inserts and tuple routing

On 2018/01/20 7:07, Robert Haas wrote: > On Fri, Jan 19, 2018 at 3:56 AM, Amit Langote > wrote: >> I rebased the patches, since they started conflicting with a recently >> committed patch [1]. > > I think that my latest commit has managed to break this pretty thoroughly. I rebased it. Here are

Re: [HACKERS] Refactoring identifier checks to consistently use strcmp

> On 24 Jan 2018, at 02:37, Michael Paquier wrote: > On Tue, Jan 23, 2018 at 04:22:22PM +0100, Daniel Gustafsson wrote: >> On 23 Jan 2018, at 05:52, Michael Paquier wrote: >>> Those are changed as well in the attached, which applies on top of your >>> v6. I have added as well in it the tests I s

Re: proposal: alternative psql commands quit and exit

Hi, > It's not really that complicated. Here's a patch. This follows what > Tom suggested in http://postgr.es/m/30157.1513058...@sss.pgh.pa.us and > what I suggested in > https://www.postgresql.org/message-id/CA%2BTgmoZswp00PtcgPfQ9zbbh7HUTgsLLJ9Z1x9E2s8Y7ep048g%40mail.gmail.com > > I've discov

Re: [HACKERS] proposal - Default namespaces for XPath expressions (PostgreSQL 11)

Hi 2018-01-23 8:13 GMT+01:00 Kyotaro HORIGUCHI : > Hello, I returned to this. > > I thouroughly checked the translator's behavior against the XPath > specifications and checkd out the documentation and regression > test. Almost everything is fine for me and this would be the last > comment from m

Re: proposal: alternative psql commands quit and exit

On 01/24/2018 10:17 AM, KAWAMICHI Ryoji wrote: > test=# select ' > test'# quit > Use \q to quit or press control-C to clear the input buffer. > test'# ' from t; > ?column? > -- > + > quit+ > > (1 row) > test=# select ' > test'# \q > test'# ' from t; > ?column? > ---

Re: Re: [HACKERS] pgbench randomness initialization

Here is a rebase, plus some more changes: I have improved the error message to tell from where the value was provided. I have removed the test to the exact values produced from the expression test run. I have added a test which run from the same seed value several times and checks that th

Is it valid to have logical replication between 2 databases on the same postgres server?

Hello hackers, I'm experimenting with Logical Replication. (https://www.postgresql.org/docs/10/static/logical-replication.html) What I'm trying to do may be impossible l but just wanted to ask here. I'm trying to do logical replication from one database to another within the same server. Ultimate

Re: [HACKERS] parallel.c oblivion of worker-startup failures

On Wed, Jan 24, 2018 at 5:25 PM, Thomas Munro wrote: > If there were some way for the postmaster to cause reason > PROCSIG_PARALLEL_MESSAGE to be set in the leader process instead of > just notification via kill(SIGUSR1) when it fails to fork a parallel > worker, we'd get (1) for free in any latch

Re: [HACKERS] Refactoring identifier checks to consistently use strcmp

On Wed, Jan 24, 2018 at 09:47:50AM +0100, Daniel Gustafsson wrote: > Attached is a rebased v7 patch which has your amendments (minus > propname) which passes make check without errors. Confirmed. I am switching the status as ready for committer for volatility-v7.patch then. > The volatility patch

Re: plpgsql function startup-time improvements

Hi 2017-12-27 21:38 GMT+01:00 Tom Lane : > Attached are patches for two performance-improvement ideas that came > to me while working on > https://www.postgresql.org/message-id/8962.1514399...@sss.pgh.pa.us > The three patches are logically independent and could be committed in > any order. But

Re: Is it valid to have logical replication between 2 databases on the same postgres server?

On 24 January 2018 at 22:41, Ryan Murphy wrote: > Hello hackers, > > I'm experimenting with Logical Replication. > (https://www.postgresql.org/docs/10/static/logical-replication.html) > What I'm trying to do may be impossible l but just wanted to ask here. > > I'm trying to do logical replication

Re: Is it valid to have logical replication between 2 databases on the same postgres server?

On 24/01/18 12:28, Craig Ringer wrote: > On 24 January 2018 at 22:41, Ryan Murphy > wrote: > > Hello hackers, > > I'm experimenting with Logical Replication. > (https://www.postgresql.org/docs/10/static/logical-replication.html >

Re: [PATCH] Logical decoding of TRUNCATE

I wonder whether this should be dealing with sequences at all. We are not currently publishing any information about sequences, so it seems weird to do it only here. Also, I'd imagine that if we ever get to publishing sequence events, then the sequence restarts would be published as independent e

Re: [PATCH] Logical decoding of TRUNCATE

On 24/01/18 13:50, Peter Eisentraut wrote: > I wonder whether this should be dealing with sequences at all. We are > not currently publishing any information about sequences, so it seems > weird to do it only here. Also, I'd imagine that if we ever get to > publishing sequence events, then the se

Re: [HACKERS][PATCH] Applying PMDK to WAL operations for persistent memory

On Tue, Jan 23, 2018 at 8:07 PM, Tsunakawa, Takayuki wrote: > From: Robert Haas [mailto:robertmh...@gmail.com] >> Oh, incidentally -- in our internal testing, we found that >> wal_sync_method=open_datasync was significantly faster than >> wal_sync_method=fdatasync. You might find that open_datasy

Re: Fix permissions check on pg_stat_get_wal_senders

On 5 January 2018 at 15:19, Simon Riggs wrote: > Yup, I got this. I saw the commit in the master branch but not in the REL_10_STABLE branch, I'm totally not up-to-date with the backpatching process, but I was wondering if it still needs to be added to REL_10_STABLE regards, Feike

Re: documentation is now XML

On 1/23/18 22:24, Bruce Momjian wrote: >> I like TeX as much as the next guy --- I wrote my thesis with it, >> a long time ago --- but there's no denying that (a) it's got hard >> limits we're approaching, (b) the downstream conversion to PDF is >> buggy, and (c) nobody is working on fixing it. >

Re: [HACKERS] Optional message to user when terminating/cancelling backend

> On 28 Sep 2017, at 14:55, Yugo Nagata wrote: > > On Sun, 3 Sep 2017 22:47:10 +0200 > Daniel Gustafsson wrote: > > I have reviewed your latest patch. Thanks! > I can apply this to the master branch and build this successfully, > and the behavior is as expected. > > However, here are some

Re: proposal: alternative psql commands quit and exit

Vik Fearing wrote: > I haven't applied the patch, but what happens if you combine these? > From your examples, I would imagine something like this: > > test=# select ' > test'# quit > Use \q to quit or press control-C to clear the input buffer. > test'# \q > test'# > > Surely that's not helpful.

Re: proposal: alternative psql commands quit and exit

On 24 January 2018 at 13:58, Alvaro Herrera wrote: > But there's a simple solution to that: make it two messages, one when > inside quotes ("Use Control-C to clear the input buffer, then \q to > quit") and one outside (current wording). Wasn't this already proposed > elsewhere in the thread? I d

Re: Failed to request an autovacuum work-item in silence

On Tue, Jan 23, 2018 at 11:44 PM, Masahiko Sawada wrote: > > On Tue, Jan 23, 2018 at 8:03 PM, Fabrízio de Royes Mello > wrote: > > > > Em ter, 23 de jan de 2018 às 03:36, Masahiko Sawada < sawada.m...@gmail.com> > > escreveu: > >> > >> Hi all, > >> > >> While reading the code, I realized that the

Re: Fix permissions check on pg_stat_get_wal_senders

On 24 January 2018 at 13:15, Feike Steenbergen wrote: > On 5 January 2018 at 15:19, Simon Riggs wrote: >> Yup, I got this. > > I saw the commit in the master branch but not in the REL_10_STABLE branch, > I'm totally not up-to-date with the backpatching process, but I was wondering > if it still n

Re: documentation is now XML

On Wed, Jan 24, 2018 at 08:21:21AM -0500, Peter Eisentraut wrote: > On 1/23/18 22:24, Bruce Momjian wrote: > >> I like TeX as much as the next guy --- I wrote my thesis with it, > >> a long time ago --- but there's no denying that (a) it's got hard > >> limits we're approaching, (b) the downstream

Re: documentation is now XML

On Tue, Jan 23, 2018 at 11:16:23PM -0500, Tom Lane wrote: > "David G. Johnston" writes: > > On Tuesday, January 23, 2018, Bruce Momjian wrote: > >> All agreed, but what alternatives are being developed? > > > I seem to recall a proposal a while back to gain margin on some of > > the limits by pr

Re: documentation is now XML

David Fetter wrote: > Speaking of legacy things, do we want to see about deprecating LaTeX > output for tables in psql? I haven't seen such a thing in production, > but maybe I'm missing something important. I've used it to prepare slides with the Beamer package. It's not a thing I do all that

Re: documentation is now XML

Alvaro Herrera writes: > David Fetter wrote: >> Speaking of legacy things, do we want to see about deprecating LaTeX >> output for tables in psql? I haven't seen such a thing in production, >> but maybe I'm missing something important. > I've used it to prepare slides with the Beamer package. I

Re: Failed to request an autovacuum work-item in silence

On Wed, Jan 24, 2018 at 12:31 PM, Fabrízio de Royes Mello < fabriziome...@gmail.com> wrote: > > > > On Tue, Jan 23, 2018 at 11:44 PM, Masahiko Sawada wrote: > > > > On Tue, Jan 23, 2018 at 8:03 PM, Fabrízio de Royes Mello > > wrote: > > > > > > Em ter, 23 de jan de 2018 às 03:36, Masahiko Sawada

Re: documentation is now XML

On Wed, Jan 24, 2018 at 10:06:49AM -0500, Tom Lane wrote: > Alvaro Herrera writes: > > David Fetter wrote: > >> Speaking of legacy things, do we want to see about deprecating LaTeX > >> output for tables in psql? I haven't seen such a thing in production, > >> but maybe I'm missing something impo

Re: [HACKERS] Optional message to user when terminating/cancelling backend

A quick suggestion from a passer-by -- would you put the new code in src/backend/storage/ipc/backend_signal.c instead? Sounds like a better place than utils/misc/; and "signal" is more general in nature than just "cancel". A bunch of stuff from utils/adt/misc.c (???) can migrate to the new file -

Re: documentation is now XML

David, Alvaro, * Alvaro Herrera (alvhe...@alvh.no-ip.org) wrote: > David Fetter wrote: > > > Speaking of legacy things, do we want to see about deprecating LaTeX > > output for tables in psql? I haven't seen such a thing in production, > > but maybe I'm missing something important. > > I've use

Re: documentation is now XML

On Wed, Jan 24, 2018 at 10:49:38AM -0500, Stephen Frost wrote: > David, Alvaro, > > * Alvaro Herrera (alvhe...@alvh.no-ip.org) wrote: > > David Fetter wrote: > > > > > Speaking of legacy things, do we want to see about deprecating > > > LaTeX output for tables in psql? I haven't seen such a thin

Re: documentation is now XML

On Wed, Jan 24, 2018 at 04:53:57PM +0100, David Fetter wrote: > On Wed, Jan 24, 2018 at 10:49:38AM -0500, Stephen Frost wrote: > > David, Alvaro, > > > > * Alvaro Herrera (alvhe...@alvh.no-ip.org) wrote: > > > David Fetter wrote: > > > > > > > Speaking of legacy things, do we want to see about de

Documentation update

Not sure if this is the correct place to report this, but a colleague and I discovered a problem with the systemd service file as described in the documentation. In the sample, there is a need for a line reading "After=syslogd.target network.target" under the [unit] tag. What we found on Redhat

Re: [HACKERS] Proposal: Local indexes for partitioned table

David Rowley wrote: > 10. You've added code to get_relation_info() to handle partitioned > indexes, but that code is skipped due to: > > /* > * Make list of indexes. Ignore indexes on system catalogs if told to. > * Don't bother with indexes for an inheritance parent, either. > */ > if (inhparen

Re: [HACKERS] Patch: Add --no-comments to skip COMMENTs with pg_dump

I wrote: > I went looking and realized that actually what we're interested in here > is the plpgsql extension, not the plpgsql language ... and in fact the > behavior I was thinking of is already there, except for some reason it's > only applied during binary upgrade. So I think we should give ser

Re: [PROPOSAL] Shared Ispell dictionaries

On Sat, Jan 13, 2018 at 06:22:41PM +0300, Arthur Zakirov wrote: > I think your proposals may be implemented in several patches, so they can > be applyed independently but consistently. I suppose I will prepare new > version of the patch with fixes and with initial design of new functions > and comm

Re:Re: [HACKERS] [PATCH]make pg_rewind to not copy useless WAL files

At 2018-01-23 09:56:48, "Stephen Frost" wrote: > >I've only read through the thread to try and understand what's going on >and the first thing that comes to mind is that you're changing >pg_rewind to not remove the WAL from before the divergence (split) >point, but I'm not sure why. As noted, t

Re: copy.c allocation constant

On 01/24/2018 04:14 AM, Bruce Momjian wrote: > On Tue, Nov 28, 2017 at 11:51:28AM -0500, Andrew Dunstan wrote: >> >> >> While reading copy.c I noticed this line: >> >> >> #define RAW_BUF_SIZE 65536        /* we palloc RAW_BUF_SIZE+1 bytes */ >> >> >> Doesn't that seem rather odd? If we're adding

Re: copy.c allocation constant

On Wed, Jan 24, 2018 at 06:25:01PM +0100, Tomas Vondra wrote: > > > > I think there there are two things to consider - aset.c and glibc. > > In AllocSet this is handled as over-sized chunk, that is chunk greater > than ALLOC_CHUNK_LIMIT (which ends up being 8kB). Which means it's > allocated as

Re: [HACKERS] Patch: Add --no-comments to skip COMMENTs with pg_dump

Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: > I wrote: > > I went looking and realized that actually what we're interested in here > > is the plpgsql extension, not the plpgsql language ... and in fact the > > behavior I was thinking of is already there, except for some reason it's > > only applie

Re: [HACKERS] [PATCH]make pg_rewind to not copy useless WAL files

Greetings, * chenhj (chjis...@163.com) wrote: > > At 2018-01-23 09:56:48, "Stephen Frost" wrote: > >I've only read through the thread to try and understand what's going on > >and the first thing that comes to mind is that you're changing > >pg_rewind to not remove the WAL from before the diverge

Re: [PROPOSAL] Shared Ispell dictionaries

Hi, On 01/24/2018 06:20 PM, Arthur Zakirov wrote: > On Sat, Jan 13, 2018 at 06:22:41PM +0300, Arthur Zakirov wrote: >> I think your proposals may be implemented in several patches, so >> they can be applyed independently but consistently. I suppose I >> will prepare new version of the patch with f

Re: Is it valid to have logical replication between 2 databases on the same postgres server?

Thanks all! I'll try creating the replication slot manually.

Re: [HACKERS] Patch: Add --no-comments to skip COMMENTs with pg_dump

Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> In further testing of that, I noticed that it made the behavior of our >> other bugaboo, the public schema, rather inconsistent. With this >> builtin-extensions hack, the plpgsql extension doesn't get dumped, >> whether or not you

Re: pgsql: Add parallel-aware hash joins.

On Tue, Jan 23, 2018 at 6:10 PM, Tom Lane wrote: > Looking more closely at the shorter series, there are four pretty obvious > step changes since 2016-09. The PNG's x-axis doesn't have enough > resolution to match these up to commits, but looking at the underlying > data, they clearly correspond

Re: copy.c allocation constant

On Wed, Jan 24, 2018 at 12:25 PM, Tomas Vondra wrote: > At the glibc level ... I'm not so sure. AFAIK glibc uses an allocator > with similar ideas (freelists, ...) so hopefully it's fine too. > > And then there are the systems without glibc, or with other libc > implementations. No idea about thos

Re: pgsql: Add parallel-aware hash joins.

On 2018-01-23 14:24:56 -0500, Robert Haas wrote: > Right, but this doesn't seem to show any big spike in the runtime at > the time when parallel hash was committed, or when the preparatory > patch to add test coverage for hash joins got committed. Rather, > there's a gradual increase over time. E

Re: Would a BGW need shmem_access or database_connection to enumerate databases?

On Thu, Dec 14, 2017 at 06:12:35PM -0500, Chapman Flack wrote: > On 12/04/2017 09:13 AM, Craig Ringer wrote: > > On 1 December 2017 at 23:04, Chapman Flack wrote: > >> Can I call RegisterDynamicBackgroundWorker when not in the postmaster, > >> but also not in a "regular backend", but rather anothe

Re: FOR EACH ROW triggers on partitioned tables

On Tue, Jan 23, 2018 at 5:10 PM, Alvaro Herrera wrote: > The main question is this: when running the trigger function, it is > going to look as it is running in the context of the partition, not in > the context of the parent partitioned table (TG_RELNAME etc). That > seems mildly ugly: some user

Re: [HACKERS] Patch: Add --no-comments to skip COMMENTs with pg_dump

Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> In further testing of that, I noticed that it made the behavior of our > >> other bugaboo, the public schema, rather inconsistent. With this > >> builtin-extensions hack, the plpgs

Re: PATCH: Exclude unlogged tables from base backups

Hi Masahiko, Thanks for the review! On 1/22/18 3:14 AM, Masahiko Sawada wrote: > On Thu, Dec 14, 2017 at 11:58 PM, Robert Haas wrote: >> >> We would also have a problem if the missing file caused something in >> recovery to croak on the grounds that the file was expected to be >> there, but I do

Re: pgsql: Add parallel-aware hash joins.

Hi, On 2018-01-24 13:11:22 -0500, Robert Haas wrote: > So for me, the additional hash index tests don't cost anything > measurable and the additional hash join tests cost about a second. I > think this probably accounts for why committers other than you keep > "adding so much time to the regressi

Re: pgindent run?

On 2018-01-23 22:22:47 -0500, Bruce Momjian wrote: > On Tue, Nov 28, 2017 at 04:38:12PM -0500, Tom Lane wrote: > > Thomas Munro writes: > > > On Wed, Nov 29, 2017 at 9:47 AM, Tom Lane wrote: > > >> I think that'd be taking it too far, especially given that the dependency > > >> on a typedefs list

Re: [PROPOSAL] Shared Ispell dictionaries

2018-01-24 20:57 GMT+03:00 Tomas Vondra : > > Thanks. I don't have time to review/test this before FOSDEM, but a > couple of comments regarding some of the points you mentioned. > Thank you for your thoughts. > > I thought about it. And it seems to me that we can use functions > > ts_unload() an

Re: copy.c allocation constant

On 2018-01-24 13:19:19 -0500, Robert Haas wrote: > On Wed, Jan 24, 2018 at 12:25 PM, Tomas Vondra > wrote: > > At the glibc level ... I'm not so sure. AFAIK glibc uses an allocator > > with similar ideas (freelists, ...) so hopefully it's fine too. > > > > And then there are the systems without gl

Re: Would a BGW need shmem_access or database_connection to enumerate databases?

Thanks! I had actually registered that one (with a related one) for CF 2018-03, having missed the deadline for -01: https://commitfest.postgresql.org/17/1467/ -Chap On 01/24/2018 01:20 PM, Bruce Momjian wrote: > On Thu, Dec 14, 2017 at 06:12:35PM -0500, Chapman Flack wrote: >> On 12/04/2017 09:1

Re: [HACKERS] Patch: Add --no-comments to skip COMMENTs with pg_dump

Stephen Frost writes: > I'm afraid we may still get some push-back from existing users of > --clean since, with the change you're proposing, we wouldn't be cleaning > up anything that's been done to the public schema when it comes to > comment changes or ACL changes, right? No, if you have a nond

Re: WIP Patch: Precalculate stable functions, infrastructure v1

Hi, On 2018-01-16 17:05:01 -0500, Tom Lane wrote: > [ I'm sending this comment separately because I think it's an issue > Andres might take an interest in. ] Thanks for that. I indeed am interested. Sorry for the late response, was very deep into the JIT patch. > Marina Polyakova writes: > >

Re: Would a BGW need shmem_access or database_connection to enumerate databases?

On Wed, Jan 24, 2018 at 01:48:05PM -0500, Chapman Flack wrote: > Thanks! I had actually registered that one (with a related one) > for CF 2018-03, having missed the deadline for -01: > > https://commitfest.postgresql.org/17/1467/ OK, thanks. I added a commitfest item annotiation to say that the

Re: copy.c allocation constant

On Wed, Jan 24, 2018 at 1:43 PM, Andres Freund wrote: > Indeed. Don't think RAW_BUF_SIZE is quite big enough for that on most > platforms though. From man mallopt: > Balancing these factors leads to a default setting of 128*1024 for the > M_MMAP_THRESHOLD parameter. > Additionally, even when

Re: Foreign keys and partitioned tables

Alvaro Herrera wrote: > This patch enables foreign key constraints to and from partitioned > tables. This version is rebased on current master. 0001: fix for a get_relation_info bug in current master. Posted in <20180124174134.ma4ui2kczmqwb4um@alvherre.pgsql> 0002: Allows local partitioned

Re: [HACKERS] Proposal: Local indexes for partitioned table

I think this is the right fix for this problem. I wonder about exploring other callers of RelationGetIndexList to see who else could be confused ... -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services >From 53506fd3a

Re: [HACKERS] Patch: Add --no-comments to skip COMMENTs with pg_dump

Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > I'm afraid we may still get some push-back from existing users of > > --clean since, with the change you're proposing, we wouldn't be cleaning > > up anything that's been done to the public schema when it comes to > > comment

Re: pgsql: Add parallel-aware hash joins.

Andres Freund writes: > On 2018-01-24 13:11:22 -0500, Robert Haas wrote: >> Now, how much should we care about the performance of software with a >> planned release date of 2018 on hardware discontinued in 2001, >> hardware that is apparently about 20 times slower than a modern >> laptop? Some, p

Re: pgindent run?

Andres Freund writes: > FWIW, I think this problem could just as well be addressed with a few > printing heuristics instead of actually needing an actual list of > typedefs. Step right up and implement that, and we'd all be happier. Certainly the typedefs list is a pain in the rear. > There'd b

Re: pgindent run?

On January 24, 2018 11:34:07 AM PST, Tom Lane wrote: >Andres Freund writes: >> There'd be one or two edge cases of bad formatting, but the >> end result would be far less painful than what we have today, were >> basically nobody can format their patches without a lot of manual >> cherry-picking

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Tue, Jan 23, 2018 at 9:43 PM, Amit Kapila wrote: > Right, but what if the worker dies due to something proc_exit(1) or > something like that before calling BarrierArriveAndWait. I think this > is part of the problem we have solved in > WaitForParallelWorkersToFinish such that if the worker exi

Re: copy.c allocation constant

On 2018-01-24 14:25:37 -0500, Robert Haas wrote: > On Wed, Jan 24, 2018 at 1:43 PM, Andres Freund wrote: > > Indeed. Don't think RAW_BUF_SIZE is quite big enough for that on most > > platforms though. From man mallopt: > > Balancing these factors leads to a default setting of 128*1024 for the

Re: pgsql: Add parallel-aware hash joins.

I wrote: > I find that to be a completely bogus straw-man argument. The point of > looking at the prairiedog time series is just to see a data series in > which the noise level is small enough to discern the signal. If anyone's > got years worth of data off a more modern machine, and they can ext

Re: [HACKERS] parallel.c oblivion of worker-startup failures

On Wed, Jan 24, 2018 at 1:57 AM, Thomas Munro wrote: > On Wed, Jan 24, 2018 at 5:25 PM, Thomas Munro > wrote: >> If there were some way for the postmaster to cause reason >> PROCSIG_PARALLEL_MESSAGE to be set in the leader process instead of >> just notification via kill(SIGUSR1) when it fails to

Re: [HACKERS] parallel.c oblivion of worker-startup failures

On Tue, Jan 23, 2018 at 9:45 PM, Amit Kapila wrote: > Hmm, I think that case will be addressed because tuple queues can > detect if the leader is not attached. It does in code path > shm_mq_receive->shm_mq_counterparty_gone. In > shm_mq_counterparty_gone, it can detect if the worker is gone by u

Re: copy.c allocation constant

Andres Freund wrote: > On 2018-01-24 14:25:37 -0500, Robert Haas wrote: > > On Wed, Jan 24, 2018 at 1:43 PM, Andres Freund wrote: > > > Indeed. Don't think RAW_BUF_SIZE is quite big enough for that on most > > > platforms though. From man mallopt: > > > Balancing these factors leads to a defa

Re: pgsql: Add parallel-aware hash joins.

Hi, On 2018-01-24 14:31:47 -0500, Tom Lane wrote: > However ... if you spend any time looking at the behavior of that, > the hashjoin tests are still problematic. I think my main problem with your arguments is that you basically seem to say that one of the more complex features in postgres can't

Re: WIP Patch: Precalculate stable functions, infrastructure v1

Andres Freund writes: > On 2018-01-16 17:05:01 -0500, Tom Lane wrote: >> I'm curious to know whether Andres has some other ideas, or whether he >> feels this is all a big wart on the compiled-expression concept. > I don't have too many "artistic" concerns from the compiled expression > POV. The b

Re: copy.c allocation constant

Hi, On 2018-01-24 17:07:01 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > glibc's malloc does add a header. My half-informed suspicion is that > > most newer malloc backing allocators will have a header, because > > maintaining a shared lookup-by-address table is pretty expensive to > > m

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Thu, Jan 25, 2018 at 8:54 AM, Peter Geoghegan wrote: > I have used Thomas' chaos-monkey-fork-process.patch to verify: > > 1. The problem of fork failure causing nbtsort.c to wait forever is a > real problem. Sure enough, the coding pattern within > _bt_leader_heapscan() can cause us to wait for

Re: pgsql: Add parallel-aware hash joins.

Andres Freund wrote: > Besides larger groups, starting the next test(s) earlier, another way to > gain pretty large improvements would be a test schedule feature that > allowed to stat dependencies between tests. So instead of manually > grouping the schedule, have 'numerology' state that it depen

Re: pgsql: Add parallel-aware hash joins.

Thomas Munro wrote: > On Wed, Jan 24, 2018 at 12:10 PM, Tom Lane wrote: > > However, the trend over the last two months is very bad, and I do > > not think that we can point to any large improvement in test > > coverage that someone committed since November. > > I'm not sure if coverage.postgres

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Wed, Jan 24, 2018 at 12:13 PM, Thomas Munro wrote: > On Thu, Jan 25, 2018 at 8:54 AM, Peter Geoghegan wrote: >> I have used Thomas' chaos-monkey-fork-process.patch to verify: >> >> 1. The problem of fork failure causing nbtsort.c to wait forever is a >> real problem. Sure enough, the coding pa

Re: copy.c allocation constant

On Thu, Jan 25, 2018 at 7:19 AM, Robert Haas wrote: > On Wed, Jan 24, 2018 at 12:25 PM, Tomas Vondra > wrote: >> At the glibc level ... I'm not so sure. AFAIK glibc uses an allocator >> with similar ideas (freelists, ...) so hopefully it's fine too. >> >> And then there are the systems without gl

Re: copy.c allocation constant

On Thu, Jan 25, 2018 at 09:30:54AM +1300, Thomas Munro wrote: > On Thu, Jan 25, 2018 at 7:19 AM, Robert Haas wrote: > > On Wed, Jan 24, 2018 at 12:25 PM, Tomas Vondra > > wrote: > >> At the glibc level ... I'm not so sure. AFAIK glibc uses an allocator > >> with similar ideas (freelists, ...) so

Re: pgsql: Add parallel-aware hash joins.

Alvaro Herrera writes: > Andres Freund wrote: >> Besides larger groups, starting the next test(s) earlier, another way to >> gain pretty large improvements would be a test schedule feature that >> allowed to stat dependencies between tests. So instead of manually >> grouping the schedule, have 'nu

Re: WIP Patch: Precalculate stable functions, infrastructure v1

On 2018-01-24 15:10:56 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2018-01-16 17:05:01 -0500, Tom Lane wrote: > >> I'm curious to know whether Andres has some other ideas, or whether he > >> feels this is all a big wart on the compiled-expression concept. > > > I don't have too many "art

Re: pgsql: Add parallel-aware hash joins.

Hi, On 2018-01-24 17:18:26 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > Besides larger groups, starting the next test(s) earlier, another way to > > gain pretty large improvements would be a test schedule feature that > > allowed to stat dependencies between tests. So instead of manuall

Re: copy.c allocation constant

On Thu, Jan 25, 2018 at 9:35 AM, Bruce Momjian wrote: > The BSD memory allocator used to allocate in powers of two, and keep the > header in a separate location. They did this so they could combine two > free, identically-sized memory blocks into a single one that was double > the size. I have n

Re: pgsql: Add parallel-aware hash joins.

Hi, On 2018-01-24 15:36:35 -0500, Tom Lane wrote: > There'd be a lot of followup work to sanitize the tests better. For > instance, if two tests transiently create tables named "foo", it doesn't > matter as long as they're not in the same group. It would matter with > this. Right. I suspect we'

Re: [HACKERS] Patch: Add --no-comments to skip COMMENTs with pg_dump

Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> No, if you have a nondefault ACL, that will still get applied. This >> arrangement would drop comment changes, but I can't get excited about >> that; it's certainly far less of an inconvenience in that scenario >> than dumping the

Re: [HACKERS] Patch: Add --no-comments to skip COMMENTs with pg_dump

Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> No, if you have a nondefault ACL, that will still get applied. This > >> arrangement would drop comment changes, but I can't get excited about > >> that; it's certainly far less of

Re: pgsql: Add parallel-aware hash joins.

Andres Freund writes: > On 2018-01-24 17:18:26 -0300, Alvaro Herrera wrote: >> Yeah, I proposed this a decade ago but never had the wits to write the >> code. > It shouldn't be too hard, right? Leaving defining the file format, > parsing it, creating the new schedule with depencencies and adaptin

Re: pgsql: Add parallel-aware hash joins.

Andres Freund writes: > On 2018-01-24 15:36:35 -0500, Tom Lane wrote: >> I'm concerned that we'd end up with a higher number of irreproducible >> test failures with no good way to investigate them. > Hm. We probably should dump the used ordering of tests somwhere upon > failure, to make it easier

Re: PATCH: Exclude unlogged tables from base backups

>> If a new unlogged relation is created after constructed the >> unloggedHash before sending file, we cannot exclude such relation. It >> would not be problem if the taking backup is not long because the new >> unlogged relation unlikely becomes so large. However, if takeing a >> backup takes a lo

Re: pgsql: Add parallel-aware hash joins.

Hi, On 2018-01-24 15:58:16 -0500, Tom Lane wrote: > Yeah. We already have topo sort code in pg_dump, maybe we could push that > into someplace like src/common or src/fe_utils? Although pg_dump hasn't > got any need for edge weights, so maybe sharing code isn't worth it. I suspect it may be more

Re: pgsql: Add parallel-aware hash joins.

On Wed, Jan 24, 2018 at 2:31 PM, Tom Lane wrote: > I find that to be a completely bogus straw-man argument. The point of > looking at the prairiedog time series is just to see a data series in > which the noise level is small enough to discern the signal. If anyone's > got years worth of data of

Re: pgsql: Add parallel-aware hash joins.

On Wed, Jan 24, 2018 at 4:01 PM, Tom Lane wrote: > The progress-display output of pg_regress would need a complete rethink > anyhow. First thought is to emit two lines per test, one when we > launch it and one when it finishes and we check the results: > > foreign_data: launched > ... > foreign_d

Re: PATCH: Exclude unlogged tables from base backups

> I agree with #1 and feel the updated docs are reasonable and > sufficient to address this case for now. > > I have retested these patches against master at d6ab720360. > > All test succeed. > > Marking "Ready for Committer". Actually, marked it "Ready for Review" to wait for Masahiko to comment/

Re: [HACKERS] Patch: Add --no-comments to skip COMMENTs with pg_dump

Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> Fair point, but doesn't it apply equally to non-default ACLs on any >> other system objects? If you tweaked the permissions on say pg_ls_dir(), >> then dump, then tweak them some more, you're going to get uncertain >> results if yo

Re: PATCH: Exclude unlogged tables from base backups

On 1/24/18 4:02 PM, Adam Brightwell wrote: >>> If a new unlogged relation is created after constructed the >>> unloggedHash before sending file, we cannot exclude such relation. It >>> would not be problem if the taking backup is not long because the new >>> unlogged relation unlikely becomes so la

Re: JIT compiling with LLVM v9.0

On Wednesday, January 24, 2018 8:20:38 AM CET Andres Freund wrote: > As the patchset is large (500kb) and I'm still quickly evolving it, I do > not yet want to attach it. The git tree is at > https://git.postgresql.org/git/users/andresfreund/postgres.git > in the jit branch > > https://git.post

  1   2   >