Re: [HACKERS] parallel query vs extensions

2016-04-16 Thread Craig Ringer
On 15 April 2016 at 12:45, Jeff Janes wrote: > I think there are a lot of extensions which create functions which > could benefit from being declared parallel safe. But how does one go > about doing that? > > create extension xml2; > select xml_valid(filler),count(*) from pgbench_accounts group

[HACKERS] Pgbench with -f and -S

2016-04-16 Thread Robins Tharakan
Hi, Pgbench allows -f and -S combinations together where the doc says that -S effectively uses the internal select-only script. Is it okay to assume that -f is disregarded here? Or are they run in round-robin fashion (although then, how does it know which read-only part of my script to run?) or s

Re: [HACKERS] Spinlocks and semaphores in 9.2 and 9.3

2016-04-16 Thread Andres Freund
On April 16, 2016 6:02:39 PM PDT, Tom Lane wrote: >I wrote: >> So at this point I'm not sure what to do. I could back out the >back-patch >> of 44cd47c1d49655c5, which would mean accepting that 9.2/9.3 are >broken >> and will never be fixed for HPPA, as well as any other architectures >that >>

Re: [HACKERS] Spinlocks and semaphores in 9.2 and 9.3

2016-04-16 Thread Tom Lane
I wrote: > So at this point I'm not sure what to do. I could back out the back-patch > of 44cd47c1d49655c5, which would mean accepting that 9.2/9.3 are broken > and will never be fixed for HPPA, as well as any other architectures that > use the same fallback memory barrier implementation. The lac

Re: [HACKERS] [BUGS] Breakage with VACUUM ANALYSE + partitions

2016-04-16 Thread Noah Misch
On Sat, Apr 16, 2016 at 03:28:23PM -0700, Andres Freund wrote: > On 2016-04-16 18:23:01 -0400, Noah Misch wrote: > > On Tue, Apr 12, 2016 at 09:00:57AM -0400, Robert Haas wrote: > > > On Mon, Apr 11, 2016 at 1:17 PM, Andres Freund wrote: > > > > I can think of a number of relatively easy ways to a

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-16 Thread Andres Freund
Hi, On 2016-04-16 18:27:06 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2016-04-16 17:52:44 -0400, Tom Lane wrote: > >> That's more than a 5X penalty, which seems like it would make the > >> feature unusable; unless there is an argument that that's an extreme > >> case that wouldn't be r

Re: [HACKERS] [BUGS] Breakage with VACUUM ANALYSE + partitions

2016-04-16 Thread Andres Freund
On 2016-04-16 18:23:01 -0400, Noah Misch wrote: > On Tue, Apr 12, 2016 at 09:00:57AM -0400, Robert Haas wrote: > > On Mon, Apr 11, 2016 at 1:17 PM, Andres Freund wrote: > > > I can think of a number of relatively easy ways to address this: > > > 1) Just zap (or issue?) all pending flush requests w

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-16 Thread Tom Lane
Andres Freund writes: > On 2016-04-16 17:52:44 -0400, Tom Lane wrote: >> That's more than a 5X penalty, which seems like it would make the >> feature unusable; unless there is an argument that that's an extreme >> case that wouldn't be representative of most real-world usage. >> Which there may we

Re: [HACKERS] [BUGS] Breakage with VACUUM ANALYSE + partitions

2016-04-16 Thread Noah Misch
On Tue, Apr 12, 2016 at 09:00:57AM -0400, Robert Haas wrote: > On Mon, Apr 11, 2016 at 1:17 PM, Andres Freund wrote: > > I can think of a number of relatively easy ways to address this: > > 1) Just zap (or issue?) all pending flush requests when getting an > >smgrinval/smgrclosenode > > 2) Do

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-16 Thread Andres Freund
On 2016-04-16 17:52:44 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2016-04-16 16:44:52 -0400, Noah Misch wrote: > >> That is more controversial than the potential ~2% regression for > >> old_snapshot_threshold=-1. Alvaro[2] and Robert[3] are okay releasing > >> that way, and Andres[4] i

Re: [HACKERS] Memory leak in GIN index build

2016-04-16 Thread Julien Rouhaud
On 16/04/2016 20:45, Tom Lane wrote: > Julien Rouhaud writes: > >> Also, in dataPlaceToPageLeaf() and ginVacuumPostingTreeLeaf(), shouldn't >> the START_CRIT_SECTION() calls be placed before the xlog code? > > Yeah, they should. Evidently somebody kluged it to avoid doing a palloc > inside a cr

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-16 Thread Tom Lane
Andres Freund writes: > On 2016-04-16 16:44:52 -0400, Noah Misch wrote: >> That is more controversial than the potential ~2% regression for >> old_snapshot_threshold=-1. Alvaro[2] and Robert[3] are okay releasing >> that way, and Andres[4] is not. > FWIW, I could be kinda convinced that it's tem

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-16 Thread Andres Freund
On 2016-04-16 16:44:52 -0400, Noah Misch wrote: > That is more controversial than the potential ~2% regression for > old_snapshot_threshold=-1. Alvaro[2] and Robert[3] are okay releasing > that way, and Andres[4] is not. FWIW, I could be kinda convinced that it's temporarily ok, if there'd be a c

[HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-16 Thread Noah Misch
On Wed, Apr 13, 2016 at 03:21:31PM -0500, Kevin Grittner wrote: > If 2201d801 was not included in your -1 tests, have you identified > where the 2% extra run time is going on -1 versus reverted? Since > several other threads lately have reported bigger variation than > that based on random memory

Re: [HACKERS] Memory leak in GIN index build

2016-04-16 Thread Tom Lane
Julien Rouhaud writes: > After some digging, the leak comes from walbufbegin palloc in > registerLeafRecompressWALData(). > IIUC, walbufbegin isn't pfree-d and can't be before XLogInsert() is > called, which happens in ginPlaceToPage(). Hmm. > I don't see a simple way to fix that. My first idea

[HACKERS] Memory leak in GIN index build

2016-04-16 Thread Julien Rouhaud
Hello, Another colleague provided a report of memory leak, during a GIN index build. Test case to reproduce the attached (need to create a gin index on the val column after loading). Sorry, it generates a 24GB table, and memory start leaking with a 1GB maintenance_work_mem after reaching 8 or 9 ti

[HACKERS] Spinlocks and semaphores in 9.2 and 9.3

2016-04-16 Thread Tom Lane
This rabbit hole keeps getting deeper and deeper :-( I realized a couple days ago that it had been close to three years since I last tried building the further-back branches on my ancient HPPA box. Not so surprisingly, things were broken: commits 37de8de9e33606a0 et al had introduced use of memory

Re: [HACKERS] Updated backup APIs for non-exclusive backups

2016-04-16 Thread Magnus Hagander
On Wed, Apr 13, 2016 at 4:07 AM, Noah Misch wrote: > On Tue, Apr 12, 2016 at 10:08:23PM +0200, Magnus Hagander wrote: > > On Tue, Apr 12, 2016 at 8:39 AM, Noah Misch wrote: > > > On Mon, Apr 11, 2016 at 11:22:27AM +0200, Magnus Hagander wrote: > > > > Well, if we *don't* do the rewrite before we

Re: [HACKERS] Disallow unique index on system columns

2016-04-16 Thread Tom Lane
David Rowley writes: > On 15 April 2016 at 13:43, David Rowley wrote: >> The attached patch just disallows any index containing a system >> column, apart from OID. > Seems I only did half the job as I forgot to think to check for system > columns that are hidden away inside expressions or predica

[HACKERS] OS scheduler bugs affecting high-concurrency contention

2016-04-16 Thread Kevin Grittner
There is a paper that any one interested in performance at high concurrency, especially in Linux, should read[1]. While doing other work, a group of researchers saw behavior that they suspected was due to scheduler bugs in Linux. There were no tools that made proving that practical, so they devel

Re: [HACKERS] Refactor pg_dump as a library?

2016-04-16 Thread Andreas Karlsson
On 04/14/2016 07:28 PM, David Steele wrote: As far as I know pg_dump share locks everything before it starts so there shouldn't be issues with concurrent DDL. Try creating a new inherited table with FKs, etc. during a pg_dump and you'll see lots of fun lock waits. I am pretty sure that it does

Re: [HACKERS] Small fix: avoid passing null pointers to memcpy()

2016-04-16 Thread Piotr Stefaniak
On 2016-04-03 09:24, Piotr Stefaniak wrote: from running the regression test suite (including TAP tests) and also sqlsmith, I've got a couple of places where UBSan reported calls to memcpy() with null pointer passed as either source or destination. Patch attached. Patch updated. Since this ti

Re: [HACKERS] Detrimental performance impact of ringbuffers on performance

2016-04-16 Thread Amit Kapila
On Thu, Apr 14, 2016 at 10:22 AM, Peter Geoghegan wrote: > > On Tue, Apr 12, 2016 at 11:38 AM, Andres Freund wrote: > >> And, on the other hand, if we don't do something like that, it will be > >> quite an exceptional case to find anything on the free list. Doing it > >> just to speed up develop

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-16 Thread Amit Kapila
On Fri, Apr 15, 2016 at 11:30 AM, Kyotaro HORIGUCHI < horiguchi.kyot...@lab.ntt.co.jp> wrote: > > At Fri, 15 Apr 2016 08:52:56 +0530, Amit Kapila wrote : > > > > How about if we do all the parsing stuff in temporary context and then copy > > the results using TopMemoryContext? I don't think it wi