Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

On Wed, Apr 13, 2016 at 11:24 AM, Etsuro Fujita wrote: > On 2016/04/13 3:14, Robert Haas wrote: >> >> I'm wondering why we are fixing this specific case and not any of the >> other calls to PQexec() or PQexecParams() in postgres_fdw.c. >> >> I mean, many of those instances are cases where the quer

Re: [HACKERS] Performance degradation in commit 6150a1b0

On Tue, Apr 12, 2016 at 10:30 PM, Noah Misch wrote: > That sounds like this open item is ready for CLOSE_WAIT status; is it? I just retested this on power2. Here are the results. I retested 3fed4174 and 6150a1b0 plus master as of deb71fa9. 5-minute pgbench -S runs, scale factor 300, with predi

Re: [HACKERS] raw output from copy

2016-04-12 22:48 GMT+02:00 Tom Lane : > Pavel Stehule writes: > > I had a idea about additional options of COPY RAW statements. One can be > > CAST function. These CAST functions can be used to any for any format. > > Uh, what? CAST() is not about external representations of values, and > overlo

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

On 2016-04-12 23:52:14 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > > I'm kinda inclined to apply that portion (or just the whole patch with > > the spurious #ifdef 0 et al fixed) into 9.6; and add the necessary > > checks in a few places. Because I really think this is likely to hit >

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

Andres Freund wrote: > I'm kinda inclined to apply that portion (or just the whole patch with > the spurious #ifdef 0 et al fixed) into 9.6; and add the necessary > checks in a few places. Because I really think this is likely to hit > unsuspecting users. !!! Be sure to consult with the RMT befo

Re: [HACKERS] Performance degradation in commit 6150a1b0

On Tue, Apr 12, 2016 at 05:36:07PM -0400, Robert Haas wrote: > So the current status of this issue is: > > 1. Andres committed a patch (008608b9d51061b1f598c197477b3dc7be9c4a64) > to reduce the size of an LWLock by an amount equal to the size of a > mutex (modulo alignment). > > 2. Andres also co

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

On 2016/04/13 3:14, Robert Haas wrote: I'm wondering why we are fixing this specific case and not any of the other calls to PQexec() or PQexecParams() in postgres_fdw.c. I mean, many of those instances are cases where the query isn't likely to run for very long, but certainly "FETCH %d FROM c%u"

Re: [HACKERS] Updated backup APIs for non-exclusive backups

On Tue, Apr 12, 2016 at 10:08:23PM +0200, Magnus Hagander wrote: > On Tue, Apr 12, 2016 at 8:39 AM, Noah Misch wrote: > > On Mon, Apr 11, 2016 at 11:22:27AM +0200, Magnus Hagander wrote: > > > Well, if we *don't* do the rewrite before we release it, then we have to > > > instead put information ab

Re: [HACKERS] Parallel Queries and PostGIS

On 1 April 2016 at 17:12, David Rowley wrote: > On 30 March 2016 at 09:14, Robert Haas wrote: >> On Tue, Mar 29, 2016 at 3:48 PM, Paul Ramsey >> wrote: I have no idea why the cost adjustments that you need are different for the scan case and the aggregate case. That does seem problem

Re: [HACKERS] Why doesn't src/backend/port/win32/socket.c implement bind()?

Magnus Hagander writes: > On Mon, Jan 11, 2016 at 6:19 AM, Amit Kapila > wrote: >> On Sun, Jan 10, 2016 at 11:55 PM, Tom Lane wrote: >>> I think the reason why we're getting "No error" instead of a useful >>> strerror report is that socket.c doesn't provide an implementation >>> of bind() that i

Re: [HACKERS] Detrimental performance impact of ringbuffers on performance

Robert, Andres, * Andres Freund (and...@anarazel.de) wrote: > On 2016-04-12 14:29:10 -0400, Robert Haas wrote: > > On Wed, Apr 6, 2016 at 6:57 AM, Andres Freund wrote: > > That does not seem like a good idea from here. One of the ideas I > > still want to explore at some point is having a backgr

Re: [HACKERS] fd.c: flush data problems on osx

Michael Paquier writes: > On Mon, Mar 21, 2016 at 9:09 PM, Stas Kelvich > wrote: >> On 21 Mar 2016, at 14:53, Andres Freund wrote: >>> I think we still need to fix the mmap() implementation to support the >>> offset = 0, nbytes = 0 case (via fseek(SEEK_END). >> It is already in this diff. I’

[HACKERS] Pglogical questions and problems

Hello, Alright -- here we go PostgreSQL 9.5.2, Ubuntu Trusty, All packages from apt.postgresql.org except PgLogical which is from 2Q: I have the following setup: Origin0->Replica0 Table: logical_test(id bigserial primary key) Origin0: SELECT pglogical.create_node( node_name := 'origin',

Re: [HACKERS] Choosing parallel_degree

On 11/04/2016 22:53, Julien Rouhaud wrote: > On 11/04/2016 17:44, Robert Haas wrote: >> >> We should probably add the number of workers actually obtained to the >> EXPLAIN ANALYZE output. That's been requested before. >> > > If it's not too late for 9.6, it would be very great. > Just in case I

Re: [HACKERS] Speedup twophase transactions

On Wed, Apr 13, 2016 at 1:53 AM, Stas Kelvich wrote: >> On 12 Apr 2016, at 15:47, Michael Paquier wrote: >> >> It looks to be the case... The PREPARE phase replayed after the >> standby is restarted in recovery creates a series of exclusive locks >> on the table created and those are not taken on

Re: [HACKERS] Parallel Aggregate costs don't consider combine/serial/deserial funcs

On 13 April 2016 at 08:52, Robert Haas wrote: > On Sun, Apr 10, 2016 at 8:47 AM, David Rowley > wrote: >> I realised a few days ago that the parallel aggregate code does not >> cost for the combine, serialisation and deserialisation functions at >> all. > > Oops. > >> I've attached a patch which

Re: [HACKERS] Performance degradation in commit 6150a1b0

On Thu, Mar 31, 2016 at 10:13 AM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Mar 31, 2016 at 6:45 AM, Andres Freund wrote: >>> On 2016-03-31 06:43:19 -0400, Robert Haas wrote: To which proposal are you referring? > >>> 1) in >>> http://www.postgresql.org/message-id/20160328130904.4mh

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

On Tue, Apr 12, 2016 at 2:04 PM, Robert Haas wrote: > On Tue, Apr 12, 2016 at 4:32 PM, David G. Johnston > wrote: > > I give a solid +10 to Robert's opinions on the matter and aside from > > figuring out if and how to fit first-number versioning dynamics into our > > release policies I think the

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

On Tue, Apr 12, 2016 at 4:32 PM, David G. Johnston wrote: > I give a solid +10 to Robert's opinions on the matter and aside from > figuring out if and how to fit first-number versioning dynamics into our > release policies I think the community is doing a sufficient job on the > communication and

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

On Tue, Apr 12, 2016 at 4:07 PM, Oleg Bartunov wrote: > Our roadmap http://www.postgresql.org/developer/roadmap/ is the problem. We > don't have clear roadmap and that's why we cannot plan future feature full > release. There are several postgres-centric companies, which have most of > developers,

Re: [HACKERS] Parallel Aggregate costs don't consider combine/serial/deserial funcs

On Sun, Apr 10, 2016 at 8:47 AM, David Rowley wrote: > I realised a few days ago that the parallel aggregate code does not > cost for the combine, serialisation and deserialisation functions at > all. Oops. > I've attached a patch which fixes this. I've committed this patch. I wonder if it's g

Re: [HACKERS] raw output from copy

Pavel Stehule writes: > I had a idea about additional options of COPY RAW statements. One can be > CAST function. These CAST functions can be used to any for any format. Uh, what? CAST() is not about external representations of values, and overloading it for that purpose doesn't seem like a part

Re: [HACKERS] Problems with huge_pages and IBM Power8

reiner peterke writes: >> On Apr 12, 2016, at 10:20 PM, Andres Freund wrote: >> I've a bit of a hard time believing that this is related to huge pages. > Well all i have at the moment is that when we disabled huge pages on the > kernel level and then restarted postgres there were no additional

Re: [HACKERS] Problems with huge_pages and IBM Power8

> On Apr 12, 2016, at 10:20 PM, Andres Freund wrote: > > On 2016-04-12 21:58:14 +0200, reiner peterke wrote: >> Hi >> >> We have been doing some testing with Postgres (9.5.2) compiled on a Power8 >> running Centos 7 >> >> When working with huge_pages, we initially got this error. >> >> munma

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

On Tue, Apr 12, 2016 at 1:07 PM, Oleg Bartunov wrote: > > >> * we don't *know* that any of the above items will require a backwards >> compatibility break. >> >> People keep talking about "we might want to break compatibility/file >> format one day". But nobody is working on anything which will

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

On 04/12/2016 01:07 PM, Oleg Bartunov wrote: > > Our roadmap http://www.postgresql.org/developer/roadmap/ is the problem. > We don't have clear roadmap and that's why we cannot plan future feature > full release. As someone who's worked at multiple proprietary software companies, having a roadma

Re: [HACKERS] Problems with huge_pages and IBM Power8

Andres Freund writes: > On 2016-04-12 21:58:14 +0200, reiner peterke wrote: >> Looking for some insight into this issue. the error from the postgres >> log on ubuntu is below. It apperas to be related to semephores. > I've a bit of a hard time believing that this is related to huge pages. I'm

Re: [HACKERS] Problems with huge_pages and IBM Power8

On 2016-04-12 21:58:14 +0200, reiner peterke wrote: > Hi > > We have been doing some testing with Postgres (9.5.2) compiled on a Power8 > running Centos 7 > > When working with huge_pages, we initially got this error. > > munmap(0x3efbe400) failed: Invalid argument *munmap*, not mmap faile

Re: [HACKERS] raw output from copy

2016-04-12 12:22 GMT+02:00 Ants Aasma : > On 8 Apr 2016 9:14 pm, "Pavel Stehule" wrote: > > 2016-04-08 20:54 GMT+02:00 Andrew Dunstan : > >> I should add that I've been thinking about this some more, and that I > now agree that something should be done to support this at the SQL level, > mainly s

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

On Tue, Apr 12, 2016 at 2:53 PM, Kevin Grittner wrote: > Readonly with client and job counts matching scale. Single-socket i7, BTW. >> A lot of this will be different between >> single-socket and multi-socket servers; as soon as you have the latter >> the likelihood of contention being bad goes

Re: [HACKERS] Updated backup APIs for non-exclusive backups

On Tue, Apr 12, 2016 at 8:39 AM, Noah Misch wrote: > On Mon, Apr 11, 2016 at 11:22:27AM +0200, Magnus Hagander wrote: > > On Thu, Apr 7, 2016 at 5:15 AM, Noah Misch wrote: > > > Unless you especially want to self-impose the same tight resolution > > > schedule > > > that 9.6 regressions experien

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

On Tue, Apr 12, 2016 at 9:25 PM, Josh berkus wrote: > On 04/12/2016 10:43 AM, Robert Haas wrote: > > 1. Large backward compatibility breaks are bad. Therefore, if any of > > these things are absolutely impossible to do without major > > compatibility breaks, we shouldn't do them at all. > > +1 >

[HACKERS] Problems with huge_pages and IBM Power8

Hi We have been doing some testing with Postgres (9.5.2) compiled on a Power8 running Centos 7 When working with huge_pages, we initially got this error. munmap(0x3efbe400) failed: Invalid argument after a bit of investigation we noticed that hugepagesize is har coded to 2MB src/backend/p

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

On Tue, Apr 12, 2016 at 2:28 PM, Andres Freund wrote: > On 2016-04-12 14:17:12 -0500, Kevin Grittner wrote: >> Well, something is different between your environment and mine, >> since I saw no difference at scale 100 and 2.2% at scale 200. > > In a readonly test or r/w? Readonly with client and j

Re: [HACKERS] Support for N synchronous standby servers - take 2

On Tue, Apr 12, 2016 at 9:04 AM, Masahiko Sawada wrote: > On Mon, Apr 11, 2016 at 8:47 PM, Fujii Masao wrote: >> On Mon, Apr 11, 2016 at 5:52 PM, Masahiko Sawada >> wrote: >>> On Mon, Apr 11, 2016 at 1:31 PM, Fujii Masao wrote: On Mon, Apr 11, 2016 at 10:58 AM, Masahiko Sawada wrot

Re: [HACKERS] Remove unused function from walsender.c

On Tue, Apr 12, 2016 at 6:36 AM, Simon Riggs wrote: > On 11 April 2016 at 08:05, Fujii Masao wrote: > >> >> There is an unused function GetOldestWALSendPointer() in walsender.c. >> Per comment, it was introduced because we may need it in the future for >> synchronous replication. >> >> Now we hav

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

Hi, On 2016-04-12 14:17:12 -0500, Kevin Grittner wrote: > Well, something is different between your environment and mine, > since I saw no difference at scale 100 and 2.2% at scale 200. In a readonly test or r/w? A lot of this will be different between single-socket and multi-socket servers; as

Re: [HACKERS] WIP: Upper planner pathification

Robert Haas writes: > On Thu, Mar 17, 2016 at 2:31 PM, Tom Lane wrote: >> So what I would now propose is >> >> typedef void (*create_upper_paths_hook_type) (PlannerInfo *root, >> UpperRelationKind stage, >>

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

On Tue, Apr 12, 2016 at 1:56 PM, Andres Freund wrote: > On 2016-04-12 13:44:00 -0500, Kevin Grittner wrote: >> On Tue, Apr 12, 2016 at 12:38 PM, Andres Freund wrote: >>> On 2016-04-12 16:49:25 +, Kevin Grittner wrote: On a big NUMA machine with 1000 connections in saturation load th

Re: [HACKERS] [COMMITTERS] pgsql: Allow Pin/UnpinBuffer to operate in a lockfree manner.

On 2016-04-12 07:00, Andres Freund wrote: On 2016-04-12 00:32:13 -0400, Tom Lane wrote: I wrote: This commit has broken buildfarm member gaur, and no doubt pademelon will be equally unhappy once it catches up to HEAD. Spoke too soon, actually: pademelon did not get as far as initdb. cc: "buf

Re: [HACKERS] pgsql: Add the "snapshot too old" feature

On 2016-04-12 13:57:26 -0500, Kevin Grittner wrote: > I have booked two large NUMA machines for Friday to look for any > remaining performance regressions on high-concurrency loads on such > machines. Anyone with ideas on particular tests that they feel > should be included, please let me know bef

Re: [HACKERS] pgsql: Add the "snapshot too old" feature

On Mon, Apr 11, 2016 at 12:31 PM, Kevin Grittner wrote: > On Mon, Apr 11, 2016 at 8:20 AM, Alexander Korotkov >> So, for read-only benchmark this patch introduces more than 5 times >> regression on big machine. > > I did not schedule a saturation test on a big machine. I guess I > should have do

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

On 2016-04-12 13:44:00 -0500, Kevin Grittner wrote: > On Tue, Apr 12, 2016 at 12:38 PM, Andres Freund wrote: > > On 2016-04-12 16:49:25 +, Kevin Grittner wrote: > >> On a big NUMA machine with 1000 connections in saturation load > >> there was a performance regression due to spinlock contentio

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

On Tue, Apr 12, 2016 at 2:27 PM, Andres Freund wrote: > none but 2) seem likely to require a substantial compatibility break. And even that doesn't require one, if you keep the only system around and make the new system optional via some sort of pluggable storage API. Which, to me, seems like th

[HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

On Tue, Apr 12, 2016 at 12:38 PM, Andres Freund wrote: > On 2016-04-12 16:49:25 +, Kevin Grittner wrote: >> On a big NUMA machine with 1000 connections in saturation load >> there was a performance regression due to spinlock contention, for >> acquiring values which were never used. Just fill

Re: [HACKERS] Detrimental performance impact of ringbuffers on performance

On 2016-04-12 14:29:10 -0400, Robert Haas wrote: > On Wed, Apr 6, 2016 at 6:57 AM, Andres Freund wrote: > > While benchmarking on hydra > > (c.f. > > http://archives.postgresql.org/message-id/20160406104352.5bn3ehkcsceja65c%40alap3.anarazel.de), > > which has quite slow IO, I was once more annoye

Re: [HACKERS] Detrimental performance impact of ringbuffers on performance

On Wed, Apr 6, 2016 at 6:57 AM, Andres Freund wrote: > While benchmarking on hydra > (c.f. > http://archives.postgresql.org/message-id/20160406104352.5bn3ehkcsceja65c%40alap3.anarazel.de), > which has quite slow IO, I was once more annoyed by how incredibly long > the vacuum at the the end of a p

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

On 2016-04-12 11:25:21 -0700, Josh berkus wrote: > Here's the features I can imagine being worth major backwards > compatibility breaks: > > 1. Fully pluggable storage with a clean API. > > 2. Total elimination of VACUUM or XID freezing > > 3. Fully transparent-to-the user MM replication/cluster

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

On 04/12/2016 10:43 AM, Robert Haas wrote: > 1. Large backward compatibility breaks are bad. Therefore, if any of > these things are absolutely impossible to do without major > compatibility breaks, we shouldn't do them at all. +1 > 2. Small backward compatibility breaks are OK, but don't requir

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

On Mon, Apr 11, 2016 at 9:45 PM, Michael Paquier wrote: > On Mon, Apr 11, 2016 at 5:16 PM, Etsuro Fujita > wrote: >> On 2016/04/11 12:30, Michael Paquier wrote: >>> >>> + if ((cancel = PQgetCancel(entry->conn))) >>> + { >>> + P

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

On Tue, Apr 12, 2016 at 12:32 PM, Justin Clift wrote: > Yeah. Moving the discussion here was more to determine which items > really would need a backwards compatible break. eg no other approach can > be found. > > Seems I started it off badly, as no-one's yet jumped in to discuss the > initial p

[HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

On 2016-04-12 16:49:25 +, Kevin Grittner wrote: > On a big NUMA machine with 1000 connections in saturation load > there was a performance regression due to spinlock contention, for > acquiring values which were never used. Just fill with dummy > values if we're not going to use them. FWIW, I

[HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

Kevin Grittner wrote: > FWIW, I spent a fair amount of time trying to make it PGC_SIGHUP, > since it would be very nice to allow that; but I kept running into > one problem after another with it, some of which were very hard to > see how to fix. My inclination is that trying to comment all the >

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

Kevin Grittner writes: > On Tue, Apr 12, 2016 at 12:08 PM, Alvaro Herrera > wrote: >> old_snapshot_threshold is PGC_POSTMASTER, so this is okay AFAICS, but >> perhaps it'd be a good idea to add a oneline comment to guc.c indicating >> to verify this code if there's an intention to lift that limit

[HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

On Tue, Apr 12, 2016 at 12:08 PM, Alvaro Herrera wrote: > Kevin Grittner wrote: >> Avoid extra locks in GetSnapshotData if old_snapshot_threshold < 0 >> >> On a big NUMA machine with 1000 connections in saturation load >> there was a performance regression due to spinlock contention, for >> acquir

[HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

Kevin Grittner wrote: > Avoid extra locks in GetSnapshotData if old_snapshot_threshold < 0 > > On a big NUMA machine with 1000 connections in saturation load > there was a performance regression due to spinlock contention, for > acquiring values which were never used. Just fill with dummy > value

Re: [HACKERS] Speedup twophase transactions

> On 12 Apr 2016, at 15:47, Michael Paquier wrote: > > On Mon, Apr 11, 2016 at 7:16 PM, Stas Kelvich wrote: >> Michael, it looks like that you are the only one person who can reproduce >> that bug. I’ve tried on bunch of OS’s and didn’t observe that behaviour, >> also looking at your backtraces

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

There are only 3 fundamental options I see, the cleaner can wait, "help", or move on. "Helping" is what it does now and is dangerous. Moving on gives the above-discussed unthrottling problem. Waiting has two problems. The act of waiting will cause autovacuums to be canceled, unless ugly hacks

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

On 12 Apr 2016, at 17:23, Merlin Moncure wrote: > On Mon, Apr 11, 2016 at 11:39 AM, Justin Clift wrote: >> Moving over a conversation from the pgsql-advocacy mailing list. In it >> Simon (CC'd) raised the issue of potentially creating a >> backwards-compatibility >> breaking release at some poi

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

This restricts the memory used by ordinary backends when doing the cleanup to be work_mem. Shouldn't we let them use maintenance_work_mem? Only one backend can be doing this clean up of a given index at any given time, so we don't need to worry about many concurrent allocations of maintenance_work

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

On Tue, Apr 12, 2016 at 9:36 AM, Craig Ringer wrote: > Repeating the mapping at each checkpoint sounds pretty reasonable and means > we always know what we need. There's no need to bloat each record with an > extension name and no need for any kind of ugly global registration. The > mapping table

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

On Mon, Apr 11, 2016 at 11:39 AM, Justin Clift wrote: > Moving over a conversation from the pgsql-advocacy mailing list. In it > Simon (CC'd) raised the issue of potentially creating a > backwards-compatibility > breaking release at some point in the future, to deal with things that > might have

Re: [HACKERS] WIP: Covering + unique indexes.

Attached version has fix of pg_dump suggested by Stephen Frost in -committers thread. http://postgresql.nabble.com/pgsql-CREATE-INDEX-INCLUDING-column-td5897653.html Sooner or later, I'd like to see this patch fin

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

On 2016-04-12 19:42:11 +0530, Amit Kapila wrote: > Yes, it seems generally it is a good idea, but not sure if it is a complete > fix for variation in performance we are seeing when we change shared memory > structures. I didn't suspect it would be. More whether it'd be beneficial performance wise.

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

On Tue, Apr 12, 2016 at 6:42 PM, Tom Lane wrote: > Alexander Korotkov writes: > > I agree with both of these ideas. Patch is attached. > > Pushed with minor cleanup. > Great, thanks! -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] [COMMITTERS] pgsql: Allow Pin/UnpinBuffer to operate in a lockfree manner.

I wrote: > It looks like that compiler adheres to the C89 restriction that an > initializer for an array or struct must contain only link-time-constant > expressions, even if the target object is of dynamic scope. > The macro works with a link-time-constant pointer argument, but not > with one that

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

Alexander Korotkov writes: > I agree with both of these ideas. Patch is attached. Pushed with minor cleanup. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

On Tue, Apr 12, 2016 at 6:34 PM, Teodor Sigaev wrote: > I agree with both of these ideas. Patch is attached. >> > > Hmm, you accidentally forget to change flag type of GenericXLogRegister in > contrib/bloom signature. Fixed. -- Alexander Korotkov Postgres Professional: http://www.postgresp

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

I agree with both of these ideas. Patch is attached. Hmm, you accidentally forget to change flag type of GenericXLogRegister in contrib/bloom signature. -- Teodor Sigaev E-mail: teo...@sigaev.ru WWW: http://

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

Alexander Korotkov writes: > Attached patch is intended to fix this. It zeroes "hole" in both > GenericXLogFinish() and generic_redo(). Pushed. I rewrote the comments a bit and modified GenericXLogFinish so that it doesn't copy data only to overwrite it with zeroes. reg

Re: [HACKERS] Preprocessor condition fix

Christian Ullrich writes: > * Tom Lane wrote: >> Oh? Then we should not need that one (the /D switch in win32.mak) at all. >> Should we just remove it? > We have both confirmed several times that nothing depends on it. I think it > can go. Done. regards, tom lane --

Re: [HACKERS] [COMMITTERS] pgsql: Allow Pin/UnpinBuffer to operate in a lockfree manner.

Andres Freund writes: > On 2016-04-12 00:32:13 -0400, Tom Lane wrote: >> Apparently, assigning the result of init_spin_delay() is not as portable >> as you thought. > Hm. I'm not sure why not though? Is it because delayStatus isn't static > or global scope? It looks like that compiler adheres to

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

On Tue, Apr 12, 2016 at 3:48 PM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > On Tue, Apr 12, 2016 at 12:40 AM, Andres Freund > wrote: > >> I did get access to the machine (thanks!). My testing shows that >> performance is sensitive to various parameters influencing memory >> allocati

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

On 12 Apr 2016, at 14:12, Yury Zhuravlev wrote: > Justin Clift wrote: >> Simon included a short starter list of potentials which might be in >> that category: >> >> * SQL compliant identifiers >> * Remove RULEs >> * Change recovery.conf >> * Change block headers >> * Retire template0, templa

Re: [HACKERS] Preprocessor condition fix

Christian Ullrich writes: > * Tom Lane wrote: >> Hm, my grep found another one ... > Oh, sorry. I saw that one, but thought it was intentional because _WIN64 > is defined automatically anyway. Oh? Then we should not need that one (the /D switch in win32.mak) at all. Should we just remove it?

Re: [HACKERS] Preprocessor condition fix

* From: Tom Lane [mailto:t...@sss.pgh.pa.us] > Christian Ullrich writes: > > * Tom Lane wrote: > >> Hm, my grep found another one ... > > > Oh, sorry. I saw that one, but thought it was intentional because _WIN64 > > is defined automatically anyway. > > Oh? Then we should not need that one (t

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

On 12 April 2016 at 20:48, Robert Haas wrote: > > So how do we look at a generic log record, say "ok, the upstream that > wrote > > this says it needs to invoke registered generic xlog hook 42, which is > > from at " ? > > Record enough information in WAL that the rmgrs can have names instead

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

On Tue, Apr 12, 2016 at 3:33 AM, Tom Lane wrote: > ... BTW, with respect to the documentation angle, it seems to me > that it'd be better if GenericXLogRegister were renamed to > GenericXLogRegisterBuffer, or perhaps GenericXLogRegisterPage. > I think this would make the documentation clearer, an

Re: [HACKERS] [COMMITTERS] pgsql: Code cleanup in the wake of recent LWLock refactoring.

2016-04-12 14:51 GMT+02:00 Robert Haas : > On Tue, Apr 12, 2016 at 4:18 AM, Pavel Stehule > wrote: > > If there will be simple way, how to fix it, then I'll fix my extensions. > But > > new API is working only when the extension has own share memory segment. > For > > some complex extensions like

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

Justin Clift wrote: Simon included a short starter list of potentials which might be in that category: * SQL compliant identifiers * Remove RULEs * Change recovery.conf * Change block headers * Retire template0, template1 * Optimise FSM * Add heap metapage * Alter tuple headers

Re: [HACKERS] Choosing parallel_degree

On 12 April 2016 at 13:53, Robert Haas wrote: > On Mon, Apr 11, 2016 at 5:45 PM, Simon Riggs > wrote: > > On 8 April 2016 at 17:49, Robert Haas wrote: > >> With the patch, you can - if you wish - substitute > >> some other number for the one the planner comes up with. > > > > I saw you're using

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

On Sun, Apr 10, 2016 at 7:49 AM, Tom Lane wrote: > 1. It doesn't seem like generic_xlog.c has thought very carefully about > the semantics of the "hole" between pd_lower and pd_upper. The mainline > XLOG code goes to some lengths to ensure that the hole stays all-zeroes; > for example RestoreBlo

Re: [HACKERS] [BUGS] Breakage with VACUUM ANALYSE + partitions

On Mon, Apr 11, 2016 at 1:17 PM, Andres Freund wrote: > I can think of a number of relatively easy ways to address this: > 1) Just zap (or issue?) all pending flush requests when getting an >smgrinval/smgrclosenode > 2) Do 1), but filter for the closed relnode > 3) Actually handle the case of

Re: [HACKERS] Choosing parallel_degree

On Mon, Apr 11, 2016 at 5:45 PM, Simon Riggs wrote: > On 8 April 2016 at 17:49, Robert Haas wrote: >> With the patch, you can - if you wish - substitute >> some other number for the one the planner comes up with. > > I saw you're using AccessExclusiveLock, the reason being it affects SELECTs. > >

Re: [HACKERS] Choosing parallel_degree

On 04/11/2016 08:57 PM, Julien Rouhaud wrote: >Expected = Expecting worker8 information , also loops=10 (including the >Master) > Even if you set a per-table parallel_degree higher than max_parallel_degree, it'll be maxed at max_parallel_degree. Then, the explain shows that the planner assumed

Re: [HACKERS] [COMMITTERS] pgsql: Code cleanup in the wake of recent LWLock refactoring.

On Tue, Apr 12, 2016 at 4:18 AM, Pavel Stehule wrote: > If there will be simple way, how to fix it, then I'll fix my extensions. But > new API is working only when the extension has own share memory segment. For > some complex extensions like Orafce it means expensive refactoring. > > What is wors

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

On Mon, Apr 11, 2016 at 9:54 PM, Craig Ringer wrote: > If you rely on shared_preload_libraries, then "oops, I forgot to put > myextension in shared_preload_libraries on the downstream" becomes a Bad > Thing. There's no way to go back and redo the work you've passed over. You > have to recreate the

Re: [HACKERS] Speedup twophase transactions

On Mon, Apr 11, 2016 at 7:16 PM, Stas Kelvich wrote: > Michael, it looks like that you are the only one person who can reproduce > that bug. I’ve tried on bunch of OS’s and didn’t observe that behaviour, also > looking at your backtraces I can’t get who is holding this lock (and all of > that ha

Re: [HACKERS] Update copyright in genericdesc.c

* Amit Langote (langote_amit...@lab.ntt.co.jp) wrote: > Attached fixes copyright in file mentioned in $subject. Fixed. Thanks! Stephen signature.asc Description: Digital signature

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

On Tue, Apr 12, 2016 at 3:08 AM, Michael Paquier wrote: > What I thought we should be able to do with that should not be only > limited to indexes, aka to: > - Be able to register and log full pages > - Be able to log data associated to a block > - Be able to have record data > - Use at recovery

Re: [HACKERS] raw output from copy

On 8 Apr 2016 9:14 pm, "Pavel Stehule" wrote: > 2016-04-08 20:54 GMT+02:00 Andrew Dunstan : >> I should add that I've been thinking about this some more, and that I now >> agree that something should be done to support this at the SQL level, mainly >> so that clients can manage very large pieces

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

On Tue, Apr 12, 2016 at 12:40 AM, Andres Freund wrote: > I did get access to the machine (thanks!). My testing shows that > performance is sensitive to various parameters influencing memory > allocation. E.g. twiddling with max_connections changes > performance. With max_connections=400 and the p

Re: [HACKERS] Choosing parallel_degree

On 12 April 2016 at 07:58, Amit Kapila wrote: > On Tue, Apr 12, 2016 at 3:15 AM, Simon Riggs > wrote: > >> On 8 April 2016 at 17:49, Robert Haas wrote: >> >> >>> With the patch, you can - if you wish - substitute >>> some other number for the one the planner comes up with. >> >> >> I saw you're

Re: [HACKERS] [BUGS] Breakage with VACUUM ANALYSE + partitions

Hello Andres, I can think of a number of relatively easy ways to address this: 1) Just zap (or issue?) all pending flush requests when getting an smgrinval/smgrclosenode 2) Do 1), but filter for the closed relnode 3) Actually handle the case of the last open segment not being RELSEG_SIZE pr

Re: [HACKERS] [COMMITTERS] pgsql: Code cleanup in the wake of recent LWLock refactoring.

Hi 2016-02-13 15:54 GMT+01:00 Tom Lane : > Simon Riggs writes: > > On 10 February 2016 at 16:36, Tom Lane wrote: > >> FWIW, I wasn't paying attention either, but I'm convinced by Robert's > >> argument. Avoiding coupling between extensions is worth an API break. > > > Old APIs - why can't we k

Re: [HACKERS] Choosing parallel_degree

On 04/11/2016 09:14 PM, Robert Haas wrote: postgres=# explain analyze verbose select * from abd where n<=1; >>ERROR: requested shared memory size overflows size_t >> >>if we remove the analyze keyword then query running successfully. >> >>Expected = Is it not better to throw the error at the ti

Re: [HACKERS] Parser extensions (maybe for 10?)

On 12 April 2016 at 13:51, Tom Lane wrote: > Craig Ringer writes: > > The other area where there's room for extension without throwing out the > > whole thing and rebuilding is handling of new top-level statements. We > can > > probably dispatch the statement text to a sub-parser provided by an

[HACKERS] Update copyright in genericdesc.c

Hi, Attached fixes copyright in file mentioned in $subject. Thanks, Amit diff --git a/src/backend/access/rmgrdesc/genericdesc.c b/src/backend/access/rmgrdesc/genericdesc.c index caa9a03..0796bb8 100644 --- a/src/backend/access/rmgrdesc/genericdesc.c +++ b/src/backend/access/rmgrdesc/genericdesc.c