Re: [HACKERS] Proposal for changes to recovery.conf API

2016-11-04 Thread Simon Riggs
uot; approach is almost exactly what we have now, I think we should do that first, get this patch committed and then sit back and discuss an improved API and see what downsides it introduces. Further delay on this isn't helpful for other patches going in. -- Simon Riggshttp:/

Re: [HACKERS] Partitioned tables and relfilenode

2017-02-19 Thread Simon Riggs
On 16 February 2017 at 11:32, Simon Riggs wrote: > On 10 February 2017 at 06:19, Amit Langote > wrote: > >> the "right thing" here being that the >> command's code either throws an error or warning (in some cases) if the >> specified table is a pa

Re: [HACKERS] Documentation improvements for partitioning

2017-02-19 Thread Simon Riggs
ce, but this was already a really big > patch, and IMHO quite a good one. Please explain these personal comments against me. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing li

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-20 Thread Simon Riggs
On 20 February 2017 at 09:15, Amit Kapila wrote: > On Mon, Feb 20, 2017 at 7:26 AM, Masahiko Sawada > wrote: >> On Fri, Feb 17, 2017 at 3:41 AM, Robert Haas wrote: >>> On Thu, Feb 16, 2017 at 6:17 AM, Simon Riggs wrote: >>>> On 15 February 2017 at 08

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-20 Thread Simon Riggs
end of xact, so patch attached to remove that call, plus some comments to explain that. This reduces the cause. Also, another patch to reduce the calls to SnapshotResetXmin() using a simple heuristic to reduce the effects. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Developmen

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-20 Thread Simon Riggs
On 20 February 2017 at 16:53, Robert Haas wrote: > On Mon, Feb 20, 2017 at 6:02 PM, Simon Riggs wrote: >> On 15 February 2017 at 19:15, Andres Freund wrote: >> >>> I think I previously >>> mentioned, even just removing the MyPgXact->xmin assignment in &

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-20 Thread Simon Riggs
s the reverse. If I had > to guess, it would be that neither the costs nor the savings from this > are in the slightest way noticeable on a macrobenchmark, and therefore > there's not much point in changing it, but that could be 100% wrong. Given Andres' earlier measurements, it

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-20 Thread Simon Riggs
On 20 February 2017 at 10:27, Amit Kapila wrote: > On Mon, Feb 20, 2017 at 3:01 PM, Simon Riggs wrote: >> On 20 February 2017 at 09:15, Amit Kapila wrote: >>> On Mon, Feb 20, 2017 at 7:26 AM, Masahiko Sawada >>> wrote: >>>> On Fri, Feb 17, 2017 at 3:

Re: [HACKERS] [doc fix] Really trivial fix for BRIN documentation

2017-02-21 Thread Simon Riggs
On 21 February 2017 at 06:34, Tsunakawa, Takayuki wrote: > Hello, > > This is just a correction from "index" to "table". I was a bit confused when > I first read this part. Pushed, but using "heap" rather than "table", for clarity. Thanks

Re: [HACKERS] Measuring replay lag

2017-02-21 Thread Simon Riggs
On 17 February 2017 at 07:45, Thomas Munro wrote: > On Fri, Feb 17, 2017 at 12:45 AM, Simon Riggs wrote: >> Feeling happier about this for now at least. > > Thanks! And happier again, leading me to move to the next stage of review, focusing on the behaviour emerging from the

Re: [HACKERS] Documentation improvements for partitioning

2017-02-23 Thread Simon Riggs
talking about? Which things have been exaggerated, and by whom? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Documentation improvements for partitioning

2017-02-23 Thread Simon Riggs
lly in restricted form. I think we should probably avoid trying to UPDATE rows from one partition to another in this release, since that seems likely to be buggy and seems like would only be needed in relatively few cases. Let me know if I can help with these. -- Simon Riggshttp://ww

Re: [HACKERS] dropping partitioned tables without CASCADE

2017-02-23 Thread Simon Riggs
On 23 February 2017 at 06:40, Ashutosh Bapat wrote: > I think this is ready for committer. Thanks for writing and reviewing this. I'll be happy to review and commit. Please add to CF. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Re

Re: [HACKERS] Documentation improvements for partitioning

2017-02-23 Thread Simon Riggs
On 23 February 2017 at 17:27, Peter Geoghegan wrote: > On Thu, Feb 23, 2017 at 8:08 AM, Simon Riggs wrote: >> What claims are you talking about? Which things have been exaggerated, >> and by whom? > > * The specious argument that we should "just" have CREATE IN

Re: [HACKERS] Measuring replay lag

2017-02-23 Thread Simon Riggs
On 21 February 2017 at 21:38, Thomas Munro wrote: > On Tue, Feb 21, 2017 at 6:21 PM, Simon Riggs wrote: >> And happier again, leading me to move to the next stage of review, >> focusing on the behaviour emerging from the design. >> >> So my current understanding is

Re: [HACKERS] Documentation improvements for partitioning

2017-02-23 Thread Simon Riggs
on >> >> As missing items, features 3 and 4 seem achievable in this release, >> potentially in restricted form. > > Simon, this is ridiculous. We're 4 or 5 days away from the start of > the last CommitFest. We have time to fix bugs and improve > documentation and maybe

Re: [HACKERS] Proposal: GetOldestXminExtend for ignoring arbitrary vacuum flags

2017-02-23 Thread Simon Riggs
On 16 February 2017 at 05:24, Seki, Eiji wrote: > Simon Riggs wrote: >> Please persuade us with measurements that allowing this impact on >> ANALYZE would really improve performance at least in your case, and >> also examine the effect of this on the accuracy and usefulne

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-23 Thread Simon Riggs
est my patch also please? -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] UPDATE of partition key

2017-02-24 Thread Simon Riggs
rs with an ERROR and a retryable SQLCODE, since the UPDATE will succeed on next execution. 2. I know that DB2 handles this by having the user specify WITH ROW MOVEMENT to explicitly indicate they accept the issue and want update to work even with that. We could have an explicit option to allow that.

Re: [HACKERS] Proposal for changes to recovery.conf API

2017-02-24 Thread Simon Riggs
On 12 January 2017 at 13:34, Peter Eisentraut wrote: > On 1/11/17 5:27 AM, Simon Riggs wrote: >> The main area of "design doubt" remains the implementation of the >> recovery_target parameter set. Are we happy with the user interface >> choices in the patch, g

Re: [HACKERS] Allow pg_dumpall to work without pg_authid

2017-02-25 Thread Simon Riggs
successfully. > > pg_dumpall --globals-only --no-role-password > a.sql It looks to me like you'd need to use --no-security-labels as well, in most cases since that accesses pg_authid also. The patch seemed to be missing substantial chunks of coding, so I've added

[HACKERS] Reduce lock levels for reloptions

2017-02-25 Thread Simon Riggs
%2BTgmoYe5eDhjRodo3uOtVFGiDWwO2zGUp_mDHeSxuEqq-jS_A%40mail.gmail.com Earlier patch, dropped from 9.6 https://www.postgresql.org/message-id/CANP8%2Bj%2B0fuq2MWKvUsQTJFt8EF_z-Tyn-Q61J2A%2BVT2Uzuf-Rg%40mail.gmail.com -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training

Re: [HACKERS] Proposal for changes to recovery.conf API

2017-02-25 Thread Simon Riggs
an error of implementation, not intention. I had it on my list to keep, at your request. New version coming up. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hacker

Re: [HACKERS] dropping partitioned tables without CASCADE

2017-02-25 Thread Simon Riggs
On 23 February 2017 at 16:33, Simon Riggs wrote: > I'll be happy to review Patch looks OK so far, but fails on a partition that has partitions, probably because of the way we test relkind in the call to StoreCatalogInheritance1(). Please add a test for that so we can check autom

Re: [HACKERS] Documentation improvements for partitioning

2017-02-27 Thread Simon Riggs
indexes, all of which would have their own independent sets of caveats. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your su

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-27 Thread Simon Riggs
sponsiveness from us. Thanks Michael, Magnus. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://w

Re: [HACKERS] Documentation improvements for partitioning

2017-02-28 Thread Simon Riggs
On 28 February 2017 at 08:14, Amit Langote wrote: > OK. So, I will start writing the patch with above general skeleton and > hopefully post it within this week and you can improve it as fit. Will do, thanks. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Devel

[HACKERS] Avoiding bloat in CIC

2017-02-28 Thread Simon Riggs
In CREATE INDEX CONCURRENTLY it seems possible to release the index build snapshot early, so we can reset our xmin before we complete the sort and write the main part of the index. Patch to implement this attached. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development

Re: [HACKERS] avoid bloat from CREATE INDEX CONCURRENTLY

2017-02-28 Thread Simon Riggs
lobal xmin to advance, > > Um ... isn't there a transaction boundary there anyway? Yes, the patch releases the snapshot early, so it does not hold it once the build scan has completed. This allows the sort and build phases to occur without holding back the xmin. -- Simon Riggs

Re: [HACKERS] Allow pg_dumpall to work without pg_authid

2017-02-28 Thread Simon Riggs
t "for the benefit of forks" but for users > of core too. If this was of benefit only to one company it would not get time or attention from me. I thought that would have been obvious enough, but if not, I'm happy to say it clearly. The enhanced patch by me removes any mention

Re: [HACKERS] avoid bloat from CREATE INDEX CONCURRENTLY

2017-02-28 Thread Simon Riggs
On 28 February 2017 at 13:30, Tom Lane wrote: > Simon Riggs writes: >> On 28 February 2017 at 13:05, Tom Lane wrote: >>> Um ... isn't there a transaction boundary there anyway? > >> Yes, the patch releases the snapshot early, so it does not hold it >> o

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-28 Thread Simon Riggs
.131330194 > 128 28455 28618 0.5728342998 > 180 26739 26879 0.5235797898 > 196 27820 27963 0.5140186916 > 256 28763 28969 0.7161978931 > > Also, Excel sheet (results-readwrite-300-SF.xlsx) containing the results > for all the 3 runs is attached. > -- Simon Riggs

Re: [HACKERS] Should we cacheline align PGXACT?

2017-03-01 Thread Simon Riggs
On 1 March 2017 at 04:50, Ashutosh Sharma wrote: > On Tue, Feb 28, 2017 at 11:44 PM, Simon Riggs > wrote: > >> On 28 February 2017 at 11:34, Ashutosh Sharma >> wrote: >> >> >>> So, Here are the pgbench results I got with ' >>> *reduce_pg

Re: [HACKERS] Allow pg_dumpall to work without pg_authid

2017-03-04 Thread Simon Riggs
On 28 February 2017 at 17:49, Simon Riggs wrote: > I've edited the stated reason for the patch on the CF app, so its > clearer as to why this might be acceptable. Robins, I'm looking to commit the patch version I posted, so I would like your comments that it does continue to so

Re: [HACKERS] dropping partitioned tables without CASCADE

2017-03-04 Thread Simon Riggs
On 27 February 2017 at 02:38, Amit Langote wrote: > On 2017/02/26 5:30, Simon Riggs wrote: >> On 23 February 2017 at 16:33, Simon Riggs wrote: >> >>> I'll be happy to review >> >> Patch looks OK so far, but fails on a partition that has partitions, >&

Re: [HACKERS] Measuring replay lag

2017-03-04 Thread Simon Riggs
On 1 March 2017 at 10:47, Thomas Munro wrote: > On Fri, Feb 24, 2017 at 9:05 AM, Simon Riggs wrote: >> On 21 February 2017 at 21:38, Thomas Munro >> wrote: >>> However, I think a call like LagTrackerWrite(SendRqstPtr, >>> GetCurrentTimestamp()) needs to g

Re: [HACKERS] Measuring replay lag

2017-03-04 Thread Simon Riggs
s have been successfully applied then it should not continue to show 14 secs for the next 2 hours. IMHO the lag time should drop to zero in a reasonable time and stay at zero for those 2 hours because there is no current lag. If we want to show historical lag data, I'm supportive of the idea, b

Re: [HACKERS] dropping partitioned tables without CASCADE

2017-03-05 Thread Simon Riggs
hink that's what this patch fixes. Do you see this behaviour after > applying the patch? It does seems as if I've made a mistake there. The patch passes. Thanks for checking. I will apply tomorrow if no further comments. -- Simon Riggshttp://www.2ndQuadrant.com/ Po

Re: [HACKERS] dropping partitioned tables without CASCADE

2017-03-05 Thread Simon Riggs
On 6 March 2017 at 00:51, Amit Langote wrote: > On 2017/03/05 16:20, Simon Riggs wrote: >> I notice also that >> \d+ >> does not show which partitions have subpartitions. > > Do you mean showing just whether a partition is itself partitioned or > showing its parti

Re: [HACKERS] dropping partitioned tables without CASCADE

2017-03-05 Thread Simon Riggs
On 6 March 2017 at 04:00, Ashutosh Bapat wrote: > On Mon, Mar 6, 2017 at 8:35 AM, Simon Riggs wrote: >> On 6 March 2017 at 00:51, Amit Langote wrote: >>> On 2017/03/05 16:20, Simon Riggs wrote: >>>> I notice also that >>>> \d+ >>>

Re: [HACKERS] dropping partitioned tables without CASCADE

2017-03-05 Thread Simon Riggs
. "has partitions" is not part of the DDL, whereas "FOR VALUES FROM (0) TO (100)" is. So ISTM sensible to differentiate between DDL and non-ddl using upper and lower case. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Supp

Re: [HACKERS] PATCH: Configurable file mode mask

2017-03-06 Thread Simon Riggs
ome files in one mode, some in another. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Patch to improve performance of replay of AccessExclusiveLock

2017-03-07 Thread Simon Riggs
but if the common case COMMITs don't scan the list that would be most of the problem solved. If we did need a hash table, we should just use a normal hash table? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Patch to improve performance of replay of AccessExclusiveLock

2017-03-07 Thread Simon Riggs
Don't understand this. I'm talking about setting a flag on commit/abort WAL records, like the attached. We just need to track info so we can set the flag at EOXact and we're done. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, R

Re: [HACKERS] BRIN de-summarize ranges

2017-03-07 Thread Simon Riggs
and research to work out how to do that, so we can only do it this way at present. Other than that the patch looks fairly straightforward, so adding a few tests will be all we need here. Plus function docs. I'm on the hook to write brin docs anyway, so no more needed here. -- Simon Ri

Re: [HACKERS] Need a builtin way to run all tests faster manner

2017-03-07 Thread Simon Riggs
a way of seeing if the build farm likes things. If they do, we can push to the main repo. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.o

Re: [HACKERS] Logical decoding on standby

2017-03-07 Thread Simon Riggs
it > message. Anyone feel strongly about how granular this should be? > > This patch series is a pre-requisite for supporting logical > replication using a physical standby as a source, but does not its > self enable logical replication from a physical standby. Patch 4 committed.

Re: [HACKERS] Measuring replay lag

2017-03-15 Thread Simon Riggs
On 14 March 2017 at 07:39, Thomas Munro wrote: > > On Mon, Mar 6, 2017 at 3:22 AM, Craig Ringer wrote: >> On 5 March 2017 at 15:31, Simon Riggs wrote: >>> What we want from this patch is something that works for both, as much >>> as that is possible. >> &

Re: [HACKERS] Measuring replay lag

2017-03-15 Thread Simon Riggs
On 14 March 2017 at 07:39, Thomas Munro wrote: > Hi, > > Please see separate replies to Simon and Craig below. > > On Sun, Mar 5, 2017 at 8:38 PM, Simon Riggs wrote: >> On 1 March 2017 at 10:47, Thomas Munro wrote: >>> I do see why a new user trying this fe

Re: [HACKERS] Patch to improve performance of replay of AccessExclusiveLock

2017-03-15 Thread Simon Riggs
On 8 March 2017 at 10:02, David Rowley wrote: > On 8 March 2017 at 01:13, Simon Riggs wrote: >> Don't understand this. I'm talking about setting a flag on >> commit/abort WAL records, like the attached. > > There's nothing setting a flag in the attached. I

Re: [HACKERS] Measuring replay lag

2017-03-15 Thread Simon Riggs
hat is the most recent lag measurement" and "how long until we are caught up" as possible intrepretations of these values. Thanks. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via p

Re: [HACKERS] Patch to improve performance of replay of AccessExclusiveLock

2017-03-18 Thread Simon Riggs
ock to the subxid rather than the top xid. That could increase lock traffic, but less likely. It also solves the problem of early release when AELs held by subxids. (2) looks safe enough, so patch attached. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Devel

Re: [HACKERS] Patch to improve performance of replay of AccessExclusiveLock

2017-03-18 Thread Simon Riggs
On 16 March 2017 at 19:08, Amit Kapila wrote: > On Thu, Mar 16, 2017 at 6:01 AM, Simon Riggs wrote: >> On 8 March 2017 at 10:02, David Rowley wrote: >>> On 8 March 2017 at 01:13, Simon Riggs wrote: >>>> Don't understand this. I'm talking about setting

Re: [HACKERS] Patch to improve performance of replay of AccessExclusiveLock

2017-03-18 Thread Simon Riggs
don't see the gain from adding that to each xact state. I'd suggest refactoring my patch so that the existign MyXactAccessedTempRel becomes MyXactFlags and we can just set a flag in the two cases (temp rels and has-aels). That way we still have one Global but it doesn't get any uglie

Re: [HACKERS] PinBuffer() no longer makes use of strategy

2017-03-19 Thread Simon Riggs
oiling by large scans. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/m

Re: [HACKERS] Logical decoding on standby

2017-03-19 Thread Simon Riggs
On 13 March 2017 at 10:56, Craig Ringer wrote: > On 7 March 2017 at 21:08, Simon Riggs wrote: > >> Patch 4 committed. Few others need rebase. > > Since this patch series Patch 1 fails since feature has already been applied. If other reason, let me know. -- Simon Riggs

Re: [HACKERS] Logical decoding on standby

2017-03-19 Thread Simon Riggs
cause failure because in read_local_xlog_page() we say that we are reading from history when state->currTLI != ThisTimeLineID explicitly rather than use sendTimeLineIsHistoric So it looks like we could do with a few extra comments If you correct these I'll commit it tomorrow. -- Simon Riggs

Re: [HACKERS] logical decoding of two-phase transactions

2017-03-20 Thread Simon Riggs
ort until after decoding. So I suggest we have a pre-prepare callback to ensure that the plugin can decide whether to decode or not. We can pass information to the plugin such as whether we have issued DDL in that xact or not. The plugin can then decide how it wishes to handle it, so if somebody doesn't like the idea of a lock then don't use one. The plugin is already responsible for many things, so this is nothing new. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [RFC, POC] Don't require a NBuffer sized PrivateRefCount array of local buffer pins

2014-06-22 Thread Simon Riggs
much memory in idle backends and small configs, plus we know we are using too much memory in larger servers. Reducing the memory usage here will reduce CPU L2 cache churn as well as increasing available RAM. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 2

Re: [HACKERS] Allowing join removals for more join types

2014-06-22 Thread Simon Riggs
= c.id GROUP BY c.id) b ON a.id = b.id AND b.dummy = 1; i.e. c.id is not in the join, but we know from subselect that c.id = b.id and b.id is in the join -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via

Re: [HACKERS] pg_resetxlog to clear backup start/end locations.

2014-06-22 Thread Simon Riggs
On 13 June 2014 12:27, Fujii Masao wrote: > I think that pg_resetxlog should reset backup locations by default > since they are useless (rather harmful) after pg_resetxlog. Thought? +1 Do we regard that point as a bug that should be backpatched? -- Simon Riggs http

Re: [HACKERS] [RFC, POC] Don't require a NBuffer sized PrivateRefCount array of local buffer pins

2014-06-22 Thread Simon Riggs
obenchmarks won't reveal the beneficial L2 and RAM effects of the patch, so I suggest we just need to do a pgbench, a 2-way nested join and a 10-way nested join with an objective of no significant difference or better. The RAM and L2 effects are enough to justify this, since it will h

Re: [HACKERS] Allowing join removals for more join types

2014-06-22 Thread Simon Riggs
On 22 June 2014 12:51, Simon Riggs wrote: > Looks good on initial look. Tests 2 and 3 seem to test the same thing. There are no tests which have multiple column clauselist/sortlists, nor tests for cases where the clauselist is a superset of the sortlist. Test comments should refer to &q

Re: [HACKERS] Allowing join removals for more join types

2014-06-24 Thread Simon Riggs
nt on the transitive closure question? Should we add a test for that, whether or not it works yet? Other than that it looks pretty good to commit, so I'll wait a week for other objections then commit. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Su

Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-06-24 Thread Simon Riggs
t be buggy. > The attached patch is a broken implemention that still needs the lookup code > fixed to reference the correct RTE. The failing regression tests show where > the problems lie. > > Any help on this would be really appreciated. I'd suggest we just drop the targetli

Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-06-24 Thread Simon Riggs
be worth worrying about. So I think we can remove a NOT NULL constraint without too much problem. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-06-24 Thread Simon Riggs
On 24 June 2014 23:44, Tom Lane wrote: > Simon Riggs writes: >> Having said that, any join plan that relies upon a constraint will >> still be valid even if we drop a constraint while the plan executes >> because any new writes will not be visible to the executing

Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-06-24 Thread Simon Riggs
On 24 June 2014 23:52, Tom Lane wrote: > Simon Riggs writes: >> On 24 June 2014 23:44, Tom Lane wrote: >>> Simon Riggs writes: >>>> Having said that, any join plan that relies upon a constraint will >>>> still be valid even if we drop a constraint wh

Re: [HACKERS] Allowing join removals for more join types

2014-06-24 Thread Simon Riggs
On 24 June 2014 23:48, Tom Lane wrote: > Simon Riggs writes: >> Other than that it looks pretty good to commit, so I'll wait a week >> for other objections then commit. > > I'd like to review this before it goes in. I've been waiting for it to > get mar

Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-06-25 Thread Simon Riggs
On 24 June 2014 23:22, Simon Riggs wrote: >> On a more positive or even slightly exciting note I think I've managed to >> devise a way that ANTI JOINS can be used for NOT IN much more often. It >> seems that find_nonnullable_vars will analyse a quals list to find >&g

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2014-06-26 Thread Simon Riggs
dit feature in core 9.5, but I don't see those adding SQL and adding specific catalog columns as giving us anything at all. It will still be an audit feature without them. Many, many users do not want audit. Many do. So the idea is to allow audit to be added in a way that doesn't affec

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2014-06-26 Thread Simon Riggs
situation. I personally never had the time or money to fix that situation until now, so I'm hoping and expecting that that will change in 9.5, as discussed in developer meeting. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training &a

Re: [HACKERS] Allowing join removals for more join types

2014-06-26 Thread Simon Riggs
group by > clause, but I'm not really sure it's testing anything new or different. > > EXPLAIN (COSTS OFF) > SELECT a.id FROM a > LEFT JOIN (SELECT b.id,1 as dummy FROM b INNER JOIN c ON b.id = c.id GROUP > BY b.id) b ON a.id = b.id AND b.dummy = 1; OK, agreed, no need

Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-06-26 Thread Simon Riggs
would agree with Tom that the common usage is to do NOT IN against a table with no where clause, so this would hit the most frequent use case. Maybe Tom will have a flash of insight before commit, or maybe we figure out a way to extend it later. Let's document it and move on. --

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2014-06-26 Thread Simon Riggs
a chance of assessing things in the right context. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

2014-06-29 Thread Simon Riggs
On 29 June 2014 10:01, Thomas Munro wrote: > Would anyone who is interested in a SKIP LOCKED feature and > attending the CHAR(14)/PGDay UK conference next week be > interested in a birds-of-a-feather discussion? Sounds like a plan. I'll check my schedule. --

Re: [HACKERS] Proposal for CSN based snapshots

2014-06-30 Thread Simon Riggs
s transactions in shared memory, hopefully DSM. The above mechanism sounds like it might slow down transaction commit. Could we prototype that and measure the speed? More generally, do we have any explicit performance goals or acceptance criteria for this patch? -- Simon Riggs

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2014-07-01 Thread Simon Riggs
hardcoding locations is seriously painful and timeconsuming, one of the reasons why extensions and background workers exist. But if you missed it before, I will say it again: we have a patch now and 2ndQuadrant has a limit on further funding. If you want to reject pgaudit and move on to something els

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2014-07-02 Thread Simon Riggs
on - that is good since there is no big rush. pgaudit is a sincere attempt to add audit functionality to Postgres. If you or anyone else wants to make a similarly sincere attempt to add audit functionality to Postgres, lets see the design and its connection to requirements. -- Simon Riggs

Re: [HACKERS] [PATCH] introduce XLogLockBlockRangeForCleanup()

2014-07-03 Thread Simon Riggs
Locked(buf); else UnlockBufHdr(buf); since otherwise we would access the buffer flags without the spinlock and we would leak a pin if the buffer was not valid -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services --

Re: [HACKERS] Minmax indexes

2014-07-11 Thread Simon Riggs
4f8.8080...@agliodbs.com >> about a new name, but there were no suggestions supported by more than >> one person. If a brilliant new name comes up, I'm open to changing it. > > How about "summarizing indexes"? That seems reasonably descriptive. -1 for a

Re: [HACKERS] Minmax indexes

2014-07-11 Thread Simon Riggs
that's just modular programming. Not sure we need to prefix things with maybe to explain that, otherwise we'd have maybeXXX everywhere. More descriptive name would be MaintainIndexBounds() or similar. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 2

Re: [HACKERS] tweaking NTUP_PER_BUCKET

2014-07-11 Thread Simon Riggs
uot;We made it faster" might not be true if we run this on servers that are already experiencing high memory pressure. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-ha

Re: [HACKERS] tweaking NTUP_PER_BUCKET

2014-07-11 Thread Simon Riggs
On 11 July 2014 10:23, Tomas Vondra wrote: > On 11 Červenec 2014, 9:27, Simon Riggs wrote: >> On 9 July 2014 18:54, Tomas Vondra wrote: >> >>> (1) size the buckets for NTUP_PER_BUCKET=1 (and use whatever number >>> of batches this requires) >> >>

Re: [HACKERS] tweaking NTUP_PER_BUCKET

2014-07-12 Thread Simon Riggs
on memory pressure than anything else. So my earlier concerns seem less of a concern. So lets just this change done and then do more later. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers maili

Re: [HACKERS] tweaking NTUP_PER_BUCKET

2014-07-13 Thread Simon Riggs
city, but how about we solve the first challenge first and the second one second? -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make ch

Re: [HACKERS] [COMMITTERS] pgsql: Reset master xmin when hot_standby_feedback disabled.

2014-07-15 Thread Simon Riggs
On 15 July 2014 19:15, Tom Lane wrote: > Simon Riggs writes: >> Reset master xmin when hot_standby_feedback disabled. >> If walsender has xmin of standby then ensure we >> reset the value to 0 when we change from hot_standby_feedback=on >> to hot_standb

Re: [HACKERS] [COMMITTERS] pgsql: Reset master xmin when hot_standby_feedback disabled.

2014-07-15 Thread Simon Riggs
I'm happy to follow any agreed process we lay out. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http:

Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-07-15 Thread Simon Riggs
ut another way, can we look at ways to skip the test if its not likely to add value. Obviously, if we have a good feeling that we might save lots of execution time then any additional planning time is easier to justify. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Dev

Re: [HACKERS] [COMMITTERS] pgsql: Reset master xmin when hot_standby_feedback disabled.

2014-07-15 Thread Simon Riggs
On 15 July 2014 22:01, Tom Lane wrote: > Simon Riggs writes: >> On 15 July 2014 19:15, Tom Lane wrote: >>> While I'm not necessarily objecting to the content of this patch, >>> I do have a problem with the process. Where was the discussion of >>&g

[HACKERS] Production block comparison facility

2014-07-20 Thread Simon Riggs
consistent full_page_write_check = on The above changes seem easy to implement. With FPW compression, this would be a usable feature in production. Comments? -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-h

Re: [HACKERS] Built-in binning functions

2014-07-20 Thread Simon Riggs
d be confusing to have same name for different parameter profiles. So I suggest that we use the more generic function name bin(), with a second choice of discretize() (though that seems annoyingly easy to spell incorrectly) Whatever we do, it seems clear we need a section in

Re: [HACKERS] Production block comparison facility

2014-07-22 Thread Simon Riggs
On 22 July 2014 08:49, Michael Paquier wrote: > On Sun, Jul 20, 2014 at 5:31 PM, Simon Riggs wrote: >> The block comparison facility presented earlier by Heikki would not be >> able to be used in production systems. ISTM that it would be desirable >> to have something that

Re: [HACKERS] Production block comparison facility

2014-07-22 Thread Simon Riggs
ic verification at low costs. Yes, the two options I proposed are somewhat independent of each other. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.

Re: [HACKERS] Production block comparison facility

2014-07-23 Thread Simon Riggs
On 23 July 2014 15:14, Michael Paquier wrote: > I have spent some time digging more into this idea and finished with the > patch attached Thank you for investigating the idea. I'll review by Monday. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Develo

Re: [HACKERS] PL/PgSQL: EXIT USING ROLLBACK

2014-07-28 Thread Simon Riggs
this feature suggestion. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PL/PgSQL: EXIT USING ROLLBACK

2014-07-28 Thread Simon Riggs
On 28 July 2014 10:34, Marko Tiikkaja wrote: > On 7/28/14 11:27 AM, Simon Riggs wrote: >> >> On 26 July 2014 18:14, Marko Tiikkaja wrote: >> >>> Today I'd like to present a way to get rid of code like this: >> >> >> You haven't explain

Re: [HACKERS] Performance issue in pg_dump's dependency loop searching

2014-07-29 Thread Simon Riggs
On 25 July 2014 20:47, Tom Lane wrote: > Another idea would be to ...persist the optimal dump order in the database. That way we can maintain the correct dump order each time we do DDL, which is only a small incremental cost, no matter how many objects we have. -- Simon Ri

Re: [HACKERS] Production block comparison facility

2014-07-30 Thread Simon Riggs
pages are after images, then we just make the check after applying WAL. So I don't see the need for two full page images. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers ma

Re: [HACKERS] Production block comparison facility

2014-07-31 Thread Simon Riggs
sequence of actions being RestoreBackupBlock(), apply WAL then CheckBackupBlock(). That will work without much code churn, it will be just a one line addition in a few dozen places. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Service

<    1   2   3   4   5   6   7   8   9   10   >