Re: [HACKERS] Processing long AND/OR lists

2013-05-26 Thread Gurjeet Singh
My last email was written before reading this. A few episodes of 24 occurred between writing and sending that email. Added slony1-hackers, but didn't remove pgsql-hackers. Feel free to exclude pgsql lists, as this branch of conversation seems to be more Slony related than Postgres. On Sun, May 26

Re: [HACKERS] shmem startup and shutdown hooks

2013-05-26 Thread Alvaro Herrera
Amit kapila wrote: > > On Saturday, May 25, 2013 12:50 AM Alvaro Herrera wrote: > > It seems that the right place to do this is checkpointer shutdown, i.e. > > when checkpointer is told to close shop it should also invoke various > > modules' shutdown callbacks. There's no hook point there thoug

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-26 Thread Craig Ringer
On 05/25/2013 05:39 PM, Simon Riggs wrote: > 2. Name the next release after that 10.0 (would have been 9.5). We > declare now that > a) 10.0 will support on-line upgrade from 9.4 (only) > b) various major incompatibilities will be introduced in 10.0 - the > change in release number will indicate to

Re: [HACKERS] Processing long AND/OR lists

2013-05-26 Thread Gurjeet Singh
On Sun, May 26, 2013 at 11:46 AM, Tom Lane wrote: > Josh Berkus writes: > > ***15,000***? I'd say that someone has an application design issue. > > Fixing the stack overflow is a good thing, but that query is never going > > to return ... > Just for the record, it does finish in 5 sec on my la

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-26 Thread Christopher Browne
The assumption that we ought to plan expressly for an incompatibility that essentially discards pg_upgrade seems premature, particularly in advance of would-be solutions that, in some cases, mightn't actually work. If pg_upgrade doesn't work, then, at present, the plausible solutions are to either

Re: [HACKERS] Processing long AND/OR lists

2013-05-26 Thread Christopher Browne
This situation falls from a problem that we noticed a mighty long time ago in Slony, where the set of XIDs outstanding gets very large, and, attendant to that, the set of "action id" values by which tuples are being filtered, gets correspondingly large. It happens when there is a long pause in app

Re: [HACKERS] getting rid of freezing

2013-05-26 Thread Josh Berkus
Andres, I was talking this over with Jeff on the plane, and we wanted to be clear on your goals here: are you looking to eliminate the *write* cost of freezing, or just the *read* cost of re-reading already frozen pages? If just the latter, what about just adding a bit to the visibility map to i

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-26 Thread Tom Lane
Stephen Frost writes: > btw, has anyone posted the SM API proposal..? Unfortunately, I think I > had to leave before that was hashed out.. There isn't one yet. We think we understand where the pain points are, but there's still a long way to go to have a proposal. regar

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-26 Thread Stephen Frost
* Josh Berkus (j...@agliodbs.com) wrote: > and it's entirely possible that we'll be able to implement SMs without > breaking pgupgrade. I'd certainly hope so.. It's certainly not obvious, to me at least, why a new SM or supporting any of those features would require breaking pg_upgrade. Perhaps

Re: [HACKERS] MVCC catalog access

2013-05-26 Thread Michael Paquier
Perhaps we see little difference in performance because PGPROC has been separated into PGPROC and PGXACT, reducing lock contention with getting snapshot data? By the way, I grabbed a 32-core machine and did some more performance tests with some open connections with XIDs assigned using pg_cxn v2 g

Re: [HACKERS] Parallel Sort

2013-05-26 Thread Stephen Frost
* Noah Misch (n...@leadboat.com) wrote: > In particular, implementing a GPU-based strcoll() for bttextcmp > sounds like quite a project in its own right. It also wouldn't likely be helpful... To be able to use a GPU effectively, last I looked, you need to be able to move a large chunk of data to

Re: [HACKERS] background worker and normal exit

2013-05-26 Thread Michael Paquier
On Mon, May 27, 2013 at 2:04 AM, Bernd Helmle wrote: > > > --On 26. Mai 2013 11:38:55 +0900 Michael Paquier < > michael.paqu...@gmail.com> wrote: > > >> This flag makes a worker not to restart only in case of a crash. To solve >> your problem, you could as well allow your process to restart and p

Re: [HACKERS] Patch proposal: query result history in psql

2013-05-26 Thread Maciej Gajewski
Polished version of the patch. * The feature is disabled by default, enabled by backslash command \ans. Additionaly, \ansclean cleans the result history. * Escaping is applied when building COPY IN string This is patch is a diff between master:230e92c and https://github.com/maciekgajewski/psql-an

Re: [HACKERS] Parallel Sort

2013-05-26 Thread Noah Misch
On Fri, May 24, 2013 at 01:13:21PM -0500, Jim Nasby wrote: > On 5/13/13 9:28 AM, Noah Misch wrote: >> It would be great if one client session could take advantage of multiple CPU >> cores. EnterpriseDB wishes to start the trek into this problem space for 9.4 >> by implementing parallel internal (i

[HACKERS] Add regression tests for SET xxx

2013-05-26 Thread Robins Tharakan
Hi, Please find attached a patch to take code-coverage of SET (SESSION / SEED / TRANSACTION / DATESTYLE / TIME ZONE) (src/backend/commands/variable.c) from 65% to 82%. Any and all feedback is welcome. -- Robins Tharakan regress_variable.patch Description: Binary data -- Sent via pgsql-hackers

Re: [HACKERS] background worker and normal exit

2013-05-26 Thread Bernd Helmle
--On 26. Mai 2013 11:38:55 +0900 Michael Paquier wrote: This flag makes a worker not to restart only in case of a crash. To solve your problem, you could as well allow your process to restart and put it in indefinite sleep if server is not in recovery such it it will do nothing in your cas

Re: [HACKERS] [BUGS] COPY .... (FORMAT binary) syntax doesn't work

2013-05-26 Thread Tom Lane
Heikki Linnakangas writes: > On 26.05.2013 04:31, Simon Riggs wrote: >> This new format does not [work:] >> COPY pgbench_accounts FROM '/tmp/acc' (FORMAT BINARY); >> ERROR: syntax error at or near "BINARY" at character 47 > This seems to work: > --- a/src/backend/parser/gram.y > +++ b/src/backe

Re: [HACKERS] Using indexes for partial index builds

2013-05-26 Thread Jim Nasby
On 3/13/13 7:10 PM, Ants Aasma wrote: On Thu, Mar 14, 2013 at 1:51 AM, Jim Nasby wrote: On 3/12/13 9:10 AM, Ants Aasma wrote: I have a feeling this is an increasingly widespread pattern with a proliferation of mobile devices that need syncing. If you're doing that with timestamps you're aski

Re: [HACKERS] Processing long AND/OR lists

2013-05-26 Thread Tom Lane
Josh Berkus writes: > ***15,000***? I'd say that someone has an application design issue. > Fixing the stack overflow is a good thing, but that query is never going > to return ... Yeah, the parser's stack consumption seems like only the tip of the iceberg here. I find it hard to visualize a us

Re: [HACKERS] [BUGS] COPY .... (FORMAT binary) syntax doesn't work

2013-05-26 Thread Heikki Linnakangas
On 26.05.2013 04:31, Simon Riggs wrote: This works fine... COPY pgbench_accounts TO '/tmp/acc' BINARY; This new format does not COPY pgbench_accounts FROM '/tmp/acc' (FORMAT BINARY); ERROR: syntax error at or near "BINARY" at character 47 which looks like I've mistyped something. Until you rea

Re: [HACKERS] Processing long AND/OR lists

2013-05-26 Thread Josh Berkus
On 05/25/2013 09:56 AM, Gurjeet Singh wrote: > When Postgres encounters a long list of AND/OR chains, it errors out at > check_stack_depth() after a limit of few thousand. At around 10,000 > elements, the recursion at assign_expr_collations() causes the error. But > at a little higher element count

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-26 Thread Josh Berkus
> Not sure which ones Simon meant, but at least any new/better > storage manager would seem to me to be requiring > a non-pg_upgrade upgrade path unless we require the storage manager > to also include a parallel implementation of pg_upgrade. Isn't this a bit of horse-cart inversion here? We jus

Re: [HACKERS] View Index and UNION

2013-05-26 Thread Tom Lane
Stefan Keller writes: > Given following schema: > 1. TABLE a and TABLE b, each with INDEX on attribute geom. > 2. A VIEW with union: > CREATE VIEW myview AS > SELECT * FROM a > UNION > SELECT * FROM b; > 3. And a simple query with KNN index and a coordinate "mypos" : > SELECT * FROM myv

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-26 Thread Hannu Krosing
On 05/26/2013 04:22 PM, Bruce Momjian wrote: > On Sun, May 26, 2013 at 09:18:11AM -0400, Bruce Momjian wrote: >> On Sun, May 26, 2013 at 10:53:37AM +0100, Simon Riggs wrote: I consider this thread to be not thought-through, obviously. >>> My proposal has had lots of serious consideration, but

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-26 Thread Bruce Momjian
On Sun, May 26, 2013 at 09:18:11AM -0400, Bruce Momjian wrote: > On Sun, May 26, 2013 at 10:53:37AM +0100, Simon Riggs wrote: > > > I consider this thread to be not thought-through, obviously. > > > > My proposal has had lots of serious consideration, but that is not the > > topic of this thread.

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-26 Thread Bruce Momjian
On Sun, May 26, 2013 at 10:53:37AM +0100, Simon Riggs wrote: > > I consider this thread to be not thought-through, obviously. > > My proposal has had lots of serious consideration, but that is not the > topic of this thread. > > The title of the thread is a general one, with a clear objective. >

Re: [HACKERS] getting rid of freezing

2013-05-26 Thread Hannu Krosing
On 05/25/2013 01:14 PM, Simon Riggs wrote: > On 24 May 2013 17:00, Robert Haas wrote: >> On Fri, May 24, 2013 at 11:29 AM, Robert Haas wrote: >>> On Fri, May 24, 2013 at 10:53 AM, Andres Freund >>> wrote: > [all-visible cannot restore hint bits without FPI because of torn pages] I have

Re: [HACKERS] View Index and UNION

2013-05-26 Thread William King
I appear to have been able to replicate what you are talking about, but it required explicitly binding the order by in different ways. See attached files. William King Senior Engineer Quentus Technologies, INC 1037 NE 65th St Suite 273 Seattle, WA 98115 Main: (877) 211-9337 Office: (206) 388-477

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-26 Thread Simon Riggs
On 25 May 2013 21:44, Bruce Momjian wrote: > On Sat, May 25, 2013 at 10:39:30AM +0100, Simon Riggs wrote: >> There are a number of changes we'd probably like to make to the way >> things work in Postgres. This thread is not about discussing what >> those are, just to say that requirements exist an

Re: [HACKERS] View Index and UNION

2013-05-26 Thread Stefan Keller
Yes, it actually does, but the planner chooses a seq scan to prepare for that. -S. 2013/5/26 William King : > Could this scenario not be handled by a step that orders the two tables > independently, then for the view interleaves the presorted results? > Merging two sorted sets into a single sorte

[HACKERS] COPY .... (FORMAT binary) syntax doesn't work

2013-05-26 Thread Simon Riggs
This works fine... COPY pgbench_accounts TO '/tmp/acc' BINARY; This new format does not COPY pgbench_accounts FROM '/tmp/acc' (FORMAT BINARY); ERROR: syntax error at or near "BINARY" at character 47 which looks like I've mistyped something. Until you realise that this statement gives a completel