[HACKERS] improving \dt++ in psql

2014-04-30 Thread Filip Rembiałkowski
Hi I tried to start writing a patch to add "Total Size" column to \dt++ output. in src/bin/psql/describe.c we have this listTables( const char *tabtypes, const char *pattern, bool verbose, bool showSystem) I was (as a long time Pg user) dead sure that psql really sometimes cares about the n

Re: [HACKERS] Fix initdb for path with whitespace and at char

2014-04-30 Thread Amit Kapila
On Wed, Apr 30, 2014 at 4:01 PM, Heikki Linnakangas wrote: > I committed the non-invasive fixes to backbranches (and master too, just to > keep it in sync), but the attached is what I came up with for master. > > There are a couple of places in the code where we have #ifdef WIN32 code > that uses

Re: [HACKERS] hooks not working in postgresql 9.3 (building from source)

2014-04-30 Thread Michael Paquier
On Thu, May 1, 2014 at 12:33 AM, Rajmohan C wrote: >I need to use the hook feature of planner in postgresql 9.3 to perform a > task. I am building postgresql from source. This hook is called planner_hook_type and is present in planner.c. > To start with, I wanted to know how to create a hook

Re: [HACKERS] Small doc patch for pg_replication_slots

2014-04-30 Thread Robert Haas
On Tue, Apr 29, 2014 at 5:39 AM, Thomas Reiss wrote: > You can find attached a small patch to fix the pg_replication_slots > documentation. The slot_type and plugin are not in the appropriate > order, slot_name and plugin have a wrong type and xmin appears two times. Without the patch, the descri

Re: [HACKERS] Display of timestamp in pg_dump custom format

2014-04-30 Thread Gavin Flower
On 01/05/14 12:04, Bruce Momjian wrote: On Thu, May 1, 2014 at 08:27:49AM +1200, Gavin Flower wrote: On 01/05/14 02:51, Bruce Momjian wrote: The table of contents for pg_restore -l shows the time the archive was made as local time (it uses ctime()): ; Archive created at Wed Apr 30 10:

Re: [HACKERS] Display of timestamp in pg_dump custom format

2014-04-30 Thread Bruce Momjian
On Thu, May 1, 2014 at 08:27:49AM +1200, Gavin Flower wrote: > On 01/05/14 02:51, Bruce Momjian wrote: > >The table of contents for pg_restore -l shows the time the archive was > >made as local time (it uses ctime()): > > > > ; Archive created at Wed Apr 30 10:03:28 2014 > > > >Is this clear e

Re: [HACKERS] includedir_internal headers are not self-contained

2014-04-30 Thread Tom Lane
Andres Freund writes: > On 2014-04-28 13:20:47 -0400, Tom Lane wrote: >> Printing anything other than the relation OID here is irrelevant, >> misleading, and inconsistent with our practice everywhere else. > I don't think it's really comparable to the other scenarios. We should > print the oid, j

Re: [HACKERS] shm_mq inconsistent behavior of SHM_MQ_DETACHED

2014-04-30 Thread Robert Haas
On Mon, Apr 28, 2014 at 4:24 PM, Petr Jelinek wrote: > Yes, the patch fixes it for me. OK. I committed it. Thanks for the report. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)

Re: [HACKERS] GSoC on WAL-logging hash indexes

2014-04-30 Thread Tom Lane
Greg Stark writes: > But imnsho doing nothing is a bad idea. We should have long ago either > added WAL logging or removed the index type. We shouldn't have left a > foot-gun that large lying around for so long. We can't remove the hash index type, nor move it to an extension, because it is the o

Re: [HACKERS] Fix initdb for path with whitespace and at char

2014-04-30 Thread Andrew Dunstan
On 04/30/2014 03:03 PM, Andrew Dunstan wrote: On 04/30/2014 11:58 AM, Andrew Dunstan wrote: On 04/30/2014 06:31 AM, Heikki Linnakangas wrote: Andrew: you have a cygwin installation, don't you? Could you test if "pg_ctl start" works when the binaries are installed to a path that contains b

Re: [HACKERS] includedir_internal headers are not self-contained

2014-04-30 Thread Tom Lane
I wrote: > How about we change common/relpath.[hc] to export a single version of > relpath() that takes its arguments as separate plain OIDs, and then > make the other versions wrappers that are only declared in some > backend header? The wrappers could possibly be macros, allowing > things like p

Re: [HACKERS] bgworker crashed or not?

2014-04-30 Thread Robert Haas
On Mon, Apr 28, 2014 at 4:19 PM, Petr Jelinek wrote: > On 28/04/14 16:27, Robert Haas wrote: >> On Thu, Apr 24, 2014 at 1:39 AM, Craig Ringer >> wrote: >>> On 04/17/2014 08:35 AM, Craig Ringer wrote: >>> I've just noticed that the bgworker control interfaces do not honour >>> bgw.bgw_restart_time

Re: [HACKERS] GSoC on WAL-logging hash indexes

2014-04-30 Thread Alvaro Herrera
Stephen Frost wrote: > * Greg Stark (st...@mit.edu) wrote: > > I could do the legwork on either the GUC or moving hash indexes to an > > extension if there's a consensus. But I suspect either will be quite > > controversial... > If you'd like to build an extension which provides hash indexes- I s

Re: [pgsql-advocacy] [HACKERS] GSoC on WAL-logging hash indexes

2014-04-30 Thread Darren Duncan
Is there a good reason for this thread being copied to the advocacy list? It seems to me just on topic for hackers. -- Darren Duncan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Display of timestamp in pg_dump custom format

2014-04-30 Thread Gavin Flower
On 01/05/14 02:51, Bruce Momjian wrote: The table of contents for pg_restore -l shows the time the archive was made as local time (it uses ctime()): ; Archive created at Wed Apr 30 10:03:28 2014 Is this clear enough that it is local time? Should we display this better, perhaps with a t

Re: [HACKERS] GSoC on WAL-logging hash indexes

2014-04-30 Thread Jeff Janes
On Wed, Apr 30, 2014 at 12:16 PM, Greg Stark wrote: > I think the key question was if someone wanted to develop a good hash > index would they start from our current hash index or would they be > better off starting from a fresh codebase? If it were me I'd start with the current code. It would

Re: [HACKERS] GSoC on WAL-logging hash indexes

2014-04-30 Thread Jeff Janes
On Wed, Apr 30, 2014 at 11:19 AM, Peter Geoghegan wrote: > On Wed, Apr 30, 2014 at 11:03 AM, Jeff Janes wrote: > > If we don't put in the work to make them useful, then they won't ever > become > > useful. > > > > If we do put in the effort (and it would be considerable) then I think > they > >

Re: [HACKERS] GSoC on WAL-logging hash indexes

2014-04-30 Thread Stephen Frost
* Greg Stark (st...@mit.edu) wrote: > But imnsho doing nothing is a bad idea. We should have long ago either > added WAL logging or removed the index type. We shouldn't have left a > foot-gun that large lying around for so long. I certainly encourage you to work on it. :) > Incidentally something

Re: [HACKERS] pg_get_viewdefs() indentation considered harmful

2014-04-30 Thread Greg Stark
On Wed, Apr 30, 2014 at 6:47 PM, Tom Lane wrote: > I pushed a patch that does it that way, and also patches for the other > items discussed in this thread. Great! thanks a lot. This makes a really solid noticeable difference even in relatively simple cases and I know of users for whom this will m

Re: [HACKERS] GSoC on WAL-logging hash indexes

2014-04-30 Thread Greg Stark
I think the key question was if someone wanted to develop a good hash index would they start from our current hash index or would they be better off starting from a fresh codebase? If the former then we should add WAL logging and throw GSOC and other people who ask for projects at it. If the latter

Re: [HACKERS] Fix initdb for path with whitespace and at char

2014-04-30 Thread Andrew Dunstan
On 04/30/2014 11:58 AM, Andrew Dunstan wrote: On 04/30/2014 06:31 AM, Heikki Linnakangas wrote: Andrew: you have a cygwin installation, don't you? Could you test if "pg_ctl start" works when the binaries are installed to a path that contains both a space and an @ sign, like "C:\white spa

Re: [HACKERS] GSoC on WAL-logging hash indexes

2014-04-30 Thread Peter Geoghegan
On Wed, Apr 30, 2014 at 11:03 AM, Jeff Janes wrote: > If we don't put in the work to make them useful, then they won't ever become > useful. > > If we do put in the effort (and it would be considerable) then I think they > will be. But you may be correct that the effort required would perhaps be

Re: [HACKERS] Fix initdb for path with whitespace and at char

2014-04-30 Thread Tom Lane
Heikki Linnakangas writes: > I committed the non-invasive fixes to backbranches (and master too, just > to keep it in sync), but the attached is what I came up with for master. The malloc's in the new system.c file should be pg_malloc, or else have custom defenses against out-of-memory (possibly

Re: [HACKERS] GSoC on WAL-logging hash indexes

2014-04-30 Thread Robert Haas
On Wed, Apr 30, 2014 at 2:02 PM, Peter Geoghegan wrote: >> Of course our current hash indexes have *more* not less contention >> than btree but I'm pretty comfortable chalking that up to quality of >> implementation rather than anything intrinsic. > > I am not convinced of that. In commit 76837c1

Re: [HACKERS] GSoC on WAL-logging hash indexes

2014-04-30 Thread Andres Freund
On 2014-04-30 11:10:22 -0700, Jeff Janes wrote: > I've seen the simple pinning and unpinning of the root page (or the fast root, > whatever the first page we bother to pin on a regular basis is called) be a > point of contention.  When one index dominates the entire system workload, > that > one p

Re: [HACKERS] GSoC on WAL-logging hash indexes

2014-04-30 Thread Jeff Janes
On Wed, Apr 30, 2014 at 11:02 AM, Peter Geoghegan wrote: > On Wed, Apr 30, 2014 at 10:11 AM, Robert Haas > wrote: > > I thought the theoretical advantage of hash indexes wasn't that they > > were smaller but that you avoided a central contention point (the > > btree root). > > The B-Tree root is

Re: [HACKERS] GSoC on WAL-logging hash indexes

2014-04-30 Thread Josh Berkus
All, (dropping pgsql-advocacy off the cc list) On 04/30/2014 10:11 AM, Robert Haas wrote: > On Wed, Apr 30, 2014 at 12:54 PM, Peter Geoghegan wrote: >> On Wed, Apr 30, 2014 at 5:55 AM, k...@rice.edu wrote: >>> I do not think that CPU costs matter as much as the O(1) probe to >>> get a result va

Re: [HACKERS] GSoC on WAL-logging hash indexes

2014-04-30 Thread Jeff Janes
On Wed, Apr 30, 2014 at 12:26 AM, Peter Geoghegan wrote: > On Mon, Mar 3, 2014 at 8:12 AM, Robert Haas wrote: > >> As a GSoC student, I will implement WAL recovery of hash indexes using > the > >> other index types' WAL code as a guide. > > Frankly, I'm skeptical of the idea that hash indexes wi

Re: [HACKERS] GSoC on WAL-logging hash indexes

2014-04-30 Thread Peter Geoghegan
On Wed, Apr 30, 2014 at 11:01 AM, Tom Lane wrote: > It might lead to good things later; and even if > it doesn't, the lack of WAL support is an embarrassment. I don't think it will, but I do agree that the current state of affairs is an embarrassment. -- Peter Geoghegan -- Sent via pgsql-ha

Re: [HACKERS] GSoC on WAL-logging hash indexes

2014-04-30 Thread Peter Geoghegan
On Wed, Apr 30, 2014 at 10:11 AM, Robert Haas wrote: > I thought the theoretical advantage of hash indexes wasn't that they > were smaller but that you avoided a central contention point (the > btree root). The B-Tree root isn't really a central contention point at all. The locking/latching proto

Re: [HACKERS] GSoC on WAL-logging hash indexes

2014-04-30 Thread Tom Lane
Robert Haas writes: > I thought the theoretical advantage of hash indexes wasn't that they > were smaller but that you avoided a central contention point (the > btree root). > Of course our current hash indexes have *more* not less contention > than btree but I'm pretty comfortable chalking that

Re: [HACKERS] pg_get_viewdefs() indentation considered harmful

2014-04-30 Thread Tom Lane
I wrote: > I'm still dubious about halving the step distance, because there are > assorted places that adjust the indentation of specific keywords by > distances that aren't a multiple of 2 (look for odd last arguments to > appendContextKeyword). I'm not sure how that will look after we make such

Re: [HACKERS] GSoC on WAL-logging hash indexes

2014-04-30 Thread Robert Haas
On Wed, Apr 30, 2014 at 12:54 PM, Peter Geoghegan wrote: > On Wed, Apr 30, 2014 at 5:55 AM, k...@rice.edu wrote: >> I do not think that CPU costs matter as much as the O(1) probe to >> get a result value specifically for very large indexes/tables where >> even caching the upper levels of a B-tree

Re: [HACKERS] GSoC on WAL-logging hash indexes

2014-04-30 Thread Peter Geoghegan
On Wed, Apr 30, 2014 at 5:55 AM, k...@rice.edu wrote: > I do not think that CPU costs matter as much as the O(1) probe to > get a result value specifically for very large indexes/tables where > even caching the upper levels of a B-tree index would kill your > working set in memory. I know, I know,

[HACKERS] Sequential disk access during VACUUM for GiST/GIN

2014-04-30 Thread Костя Кузнецов
Hello.There is a task "Sequential disk access during VACUUM for GiST/GIN " in list GSOC14.Nobody is working on this task?Do I understand this task correctly?I must recode gistbulkdelete.GistBDItem *stack is must have items with sequential blkno as possible.I have a question:where are access to disk

Re: [HACKERS] tests for client programs

2014-04-30 Thread Andres Freund
On 2014-04-30 18:09:15 +0200, Andres Freund wrote: > The issues here don't seem to have been addressed in the commit. At > least the latin1 thing should be fixed. As an additional issue it currently doesn't seem to work in VPATH builds. That's imo a must fix. Greetings, Andres Freund -- Andr

Re: [HACKERS] tests for client programs

2014-04-30 Thread Andres Freund
On 2014-04-04 16:44:46 +0200, Andres Freund wrote: > On 2014-02-27 21:44:48 -0500, Peter Eisentraut wrote: > > +open HBA, ">>$tempdir/pgdata/pg_hba.conf"; > > +print HBA "local replication all trust\n"; > > +print HBA "host replication all 127.0.0.1/32 trust\n"; > > +print HBA "host replication all

Re: [HACKERS] Fix initdb for path with whitespace and at char

2014-04-30 Thread Andrew Dunstan
On 04/30/2014 06:31 AM, Heikki Linnakangas wrote: Andrew: you have a cygwin installation, don't you? Could you test if "pg_ctl start" works when the binaries are installed to a path that contains both a space and an @ sign, like "C:\white space\at@sign\install". I suspect it doesn't, but t

Re: [HACKERS] folder:lk/lk date:yesterday..

2014-04-30 Thread Andres Freund
On 2014-04-30 17:39:27 +0200, Andres Freund wrote: > Hi, > > Coverity flagged a couple of issues that seem to worth addressing by > changing the code instead of ignoring them: > > 01) heap_xlog_update() looks to coverity as if it could trigger a NULL > pointer dereference. That's because it t

[HACKERS] folder:lk/lk date:yesterday..

2014-04-30 Thread Andres Freund
Hi, Coverity flagged a couple of issues that seem to worth addressing by changing the code instead of ignoring them: 01) heap_xlog_update() looks to coverity as if it could trigger a NULL pointer dereference. That's because it thinks that oldtup.t_data is NULL if XLR_BKP_BLOCK(0) while re

[HACKERS] hooks not working in postgresql 9.3 (building from source)

2014-04-30 Thread Rajmohan C
hi, I need to use the hook feature of planner in postgresql 9.3 to perform a task. I am building postgresql from source. To start with, I wanted to know how to create a hook and attach that shared libray to postgresql. Hence i tried the examples given in " wiki.postgresql.org/images/e/e3/Hooks_

[HACKERS] Display of timestamp in pg_dump custom format

2014-04-30 Thread Bruce Momjian
The table of contents for pg_restore -l shows the time the archive was made as local time (it uses ctime()): ; Archive created at Wed Apr 30 10:03:28 2014 Is this clear enough that it is local time? Should we display this better, perhaps with a time zone designation? -- Bruce Momjian

Re: [HACKERS] "Considerer Harmful Considered Harmful" categorized as Mostly Harmless

2014-04-30 Thread Hannu Krosing
On 04/30/2014 02:53 PM, Andrew Dunstan wrote: > > On 04/30/2014 02:35 AM, Heikki Linnakangas wrote: >> On 04/30/2014 01:27 AM, Stephen Frost wrote: >>> * Tom Lane (t...@sss.pgh.pa.us) wrote: Josh Berkus writes: > ... so let's stop using that phrase, OK? > http://meyerweb.com/eric/comm

Re: [HACKERS] GSoC on WAL-logging hash indexes

2014-04-30 Thread k...@rice.edu
On Wed, Apr 30, 2014 at 12:26:20AM -0700, Peter Geoghegan wrote: > On Mon, Mar 3, 2014 at 8:12 AM, Robert Haas wrote: > >> As a GSoC student, I will implement WAL recovery of hash indexes using the > >> other index types' WAL code as a guide. > > Frankly, I'm skeptical of the idea that hash index

Re: [HACKERS] Considerer Harmful Considered Harmful

2014-04-30 Thread Andrew Dunstan
On 04/30/2014 02:35 AM, Heikki Linnakangas wrote: On 04/30/2014 01:27 AM, Stephen Frost wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: Josh Berkus writes: ... so let's stop using that phrase, OK? http://meyerweb.com/eric/comment/chech.html Shrug ... what I see there is a rant from a guy wit

Re: [HACKERS] includedir_internal headers are not self-contained

2014-04-30 Thread Heikki Linnakangas
On 04/29/2014 06:00 PM, Tom Lane wrote: Andrew Dunstan writes: On 04/29/2014 02:56 AM, Heikki Linnakangas wrote: On 04/28/2014 10:32 PM, Tom Lane wrote: Meh. I still think it's a bad idea to have CATALOG_VERSION_NO getting compiled into libpgcommon.a, where there will be no way to cross-chec

Re: [HACKERS] Fix initdb for path with whitespace and at char

2014-04-30 Thread Heikki Linnakangas
I committed the non-invasive fixes to backbranches (and master too, just to keep it in sync), but the attached is what I came up with for master. There are a couple of places in the code where we have #ifdef WIN32 code that uses CreateProcess with "CMD /C ..." directly. I believe those are cur

Re: [HACKERS] Minor improvement to fdwhandler.sgml

2014-04-30 Thread Etsuro Fujita
(2014/04/28 23:31), Robert Haas wrote: On Thu, Apr 24, 2014 at 7:59 AM, Etsuro Fujita wrote: The patch attached improves docs in fdwhandler.sgml a little bit. When you submit a patch, it's helpful to describe what the patch actually does, rather than just saying it makes things better. For e

Re: [HACKERS] Fix initdb for path with whitespace and at char

2014-04-30 Thread Heikki Linnakangas
On 04/30/2014 07:39 AM, Amit Kapila wrote: On Wed, Apr 30, 2014 at 3:57 AM, Tom Lane wrote: Heikki Linnakangas writes: This looks correct to me. popen() requires SYSTEMQUOTEs on Windows, like system() does. It seems right now SYSTEMQUOTE is used before popen both for Windows and non-Window

Re: [HACKERS] GSoC on WAL-logging hash indexes

2014-04-30 Thread Peter Geoghegan
On Mon, Mar 3, 2014 at 8:12 AM, Robert Haas wrote: >> As a GSoC student, I will implement WAL recovery of hash indexes using the >> other index types' WAL code as a guide. Frankly, I'm skeptical of the idea that hash indexes will ever really be useful. I realize that that's a counter-intuitive co