Re: [HACKERS] wal-size limited to 16MB - Performance issue for subsequent backup

2014-10-21 Thread Heikki Linnakangas
On 10/20/2014 11:02 PM, jes...@krogh.cc wrote: >>I do suspect the majority is from 30 concurrent processes updating an >>506GB GIN index, but it would be nice to confirm that. There is also a >>message-queue in the DB with a fairly high turnaround. > >A 506GB GIN index? Uh, interesting :). What'

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-10-21 Thread Noah Misch
On Mon, Oct 20, 2014 at 01:03:31AM -0400, Noah Misch wrote: > I reproduced narwhal's problem using its toolchain on another 32-bit Windows > Server 2003 system. The crash happens at the SHGetFolderPath() call in > pqGetHomeDirectory(). A program can acquire that function via shfolder.dll or > via

Re: [HACKERS] wal-size limited to 16MB - Performance issue for subsequent backup

2014-10-21 Thread Craig Ringer
On 10/21/2014 03:03 AM, jes...@krogh.cc wrote: > That being said, along comes the backup, scheduled ones a day and tries to > read off these wal-files, which to the backup looks like "an awfull lot of > small files", our backup utillized a single thread to read of those files > and levels of at re

Re: [HACKERS] Deferring some AtStart* allocations?

2014-10-21 Thread Andres Freund
Hi, On 2014-10-09 15:01:19 -0400, Robert Haas wrote: > /* > @@ -960,18 +966,38 @@ AtEOXact_Inval(bool isCommit) ... > + /* > + * We create invalidation stack entries lazily, so the parent > might > + * not have one. Instead of creating one, moving all the d

Re: [HACKERS] Deferring some AtStart* allocations?

2014-10-21 Thread Andres Freund
On 2014-10-08 13:52:14 -0400, Robert Haas wrote: > On Sun, Jun 29, 2014 at 9:12 PM, Tom Lane wrote: > > Meh. Even "SELECT 1" is going to be doing *far* more pallocs than that to > > get through raw parsing, parse analysis, planning, and execution startup. > > If you can find a few hundred pallocs

Re: [HACKERS] Possible micro-optimization in CacheInvalidateHeapTuple

2014-10-21 Thread Jim Nasby
On 10/13/14, 8:28 PM, Tom Lane wrote: Jim Nasby writes: CacheInvalidateHeapTuple currently does the following tests first; would there be a performance improvement to testing the system relation case first? We're almost never in bootstrap mode, so that test is almost always a waste. Is there

Re: [HACKERS] Proposal: Log inability to lock pages during vacuum

2014-10-21 Thread Tom Lane
Jim Nasby writes: > - What happens if we run out of space to remember skipped blocks? You forget some, and are no worse off than today. (This might be an event worthy of logging, if the array is large enough that we don't expect it to happen often ...) regards, tom lane

Re: [HACKERS] Proposal: Log inability to lock pages during vacuum

2014-10-21 Thread Jim Nasby
On 10/21/14, 5:39 PM, Alvaro Herrera wrote: Jim Nasby wrote: Currently, a non-freeze vacuum will punt on any page it can't get a cleanup lock on, with no retry. Presumably this should be a rare occurrence, but I think it's bad that we just assume that and won't warn the user if something bad is

[HACKERS] Spurious set in heap_prune_chain()

2014-10-21 Thread Jim Nasby
In heap_prune_chain(): tup.t_tableOid = RelationGetRelid(relation); rootlp = PageGetItemId(dp, rootoffnum); /* * If it's a heap-only tuple, then it is not the start of a HOT chain. */ if (ItemIdIsNormal(rootlp)) { htup =

Re: [HACKERS] Proposal: Log inability to lock pages during vacuum

2014-10-21 Thread Alvaro Herrera
Jim Nasby wrote: > Currently, a non-freeze vacuum will punt on any page it can't get a > cleanup lock on, with no retry. Presumably this should be a rare > occurrence, but I think it's bad that we just assume that and won't > warn the user if something bad is going on. I think if you really want

Re: [HACKERS] Simplify calls of pg_class_aclcheck when multiple modes are used

2014-10-21 Thread Fabrízio de Royes Mello
Em terça-feira, 21 de outubro de 2014, Michael Paquier < michael.paqu...@gmail.com> escreveu: > On Wed, Oct 22, 2014 at 5:03 AM, Peter Eisentraut > wrote: > >> While looking at this, I wrote a few tests cases for sequence >> privileges, because that was not covered at all. That patch is attached

Re: [HACKERS] expected/sequence_1.out obsolete?

2014-10-21 Thread Tom Lane
Michael Paquier writes: > I don't think that this is a good idea, have a look at bb3f839 explaining > that this alternate output may happen when a checkpoint kicks in. Simple > patch is attached. Pushed, thanks. (The 9.4 branch was broken too, but differently :-() regard

Re: [HACKERS] Autovacuum fails to keep visibility map up-to-date in mostly-insert-only-tables

2014-10-21 Thread Jim Nasby
On 10/21/14, 4:36 PM, Jeff Janes wrote: On Mon, Oct 20, 2014 at 5:46 PM, Andres Freund mailto:and...@2ndquadrant.com>> wrote: On 2014-10-20 17:43:26 -0700, Josh Berkus wrote: > On 10/20/2014 05:39 PM, Jim Nasby wrote: > > Or maybe vacuum isn't the right way to handle some of these sc

Re: [HACKERS] Simplify calls of pg_class_aclcheck when multiple modes are used

2014-10-21 Thread Michael Paquier
On Wed, Oct 22, 2014 at 5:03 AM, Peter Eisentraut wrote: > While looking at this, I wrote a few tests cases for sequence > privileges, because that was not covered at all. That patch is attached. > +1 for those tests. -- Michael

Re: [HACKERS] expected/sequence_1.out obsolete?

2014-10-21 Thread Michael Paquier
On Wed, Oct 22, 2014 at 4:46 AM, Peter Eisentraut wrote: > expected/sequence_1.out hasn't been updated at least since > > commit d90ced8bb22194cbb45f58beb0961251103aeff5 > Date: Thu Oct 3 16:17:18 2013 -0400 > > and nobody appears to have complained. > > Can it be removed? > I don't think that

Re: [HACKERS] expected/sequence_1.out obsolete?

2014-10-21 Thread Tom Lane
Peter Eisentraut writes: > expected/sequence_1.out hasn't been updated at least since > commit d90ced8bb22194cbb45f58beb0961251103aeff5 > Date: Thu Oct 3 16:17:18 2013 -0400 > and nobody appears to have complained. > Can it be removed? No, it needs to be fixed. The alternate output from "sele

Re: [HACKERS] Autovacuum fails to keep visibility map up-to-date in mostly-insert-only-tables

2014-10-21 Thread Jeff Janes
On Mon, Oct 20, 2014 at 5:46 PM, Andres Freund wrote: > On 2014-10-20 17:43:26 -0700, Josh Berkus wrote: > > On 10/20/2014 05:39 PM, Jim Nasby wrote: > > > Or maybe vacuum isn't the right way to handle some of these scenarios. > > > It's become the catch-all for all of this stuff, but maybe that

Re: [HACKERS] wal-size limited to 16MB - Performance issue for subsequent backup

2014-10-21 Thread Jeff Janes
On Mon, Oct 20, 2014 at 12:03 PM, wrote: > Hi. > > One of our "production issues" is that the system generates lots of > wal-files, lots is like 151952 files over the last 24h, which is about > 2.4TB worth of WAL files. I wouldn't say that isn't an issue by itself, > but the system does indeed wo

Re: [HACKERS] Allow format 0000-0000-0000 in postgresql MAC parser

2014-10-21 Thread Peter Eisentraut
On 10/17/14 6:37 PM, Ali Akbar wrote: > On a side note, i'm noticing from > http://en.wikipedia.org/wiki/MAC_address, that there is three numbering > namespace for MAC: MAC-48, EUI-48 and EUI-64. The last one is 64 bits > long (8 bytes). Currently PostgreSQL's macaddr is only 6 bytes long. > Should

Re: [HACKERS] Allow format 0000-0000-0000 in postgresql MAC parser

2014-10-21 Thread Peter Eisentraut
On 10/1/14 8:34 AM, Herwin Weststrate wrote: > It has been registered now > (https://commitfest.postgresql.org/action/patch_view?id=1585). I've got > an updated version of the patch with the documentation fix. committed -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To ma

Re: [HACKERS] Simplify calls of pg_class_aclcheck when multiple modes are used

2014-10-21 Thread Peter Eisentraut
On 8/27/14 8:02 AM, Michael Paquier wrote: > In a couple of code paths we do the following to check permissions on an > object: > if (pg_class_aclcheck(relid, userid, ACL_USAGE) != ACLCHECK_OK && > pg_class_aclcheck(relid, userid, ACL_UPDATE) != ACLCHECK_OK) > ereport(ERROR, blah); > > Wou

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-21 Thread Brightwell, Adam
Peter, You patch is missing the files src/include/catalog/pg_diralias.h, > src/include/commands/diralias.h, and src/backend/commands/diralias.c. > > (Hint: git add -N) > Yikes, sorry about that, not sure how that happened. Attached is an updated patch. -Adam -- Adam Brightwell - adam.brightw.

[HACKERS] expected/sequence_1.out obsolete?

2014-10-21 Thread Peter Eisentraut
expected/sequence_1.out hasn't been updated at least since commit d90ced8bb22194cbb45f58beb0961251103aeff5 Date: Thu Oct 3 16:17:18 2013 -0400 and nobody appears to have complained. Can it be removed? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to y

Re: [HACKERS] PostgreSQL Service Name Enhancement - Wildcard support for LDAP/DNS lookup

2014-10-21 Thread Tom Lane
"Doyle, Bryan" writes: > Would specifying a special value for the service name, perhaps [%], be an > acceptable implementation of this enhancement/fix to my above concerns? > Example: > # comment > [%] > host=%.domain.com > port=5433 > user=admin This doesn't seem like a ter

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-21 Thread Peter Eisentraut
You patch is missing the files src/include/catalog/pg_diralias.h, src/include/commands/diralias.h, and src/backend/commands/diralias.c. (Hint: git add -N) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailp

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-21 Thread Peter Eisentraut
On 10/16/14 12:01 PM, Stephen Frost wrote: > This started out as a request for a non-superuser to be able to review > the log files without needing access to the server. I think that can be done with a security-definer function. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.

[HACKERS] PostgreSQL Service Name Enhancement - Wildcard support for LDAP/DNS lookup

2014-10-21 Thread Doyle, Bryan
Hi, I am looking to patch the LDAP Service Name feature and would like some feedback prior to doing so. In general, while I personally view this as a bug fix, it could also easily be considered to simply be an enhancement to current functionality and therefore the below is written as a propos

Re: [HACKERS] run xmllint during build (was Re: need xmllint on borka)

2014-10-21 Thread Peter Eisentraut
On 9/14/14 3:34 AM, Fabien COELHO wrote: >> and rebased this patch on top of that. > > Applied and tested, everything looks fine. > > The only remaining question is whether the xmllint check should always > be called. You stated that it was stricter than sgml processing, so I > would think it wor

Re: [HACKERS] Question about RI checks

2014-10-21 Thread Nick Barnes
On Wed, Oct 22, 2014 at 3:19 AM, Kevin Grittner wrote: > > It doesn't seem like this analysis considers all of the available ON > DELETE and ON UPDATE behaviors available. Besides RESTRICT there is > CASCADE, SET NULL, SET DEFAULT, and NO ACTION. Some of those > require updating the referencing

[HACKERS] Getting rid of "accept incoming network connections" prompts on OS X

2014-10-21 Thread Tom Lane
If you do any Postgres development on OS X, you've probably gotten seriously annoyed by the way that, every single time you reinstall the postmaster executable, you get a dialog box asking whether you'd like to allow it to accept incoming network connections. (At least, you do unless you disable t

Re: [HACKERS] Question about RI checks

2014-10-21 Thread Nick Barnes
Thanks! I've been mulling this over for weeks; nice to know it wasn't just staring me in the face... So in conclusion, the lock avoids raising constraint violation errors in > a few cases in READ COMMITTED mode. In REPEATABLE READ mode, it converts > some > constraint violation errors into seriali

Re: [HACKERS] Question about RI checks

2014-10-21 Thread Kevin Grittner
Florian Pflug wrote: > So in conclusion, the lock avoids raising constraint violation errors in > a few cases in READ COMMITTED mode. In REPEATABLE READ mode, it converts some > constraint violation errors into serialization failures. Or at least that's > how it looks to me. It doesn't seem lik

Re: [HACKERS] [TODO] Track number of files ready to be archived in pg_stat_archiver

2014-10-21 Thread Brightwell, Adam
Julien, > Actually, I used the same loop as the archiver one (see > backend/postmaster/pgarch.c, function pgarch_readyXlog) to get the exact > same number of files. > Ah, I see. > If we change it in this patch, it would be better to change it everywhere. > What do you think ? > Hmm... I'd hav

Re: [HACKERS] Question about RI checks

2014-10-21 Thread Florian Pflug
(CCing Alvaro, since he implemented KEY SHARE locks) On Oct16, 2014, at 15:51 , Nick Barnes wrote: > One of the queries in ri_triggers.c has be a little baffled. > > For (relatively) obvious reasons, a FK insert triggers a SELECT 1 FROM pk_rel > ... FOR KEY SHARE. > For not-so-obvious reasons,

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-10-21 Thread Amit Kapila
On Wed, Oct 8, 2014 at 6:17 PM, Andres Freund wrote: > > On 2014-06-25 19:06:32 +0530, Amit Kapila wrote: > > 2. > > LWLockWakeup() > > { > > .. > > #ifdef LWLOCK_STATS > > lwstats->spin_delay_count += SpinLockAcquire(&lock->mutex); > > #else > > SpinLockAcquire(&lock->mutex); > > #endif > > .. >

Re: [HACKERS] Trailing comma support in SELECT statements

2014-10-21 Thread Tom Lane
Jim Nasby writes: > On 10/20/14, 11:16 AM, Andrew Dunstan wrote: >> The JSON spec is quite clear on this. Leading and trailing commas are not >> allowed. I would fight tooth and nail not to allow it for json (and by >> implication jsonb, since they use literally the same parser - in fact we do

Re: [HACKERS] Inconsistencies in documentation of row-level locking

2014-10-21 Thread Michael Paquier
On Tue, Oct 21, 2014 at 10:26 AM, Jim Nasby wrote: > Did this get committed? Should probably add it to the commitfest if not... > Already done in CF3, I should have mentioned it: https://commitfest.postgresql.org/action/patch_view?id=1594 -- Michael

Re: [HACKERS] Would you help to review our modifications

2014-10-21 Thread Merlin Moncure
On Mon, Oct 20, 2014 at 11:02 AM, David G Johnston wrote: > rohtodeveloper wrote >> So how to deal with this kind of situation if I want a implicit >> conversion? > > As of the out-of-support 8.3 release many of the implicit casts previously > defined have been changed to explicit casts. It is a

Re: [HACKERS] Patch: Add launchd Support

2014-10-21 Thread Florian Pflug
On Oct21, 2014, at 02:53 , Wim Lewis wrote: > >> 2) AFAICS, this .plist file doesn't do anything about launchd's habit of >> not waiting for the network to come up. >> If true, the job will be kept alive as long as the network is up, where >> up is defined as at least one non-loopback interfac

Re: [HACKERS] [TODO] Track number of files ready to be archived in pg_stat_archiver

2014-10-21 Thread Julien Rouhaud
On Tue, Oct 21, 2014 at 7:35 AM, Brightwell, Adam < adam.brightw...@crunchydatasolutions.com> wrote: > Julien, > > The following is an initial review: > > Thanks for the review. > * Applies cleanly to master (f330a6d). > * Regression tests updated and pass, including 'check-world'. > * Documenta

Re: [HACKERS] [PATCH] Simplify EXISTS subqueries containing LIMIT

2014-10-21 Thread Marti Raudsepp
Hi Thanks for taking a look. On Sun, Oct 19, 2014 at 1:22 PM, David Rowley wrote: > the argument for this would > have been much stronger if anti join support had just been added last week. > It's been quite a few years now and the argument for this must be getting > weaker with every release.

Re: [HACKERS] Patch: Add launchd Support

2014-10-21 Thread Marti Raudsepp
On Tue, Oct 21, 2014 at 3:53 AM, Wim Lewis wrote: > I think the idea of OnDemand is for launchd items to act a bit like inetd > does: launchd creates the listening socket (or mach port or file-change > notification) on the port specified in the plist, and only starts the > process when someone tri

Re: [HACKERS] inherit support for foreign tables

2014-10-21 Thread Etsuro Fujita
(2014/10/14 20:00), Etsuro Fujita wrote: Here are separated patches. fdw-chk.patch - CHECK constraints on foreign tables fdw-inh.patch - table inheritance with foreign tables The latter has been created on top of [1]. [1] http://www.postgresql.org/message-id/540da168.3040...@lab.ntt.co.jp

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.9

2014-10-21 Thread Amit Kapila
On Fri, Oct 17, 2014 at 11:41 PM, Andres Freund wrote: > On 2014-10-17 17:14:16 +0530, Amit Kapila wrote: > > On Tue, Oct 14, 2014 at 11:34 AM, Amit Kapila > > wrote: > > HEAD – commit 494affb + wait free lw_shared_v2 > > > > Shared_buffers=8GB; Scale Factor = 3000 > > > > Client Count/No. Of

Re: [HACKERS] alter user/role CURRENT_USER

2014-10-21 Thread Kyotaro HORIGUCHI
Hello, > Kyotaro, > > Food for thought. Couldn't you reduce the following block: > > + if (strcmp(stmt->role, "current_user") == 0) > + { > + roleid = GetUserId(); > + tuple = SearchSysCache1(AUTHOID, ObjectIdGetDatum(roleid)); > + if (!HeapTupleIsValid(tuple)) > + ereport(ERROR, > + (errcode(E

Re: [HACKERS] alter user/role CURRENT_USER

2014-10-21 Thread Kyotaro HORIGUCHI
Thank you for reviewing, 2014 13:10:57 +0530, Rushabh Lathia wrote in > I gone through patch and here is the review for this patch: > > > .) patch go applied on master branch with patch -p1 command >(git apply failed) > .) regression make check run fine > .) testcase coverage is missing