Re: [HACKERS] Long paths for tablespace leads to uninterruptible hang in Windows

2014-02-14 Thread Amit Kapila
On Fri, Feb 14, 2014 at 8:27 AM, Bruce Momjian wrote: > On Tue, Jan 7, 2014 at 12:33:33PM +0530, Amit Kapila wrote: >> On Thu, Oct 31, 2013 at 8:58 AM, Amit Kapila wrote: >> > On Wed, Oct 16, 2013 at 1:44 AM, Amit Kapila >> > wrote: >> >> On Tue, Oct 15, 2013 at 6:28 PM, Magnus Hagander >> >

Re: [HACKERS] Small psql memory fix

2014-02-14 Thread Bruce Momjian
On Fri, Feb 14, 2014 at 11:52:42PM -0500, Tom Lane wrote: > Bruce Momjian writes: > > On Fri, Feb 14, 2014 at 11:28:49PM -0500, Tom Lane wrote: > >> The first and third hunks look to me like they introduce memory > >> leaks, not fix them. The second hunk is probably safe enough, > > > The first

Re: [HACKERS] Small psql memory fix

2014-02-14 Thread Tom Lane
Bruce Momjian writes: > On Fri, Feb 14, 2014 at 11:28:49PM -0500, Tom Lane wrote: >> The first and third hunks look to me like they introduce memory >> leaks, not fix them. The second hunk is probably safe enough, > The first and third just move the free into blocks where we have already > teste

Re: [HACKERS] Small psql memory fix

2014-02-14 Thread Bruce Momjian
On Fri, Feb 14, 2014 at 11:28:49PM -0500, Tom Lane wrote: > Bruce Momjian writes: > > The attached tiny patch fixes a small leak in psql's \gset command and > > simplifies memory freeing in two places. > > The first and third hunks look to me like they introduce memory > leaks, not fix them. The

Re: [HACKERS] Small psql memory fix

2014-02-14 Thread Tom Lane
Bruce Momjian writes: > The attached tiny patch fixes a small leak in psql's \gset command and > simplifies memory freeing in two places. The first and third hunks look to me like they introduce memory leaks, not fix them. The second hunk is probably safe enough, although I'm not sure the case c

Re: [HACKERS] Patch: show xid and xmin in pg_stat_activity and pg_stat_replication

2014-02-14 Thread Robert Haas
On Wed, Feb 12, 2014 at 8:00 AM, Christian Kruse wrote: > On Wednesday 12 February 2014 11:14:56 Andres Freund wrote: >> But they do take up shared memory without really needing to. I >> personally don't find that too bad, it's not much memory. If we want to >> avoid it we could have a LocalPgBack

[HACKERS] Small psql memory fix

2014-02-14 Thread Bruce Momjian
The attached tiny patch fixes a small leak in psql's \gset command and simplifies memory freeing in two places. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + Everyone has their own god. + diff --git a/src/bin/psql/command.c

Re: [HACKERS] Recovery inconsistencies, standby much larger than primary

2014-02-14 Thread Tom Lane
Andres Freund writes: > On 2014-02-14 20:46:01 +, Greg Stark wrote: >> Going over this I think this is still a potential issue: >> On 31 Jan 2014 15:56, "Andres Freund" wrote: >>> I am not sure that explains the issue, but I think the redo action for >>> truncation is not safe across crashes.

Re: [HACKERS] Memory ordering issue in LWLockRelease, WakeupWaiters, WALInsertSlotRelease

2014-02-14 Thread Florian Pflug
On Feb14, 2014, at 19:21 , Andres Freund wrote: > On 2014-02-14 18:49:33 +0100, Florian Pflug wrote: >> Well, the assumption isn't all that new. We already have the situation that >> a PGPROC may be not on any wait queue, yet its lwWaitLink may be non-NULL. >> Currently, the process who took it of

[HACKERS] Re: [BUGS] BUG #9210: PostgreSQL string store bug? not enforce check with correct characterSET/encoding

2014-02-14 Thread digoal
HI, Thanks very much. We use dblink or foreign table migrate datas instead pg_dump now resolve the error data load problem. -- 公益是一辈子的事,I'm Digoal,Just Do It. At 2014-02-14 04:49:08,"Tom Lane" wrote: >dig...@126.com writes: >> select t, t::bytea from convert_from('\xeec1', 'sql_asci

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-14 Thread Tom Lane
Hiroshi Inoue writes: > (2014/02/15 2:32), Tom Lane wrote: >> (BTW, narwhal is evidently not trying to build plpython. I wonder >> why not?) > Due to *initializer element is not constant* error which also can be > see on my old machine. Hmm, isn't that one of the symptoms of inadequacies in --

Re: [HACKERS] should we add a XLogRecPtr/LSN SQL type?

2014-02-14 Thread Michael Paquier
On Thu, Feb 6, 2014 at 11:26 AM, Michael Paquier wrote: > Here are updated patches to use pg_lsn instead of pglsn... Should I register this patch somewhere to avoid having it lost in the void? Regards, -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make cha

Re: [HACKERS] Ctrl+C from sh can shut down daemonized PostgreSQL cluster

2014-02-14 Thread Greg Stark
On 14 Feb 2014 23:07, "Tom Lane" wrote: > > If this is, as it sounds to be, a Solaris shell bug, doesn't it > affect other daemons too? This is simmering i never exactly followed but i think if the shell doesn't support job control it's expected behaviour, not a bug. Only shells that support job

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-14 Thread Hiroshi Inoue
(2014/02/15 2:32), Tom Lane wrote: > I wrote: >> Hiroshi Inoue writes: >>> One thing I'm wondering about is that plperl is linking perlxx.lib >>> not libperlxx.a. I made a patch following plpython and it also >>> works here. >>> Is it worth trying? > >> I hadn't noticed that part of plpython's Ma

Re: [HACKERS] Recovery inconsistencies, standby much larger than primary

2014-02-14 Thread Andres Freund
On 2014-02-14 20:46:01 +, Greg Stark wrote: > Going over this I think this is still a potential issue: > > On 31 Jan 2014 15:56, "Andres Freund" wrote: > > > > > I am not sure that explains the issue, but I think the redo action for > > truncation is not safe across crashes. A XLOG_SMGR_TRU

Re: [HACKERS] New hook after raw parsing, before analyze

2014-02-14 Thread Greg Stark
On Fri, Feb 14, 2014 at 9:16 PM, David Beck wrote: > What inspired me is the scriptable query rewrite in > http://dev.mysql.com/downloads/mysql-proxy/ > The hook I proposed would be a lot nicer in Postgres because the raw parsing > is already done at this point while in mysql-proxy that has to b

Re: [HACKERS] New hook after raw parsing, before analyze

2014-02-14 Thread Tom Lane
Greg Stark writes: > On Fri, Feb 14, 2014 at 9:16 PM, David Beck wrote: >> Another point I liked in mysql is the possibility to write info schema >> plugins: >> http://dev.mysql.com/doc/refman/5.1/en/writing-information-schema-plugins.html >> Like a virtual catalog. Is there anything similar in

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-02-14 Thread Haribabu Kommi
On Feb 15, 2014 9:19 AM, "Alvaro Herrera" wrote: > > Haribabu Kommi escribió: > > > I changed the balance cost calculations a little bit to give priority to > > the user provided per table autovacuum parameters. > > If any user specified per table vacuum parameters exists and those are > > differe

Re: [HACKERS] New hook after raw parsing, before analyze

2014-02-14 Thread Greg Stark
On Fri, Feb 14, 2014 at 9:16 PM, David Beck wrote: > Another point I liked in mysql is the possibility to write info schema > plugins: > http://dev.mysql.com/doc/refman/5.1/en/writing-information-schema-plugins.html > Like a virtual catalog. Is there anything similar in Postgres? The documentat

Re: [HACKERS] Ctrl+C from sh can shut down daemonized PostgreSQL cluster

2014-02-14 Thread Tom Lane
Kevin Grittner writes: > What is surprising is that the postmaster doesn't set up its own > process group when it is running as a daemon.  We probably don't > want to change that when postgres is run directly from a command > line for development or diagnostic purposes, but Noah suggested > perhap

[HACKERS] Ctrl+C from sh can shut down daemonized PostgreSQL cluster

2014-02-14 Thread Kevin Grittner
We have had a case where a production cluster was accidentally shut down by a customer who used Ctrl+C in the same sh session in which they had (long before) run pg_ctl start.  We have only seen this in sh on Solaris.  Other shells on Solaris don't behave this way, nor does sh on tested versions of

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-02-14 Thread Alvaro Herrera
Haribabu Kommi escribió: > I changed the balance cost calculations a little bit to give priority to > the user provided per table autovacuum parameters. > If any user specified per table vacuum parameters exists and those are > different with guc vacuum parameters then the > balance cost calculati

Re: [HACKERS] [SQL] Comparison semantics of CHAR data type

2014-02-14 Thread Bruce Momjian
On Thu, Feb 13, 2014 at 09:47:01PM -0500, Bruce Momjian wrote: > On Wed, Oct 16, 2013 at 02:17:11PM -0400, Bruce Momjian wrote: > > > > You can see the UTF8 case is fine because \n is considered greater > > > > than space, but in the C locale, where \n is less than space, the > > > > false return v

Re: [HACKERS] New hook after raw parsing, before analyze

2014-02-14 Thread David Beck
I think I’m gonna need to dig into the planner to fully understand your points. Thank you for the insights. I was more into putting the knowledge of the legacy system into the an extension and my codebase. Now I see better use of the planner would help. Thank you. What inspired me is the script

Re: [HACKERS] Recovery inconsistencies, standby much larger than primary

2014-02-14 Thread Greg Stark
Going over this I think this is still a potential issue: On 31 Jan 2014 15:56, "Andres Freund" wrote: > > I am not sure that explains the issue, but I think the redo action for > truncation is not safe across crashes. A XLOG_SMGR_TRUNCATE will just > do a smgrtruncate() (and then mdtruncate) wh

Re: [HACKERS] Release schedule for 9.3.3?

2014-02-14 Thread Alvaro Herrera
Andres Freund wrote: > On 2014-02-14 13:33:46 -0500, Josh Berkus wrote: > > On 02/14/2014 01:11 PM, Andres Freund wrote: > > > Hi, > > > > > > On 2014-02-14 13:08:34 -0500, Josh Berkus wrote: > > >> Do the 9.3.3 replication fixes mean that users should reclone their > > >> replicas, like 9.3.2 did

Re: [HACKERS] HBA files w/include support?

2014-02-14 Thread Jeff Janes
On Fri, Feb 14, 2014 at 6:33 AM, Bruce Momjian wrote: > On Fri, Feb 14, 2014 at 03:28:23AM -0500, Stephen Frost wrote: > > Bruce, > > > Having @include and directory.d-style capabilities for pg_hba.conf *and* > > pg_ident.conf would make managing larger environments much better. > > There has bee

Re: [HACKERS] Release schedule for 9.3.3?

2014-02-14 Thread Andres Freund
On 2014-02-14 13:33:46 -0500, Josh Berkus wrote: > On 02/14/2014 01:11 PM, Andres Freund wrote: > > Hi, > > > > On 2014-02-14 13:08:34 -0500, Josh Berkus wrote: > >> Do the 9.3.3 replication fixes mean that users should reclone their > >> replicas, like 9.3.2 did? Or not? > > > > Which replicati

Re: [HACKERS] Release schedule for 9.3.3?

2014-02-14 Thread Josh Berkus
On 02/14/2014 01:11 PM, Andres Freund wrote: > Hi, > > On 2014-02-14 13:08:34 -0500, Josh Berkus wrote: >> Do the 9.3.3 replication fixes mean that users should reclone their >> replicas, like 9.3.2 did? Or not? > > Which replication replication fixes are you referring to? > http://git.postgresq

Re: [HACKERS] Memory ordering issue in LWLockRelease, WakeupWaiters, WALInsertSlotRelease

2014-02-14 Thread Andres Freund
On 2014-02-14 18:49:33 +0100, Florian Pflug wrote: > > I wouldn't consider it a major architecture... And I am not sure how > > much out of order a CPU has to be to be affected by this, the read side > > uses spinlocks, which in most of the spinlock implementations implies a > > full memory barrier

Re: [HACKERS] Release schedule for 9.3.3?

2014-02-14 Thread Andres Freund
Hi, On 2014-02-14 13:08:34 -0500, Josh Berkus wrote: > Do the 9.3.3 replication fixes mean that users should reclone their > replicas, like 9.3.2 did? Or not? Which replication replication fixes are you referring to? http://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=ebde6c40148c9f811

Re: [HACKERS] Release schedule for 9.3.3?

2014-02-14 Thread Josh Berkus
All, Do the 9.3.3 replication fixes mean that users should reclone their replicas, like 9.3.2 did? Or not? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgr

Re: [HACKERS] git-review: linking commits to review discussion in git

2014-02-14 Thread Peter C Rigby
On 2014-01-28 07:10:45 Heikki Linnakangas wrote: > On 01/27/2014 11:36 PM, Murtuza Mukadam wrote: >> >> Hello All, >>We have linked peer review discussions on >> 'pgsql-hackers' to their respective commits within the main >> postgresql.git repository. You can view the linked review

Re: [HACKERS] Memory ordering issue in LWLockRelease, WakeupWaiters, WALInsertSlotRelease

2014-02-14 Thread Florian Pflug
On Feb14, 2014, at 16:51 , Andres Freund wrote: > On 2014-02-14 15:03:16 +0100, Florian Pflug wrote: >> On Feb14, 2014, at 14:07 , Andres Freund wrote: >>> On 2014-02-14 13:52:45 +0100, Florian Pflug wrote: > I agree we should do that, but imo not in the backbranches. Anything > more than

Re: [HACKERS] HBA files w/include support?

2014-02-14 Thread Jerry Sievers
Tom Lane writes: > Stephen Frost writes: > >> * Tom Lane (t...@sss.pgh.pa.us) wrote: >>> In short: I suspect this approach may be fixing the wrong thing. > >> I'm curious what you're thinking would be the right thing to fix here? > > I was asking for use-cases so we could figure out what's the r

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-14 Thread Tom Lane
I wrote: > Hiroshi Inoue writes: >> One thing I'm wondering about is that plperl is linking perlxx.lib >> not libperlxx.a. I made a patch following plpython and it also >> works here. >> Is it worth trying? > I hadn't noticed that part of plpython's Makefile before. Man, > that's an ugly techniq

Re: [HACKERS] HBA files w/include support?

2014-02-14 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: > On 2014-02-14 11:10:48 -0500, Tom Lane wrote: > > Stephen Frost writes: > > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > >> In short: I suspect this approach may be fixing the wrong thing. > > > > > I'm curious what you're thinking would be the ri

Re: [HACKERS] New hook after raw parsing, before analyze

2014-02-14 Thread Greg Stark
On Fri, Feb 14, 2014 at 2:28 PM, David Beck wrote: > Why is that a bad idea of rewriting the query before it reaches > transform/analyze (without ever accessing the catalog)? > > If that flexibility is acceptable to you, where would be the best place to > put it in? Well if there are two foreig

Re: [HACKERS] Memory ordering issue in LWLockRelease, WakeupWaiters, WALInsertSlotRelease

2014-02-14 Thread knizhnik
On 02/14/2014 08:28 PM, Andres Freund wrote: On 2014-02-14 20:23:32 +0400, knizhnik wrote: we'll trade correctness for cleanliness if we continue to reset lwWaitLink without protecting against the race. That's a bit of a weird trade-off to make. It's not just cleanliness, it's being able to ac

Re: [HACKERS] Memory ordering issue in LWLockRelease, WakeupWaiters, WALInsertSlotRelease

2014-02-14 Thread Andres Freund
On 2014-02-14 20:23:32 +0400, knizhnik wrote: > >>we'll trade correctness for cleanliness if we continue to reset lwWaitLink > >>without protecting against the race. That's a bit of a weird trade-off to > >>make. > > > >It's not just cleanliness, it's being able to actually debug crashes. > > >

Re: [HACKERS] Memory ordering issue in LWLockRelease, WakeupWaiters, WALInsertSlotRelease

2014-02-14 Thread knizhnik
On 02/14/2014 07:51 PM, Andres Freund wrote: On 2014-02-14 15:03:16 +0100, Florian Pflug wrote: On Feb14, 2014, at 14:07 , Andres Freund wrote: On 2014-02-14 13:52:45 +0100, Florian Pflug wrote: I agree we should do that, but imo not in the backbranches. Anything more than than the minimal fi

Re: [HACKERS] HBA files w/include support?

2014-02-14 Thread Andres Freund
On 2014-02-14 11:10:48 -0500, Tom Lane wrote: > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> In short: I suspect this approach may be fixing the wrong thing. > > > I'm curious what you're thinking would be the right thing to fix here? > > I was asking for use-cases so we

Re: [HACKERS] HBA files w/include support?

2014-02-14 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> In short: I suspect this approach may be fixing the wrong thing. > I'm curious what you're thinking would be the right thing to fix here? I was asking for use-cases so we could figure out what's the right thing ;-) The argument a

Re: [HACKERS] HBA files w/include support?

2014-02-14 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: > On 2014-02-14 11:03:19 -0500, Stephen Frost wrote: > > Also, all of the above ignores the pg_ident side of the house, which is > > even worse as you need an entry for every user, period, if you're using > > client-side SSL certificates or Kerberos/G

Re: [HACKERS] HBA files w/include support?

2014-02-14 Thread Andres Freund
On 2014-02-14 11:03:19 -0500, Stephen Frost wrote: > Also, all of the above ignores the pg_ident side of the house, which is > even worse as you need an entry for every user, period, if you're using > client-side SSL certificates or Kerberos/GSSAPI-based authentication > with full princ names. Wel

Re: [HACKERS] HBA files w/include support?

2014-02-14 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > Having @include and directory.d-style capabilities for pg_hba.conf *and* > > pg_ident.conf would make managing larger environments much better. > > I'm a little suspicious of this, mainly because pg_hba searching is > necessarily

Re: [HACKERS] Memory ordering issue in LWLockRelease, WakeupWaiters, WALInsertSlotRelease

2014-02-14 Thread Andres Freund
On 2014-02-14 15:03:16 +0100, Florian Pflug wrote: > On Feb14, 2014, at 14:07 , Andres Freund wrote: > > On 2014-02-14 13:52:45 +0100, Florian Pflug wrote: > >>> I agree we should do that, but imo not in the backbranches. Anything > >>> more than than the minimal fix in that code should be avoided

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-14 Thread Tom Lane
Hiroshi Inoue writes: > (2014/02/12 8:30), Tom Lane wrote: >> Not very clear what's going on there; could this be a problem in >> narwhal's admittedly-ancient toolchain? > The error doesn't occur here (un)fortunately. > One thing I'm wondering about is that plperl is linking perlxx.lib > not libp

Re: [HACKERS] Memory ordering issue in LWLockRelease, WakeupWaiters, WALInsertSlotRelease

2014-02-14 Thread Florian Pflug
On Feb14, 2014, at 16:32 , Andres Freund wrote: > On 2014-02-14 10:26:07 -0500, Tom Lane wrote: >> Florian Pflug writes: >>> Another idea for a fix would be to conflate lwWaiting and lwWaitLink into >>> one >>> field. We could replace "lwWaiting" by "lwWaitLink != NULL" everywhere it's >>> teste

Re: [HACKERS] Memory ordering issue in LWLockRelease, WakeupWaiters, WALInsertSlotRelease

2014-02-14 Thread Andres Freund
On 2014-02-14 10:26:07 -0500, Tom Lane wrote: > Florian Pflug writes: > > Another idea for a fix would be to conflate lwWaiting and lwWaitLink into > > one > > field. We could replace "lwWaiting" by "lwWaitLink != NULL" everywhere it's > > tested, and set lwWaitLink to some special non-NULL value

Re: [HACKERS] HBA files w/include support?

2014-02-14 Thread Andres Freund
On 2014-02-14 10:19:30 -0500, Tom Lane wrote: > Stephen Frost writes: > > Having @include and directory.d-style capabilities for pg_hba.conf *and* > > pg_ident.conf would make managing larger environments much better. > > I'm a little suspicious of this, mainly because pg_hba searching is > neces

Re: [HACKERS] Memory ordering issue in LWLockRelease, WakeupWaiters, WALInsertSlotRelease

2014-02-14 Thread Tom Lane
Florian Pflug writes: > Another idea for a fix would be to conflate lwWaiting and lwWaitLink into one > field. We could replace "lwWaiting" by "lwWaitLink != NULL" everywhere it's > tested, and set lwWaitLink to some special non-NULL value (say 0x1) when we > enqueue a PGPROC, instead of setting i

Re: [HACKERS] HBA files w/include support?

2014-02-14 Thread Tom Lane
Stephen Frost writes: > Having @include and directory.d-style capabilities for pg_hba.conf *and* > pg_ident.conf would make managing larger environments much better. I'm a little suspicious of this, mainly because pg_hba searching is necessarily linear (and none too cheap per-entry). I think any

Re: [HACKERS] HBA files w/include support?

2014-02-14 Thread Fabrízio de Royes Mello
On Fri, Feb 14, 2014 at 12:36 PM, Stephen Frost wrote: > > * Bruce Momjian (br...@momjian.us) wrote: > > In an ideal world we would have a tool where you could plug in a > > username, database, IP address, and test pg_hba.conf file and it would > > report what line is matched. > > That's not a bad

Re: [HACKERS] HBA files w/include support?

2014-02-14 Thread Bruce Momjian
On Fri, Feb 14, 2014 at 09:34:59AM -0500, Stephen Frost wrote: > * Bruce Momjian (br...@momjian.us) wrote: > > On Fri, Feb 14, 2014 at 03:28:23AM -0500, Stephen Frost wrote: > > > Having @include and directory.d-style capabilities for pg_hba.conf *and* > > > pg_ident.conf would make managing larger

Re: [HACKERS] HBA files w/include support?

2014-02-14 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: > In an ideal world we would have a tool where you could plug in a > username, database, IP address, and test pg_hba.conf file and it would > report what line is matched. That's not a bad idea, but we don't expose the logic that figures that out today.. I

Re: [HACKERS] HBA files w/include support?

2014-02-14 Thread Magnus Hagander
On Fri, Feb 14, 2014 at 3:32 PM, Bruce Momjian wrote: > On Thu, Feb 13, 2014 at 11:28:45PM -0600, Jerry Sievers wrote: > > > One issue with this is that pg_hba.conf is order sensitive, which could > > > become a trap for the unwary if includes are used carelessly. > > > > Indeed. > > > > The othe

Re: [HACKERS] HBA files w/include support?

2014-02-14 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: > On Fri, Feb 14, 2014 at 03:28:23AM -0500, Stephen Frost wrote: > > Having @include and directory.d-style capabilities for pg_hba.conf *and* > > pg_ident.conf would make managing larger environments much better. > > There has been some talk about providing

Re: [HACKERS] HBA files w/include support?

2014-02-14 Thread Bruce Momjian
On Fri, Feb 14, 2014 at 03:28:23AM -0500, Stephen Frost wrote: > Bruce, > > * Bruce Momjian (br...@momjian.us) wrote: > > On Thu, Feb 13, 2014 at 08:24:27PM -0600, Jerry Sievers wrote: > > > I'm aware of how a pg_hba.conf file can refer to other files for > > > including @lists of users, etc. > >

Re: [HACKERS] HBA files w/include support?

2014-02-14 Thread Bruce Momjian
On Thu, Feb 13, 2014 at 11:28:45PM -0600, Jerry Sievers wrote: > > One issue with this is that pg_hba.conf is order sensitive, which could > > become a trap for the unwary if includes are used carelessly. > > Indeed. > > The other thing that comes to mind, is that as opposed to > postgresql.conf

Re: [HACKERS] New hook after raw parsing, before analyze

2014-02-14 Thread David Beck
Let me rephrase this: Let’s remove my motivations and use cases from this conversation…. Why is that a bad idea of rewriting the query before it reaches transform/analyze (without ever accessing the catalog)? If that flexibility is acceptable to you, where would be the best place to put it in?

Re: [HACKERS] Memory ordering issue in LWLockRelease, WakeupWaiters, WALInsertSlotRelease

2014-02-14 Thread Florian Pflug
On Feb14, 2014, at 14:07 , Andres Freund wrote: > On 2014-02-14 13:52:45 +0100, Florian Pflug wrote: >>> I agree we should do that, but imo not in the backbranches. Anything >>> more than than the minimal fix in that code should be avoided in the >>> stable branches, this stuff is friggin performa

Re: [HACKERS] walsender doesn't send keepalives when writes are pending

2014-02-14 Thread Andres Freund
On 2014-02-14 13:58:59 +0100, Andres Freund wrote: > On 2014-02-14 12:55:06 +, Greg Stark wrote: > > On Fri, Feb 14, 2014 at 12:05 PM, Andres Freund > > wrote: > > > There's no reason not > > > to ask for a ping when we're writing. > > > Is there a reason to ask for a ping? The point of keep

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-14 Thread Heikki Linnakangas
On 02/14/2014 10:38 AM, Kyotaro HORIGUCHI wrote: Finally, the patch you will find attached is fixed only in styling mentioned above from your last patch. This patch applies current HEAD and I confirmed that it fixes this issue but I have not checked the lastSourceFailed section. Simple file remov

Re: [HACKERS] Memory ordering issue in LWLockRelease, WakeupWaiters, WALInsertSlotRelease

2014-02-14 Thread Andres Freund
On 2014-02-14 13:52:45 +0100, Florian Pflug wrote: > > I agree we should do that, but imo not in the backbranches. Anything > > more than than the minimal fix in that code should be avoided in the > > stable branches, this stuff is friggin performance sensitive, and the > > spinlock already is a *m

Re: [HACKERS] issue with gininsert under very high load

2014-02-14 Thread Alvaro Herrera
Andres Freund wrote: > On 2014-02-14 08:06:40 +0100, Jesper Krogh wrote: > > The build in mechanism, that cleanup is i cost paid by the process who > > happened to fill the pendinglist, is really hard to deal with in > > production. More control is appreciated, perhaps even an explicit > > flush-m

Re: [HACKERS] walsender doesn't send keepalives when writes are pending

2014-02-14 Thread Andres Freund
On 2014-02-14 12:55:06 +, Greg Stark wrote: > On Fri, Feb 14, 2014 at 12:05 PM, Andres Freund > wrote: > > There's no reason not > > to ask for a ping when we're writing. > Is there a reason to ask for a ping? The point of keepalives is to > ensure there's some traffic on idle connections so

Re: [HACKERS] HBA files w/include support?

2014-02-14 Thread Alvaro Herrera
Jerry Sievers wrote: > The other thing that comes to mind, is that as opposed to > postgresql.conf and the include scenario there... one can do show all or > query from pg_stat_activity just to see what setting they ended up > with. > > I'm not aware of any way to probe what hba rules are loaded

Re: [HACKERS] walsender doesn't send keepalives when writes are pending

2014-02-14 Thread Greg Stark
On Fri, Feb 14, 2014 at 12:05 PM, Andres Freund wrote: > There's no reason not > to ask for a ping when we're writing. Is there a reason to ask for a ping? The point of keepalives is to ensure there's some traffic on idle connections so that if the connection is dead it doesn't linger forever an

Re: [HACKERS] Memory ordering issue in LWLockRelease, WakeupWaiters, WALInsertSlotRelease

2014-02-14 Thread Florian Pflug
On Feb14, 2014, at 13:36 , Andres Freund wrote: > On 2014-02-14 13:28:47 +0100, Florian Pflug wrote: >>> I don't think that can actually happen because the head of the wait list >>> isn't the lock holder's lwWaitLink, but LWLock->head. I thought the same >>> for a while... >> >> Hm, true, but doe

Re: [HACKERS] issue with gininsert under very high load

2014-02-14 Thread Andres Freund
On 2014-02-14 08:06:40 +0100, Jesper Krogh wrote: > On 14/02/14 00:49, Tom Lane wrote: > >Andres Freund writes: > >>On 2014-02-13 16:15:42 -0500, Tom Lane wrote: > >>>Something like the attached? Can somebody who's seen this problem confirm > >>>this improves matters? > >>Hm. Won't that possiby l

Re: [HACKERS] Memory ordering issue in LWLockRelease, WakeupWaiters, WALInsertSlotRelease

2014-02-14 Thread Andres Freund
On 2014-02-14 13:28:47 +0100, Florian Pflug wrote: > > I don't think that can actually happen because the head of the wait list > > isn't the lock holder's lwWaitLink, but LWLock->head. I thought the same > > for a while... > > Hm, true, but does that protect us under all circumstances? If another

Re: [HACKERS] Memory ordering issue in LWLockRelease, WakeupWaiters, WALInsertSlotRelease

2014-02-14 Thread Florian Pflug
On Feb14, 2014, at 11:45 , Andres Freund wrote: > On 2014-02-13 15:34:09 +0100, Florian Pflug wrote: >> On Feb10, 2014, at 17:38 , Andres Freund wrote: >>> On 2014-02-10 11:11:28 -0500, Tom Lane wrote: Andres Freund writes: > So what we need to do is to acquire a write barrier between t

[HACKERS] walsender doesn't send keepalives when writes are pending

2014-02-14 Thread Andres Freund
Hi, In WalSndLoop() we do: wakeEvents = WL_LATCH_SET | WL_POSTMASTER_DEATH | WL_TIMEOUT | WL_SOCKET_READABLE; if (pq_is_send_pending()) wakeEvents |= WL_SOCKET_WRITEABLE; else if (wal_sender_timeout > 0 && !ping_sent) { ... if (GetCurrentTimestamp() >= tim

Re: [HACKERS] gaussian distribution pgbench

2014-02-14 Thread Mitsumasa KONDO
I add exponential distribution random generator (and little bit refactoring:) ). I use inverse transform method to create its distribution. It's very simple method that is created by - log (rand()). We can control slope of distribution using threshold parameter. It is same as gaussian threshold.

[HACKERS] walsender can ignore send failures in WalSndLoop

2014-02-14 Thread Andres Freund
Hi, There's a small issue in abfd192b, namely one of the error cases wasn't changed when WalSndLoop was changed to be able to return. I don't think this is likely to have any grave consequences, we'll likely error out soon afterwards again. Patch attached. Greetings, Andres Freund -- Andres

Re: [HACKERS] Memory ordering issue in LWLockRelease, WakeupWaiters, WALInsertSlotRelease

2014-02-14 Thread Andres Freund
On 2014-02-13 15:34:09 +0100, Florian Pflug wrote: > On Feb10, 2014, at 17:38 , Andres Freund wrote: > > On 2014-02-10 11:11:28 -0500, Tom Lane wrote: > >> Andres Freund writes: > >>> So what we need to do is to acquire a write barrier between the > >>> assignments to lwWaitLink and lwWaiting, i.

Re: [HACKERS] Changeset Extraction v7.6

2014-02-14 Thread Erik Rijkers
On Fri, February 14, 2014 10:13, Andres Freund wrote: > Hi, > > On 2014-02-14 09:23:45 +0100, Erik Rijkers wrote: >> >0001-wal_decoding-Introduce-logical-changeset-extraction.patch.gz 159 k >> >0002-wal_decoding-logical-changeset-extraction-walsender-.patch.gz 16 k >> >0003-wal_decoding-pg_recvl

Re: [HACKERS] New hook after raw parsing, before analyze

2014-02-14 Thread David Beck
Thanks for the reply. There are two things I think I’ve been misunderstood: 1, the point is to do the rewrite without and before catalog access 2, I do want to push the join to the source and equally important pushing the where conditions there Best regards, David 2014.02.13. dátummal, 21:22 i

Re: [HACKERS] Changeset Extraction v7.6

2014-02-14 Thread Andres Freund
Hi, On 2014-02-14 09:23:45 +0100, Erik Rijkers wrote: > >0001-wal_decoding-Introduce-logical-changeset-extraction.patch.gz159 k > >0002-wal_decoding-logical-changeset-extraction-walsender-.patch.gz 16 k > >0003-wal_decoding-pg_recvlogical-Introduce-pg_receivexlog.patch.gz 15 k > >0004-wal_

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-02-14 Thread Hiroshi Inoue
(2014/02/12 8:30), Tom Lane wrote: > I wrote: >> Hiroshi Inoue writes: >>> I tried MINGW port with the attached change and successfully built >>> src and contrib and all pararell regression tests were OK. > >> I cleaned this up a bit (the if-nesting in Makefile.shlib was making >> my head hurt, n

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-14 Thread Kyotaro HORIGUCHI
Hello, Before taking up the topic.. At Thu, 13 Feb 2014 19:45:38 +0200, Heikki Linnakangas wrote > On 02/13/2014 06:47 PM, Heikki Linnakangas wrote: > > On 02/13/2014 02:42 PM, Heikki Linnakangas wrote: > >> The behavior where we prefer a segment from archive with lower TLI > >> over > >> a file

Re: [HACKERS] HBA files w/include support?

2014-02-14 Thread Stephen Frost
Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Thu, Feb 13, 2014 at 08:24:27PM -0600, Jerry Sievers wrote: > > I'm aware of how a pg_hba.conf file can refer to other files for > > including @lists of users, etc. > > > > But there is currently no support for being able to pull in entire fil

Re: [HACKERS] Changeset Extraction v7.6

2014-02-14 Thread Erik Rijkers
On Thu, February 13, 2014 17:12, Andres Freund wrote: > Hi, > > On 2014-02-11 11:22:24 -0500, Robert Haas wrote: >> [loads of comments] > > I tried to address all the points you mentioned. > >0001-wal_decoding-Introduce-logical-changeset-extraction.patch.gz 159 k >0002-wal_decoding-logical-ch