[HACKERS] patch: Use pg_mbcliplen for truncation in text-to-name conversion

2012-05-24 Thread Karl Schnaitter
The text_name function was truncating its input string to the first NAMEDATALEN-1 bytes, which is wrong if the string has multi-byte characters. This patch changes it to use pg_mbcliplen, following the namein function. diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c

Re: [HACKERS] plperl_helpers.h fix for clang

2012-05-24 Thread Alex Hunsaker
On Thu, May 24, 2012 at 12:03 PM, Peter Eisentraut wrote: > On tor, 2012-05-24 at 11:36 -0600, Alex Hunsaker wrote: >> Doh, it is indeed there in 5.16.0, looks like it got added in 5.10 >> :-(. (I was on the wrong branch...). > > It's in ppport.h. Don't see any reason not to then. +1 for the sim

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-24 Thread Sergey Koposov
On Thu, 24 May 2012, Jeff Janes wrote: Add #define LWLOCK_STATS near the top of: src/backend/storage/lmgr/lwlock.c and recompile and run a reduced-size workload. When the processes exits, they will dump a lot of data about LWLock usage to the logfile. Generally the LWLock with the most blocks o

Re: [HACKERS] Draft release notes complete

2012-05-24 Thread Bruce Momjian
On Thu, May 24, 2012 at 11:16:28PM +0100, Peter Geoghegan wrote: > On 24 May 2012 22:57, Bruce Momjian wrote: > > OK, item moved down.  We have not have "bug fix" designation.  You have > > a suggestion? > > I assumed you were going to put it beside the other compatibility note > relating to pg_s

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-24 Thread Jeff Janes
On Thu, May 24, 2012 at 3:36 PM, Sergey Koposov wrote: > Hi, > > > On Thu, 24 May 2012, Robert Haas wrote: >> >> Not sure.  It might be some other LWLock, but it's hard to tell which >> one from the information provided. > > > If you could tell what's the best way to find out the info that you nee

Re: [HACKERS] [RFC] Interface of Row Level Security

2012-05-24 Thread Florian Pflug
On May24, 2012, at 19:25 , Robert Haas wrote: > FWIW, I'm inclined to think that you should NOT be able to create a > row that references an invisible row. You might end up with that > situation anyway, because we don't know what the semantics of the > security policy are: rows might become visibl

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-24 Thread Ants Aasma
On Tue, May 22, 2012 at 11:36 PM, Ants Aasma wrote: > ... The free list > itself is a bit trickier, but if it's still necessary/useful then > SC->firstFreeBuffer and buf->freeNext are in effect a linked-list > stack, there should plenty of tested lock free algorithms floating > around for that. (b

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-24 Thread Sergey Koposov
Hi, On Thu, 24 May 2012, Robert Haas wrote: Not sure. It might be some other LWLock, but it's hard to tell which one from the information provided. If you could tell what's the best way to find out the info that you need, then I could run it reasonably quickly. S *

Re: [HACKERS] Backends stalled in 'startup' state: index corruption

2012-05-24 Thread Tom Lane
Greg Sabino Mullane writes: > Oh, almost forgot: reading your reply to the old thread reminded me of > something I saw in one of the straces right as it "woke up" and left > the startup state to do some work. Here's a summary: > 12:18:39 semop(4390981, 0x7fff66c4ec10, 1) = 0 > 12:18:39 semop(43

Re: [HACKERS] Per-Database Roles

2012-05-24 Thread Bruce Momjian
On Tue, May 22, 2012 at 10:19:12AM -0400, Robert Haas wrote: > In retrospect, I think the idea of shared catalogs was probably a bad > idea. I think we should have made roles and tablespaces database > objects rather than shared objects, and come up with some ad-hoc > method of representing the se

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-24 Thread Tom Lane
Robert Haas writes: > On Thu, May 24, 2012 at 4:46 PM, Merlin Moncure wrote: >> We don't get to skip wal of course, but we should be able to use a >> bulk insert strategy, especially if there was some way of predicting >> that a large number of tuples were going to be inserted.  I'm >> wondering

Re: [HACKERS] Draft release notes complete

2012-05-24 Thread Peter Geoghegan
On 24 May 2012 22:57, Bruce Momjian wrote: > OK, item moved down.  We have not have "bug fix" designation.  You have > a suggestion? I assumed you were going to put it beside the other compatibility note relating to pg_stat_statements, "Change pg_stat_statements' total_time column to be measured

Re: [HACKERS] heap metapages

2012-05-24 Thread Bruce Momjian
On Tue, May 22, 2012 at 09:52:30AM +0100, Simon Riggs wrote: > Having pg_upgrade touch data files is both dangerous and difficult to > back out in case of mistake, so I am wary of putting the metapage at > block 0. Doing it the way I suggest means the .meta files would be > wholly new and can be de

Re: [HACKERS] Draft release notes complete

2012-05-24 Thread Bruce Momjian
On Thu, May 24, 2012 at 10:34:22PM +0100, Peter Geoghegan wrote: > In passing, I noticed this: > > """ > E.1.3.12.2. pg_stat_statements > > Improve pg_stat_statements to aggregate similar queries (Peter > Geoghegan, Tom Lane) > > Improve pg_stat_statements' handling of PREPARE/EXECUTE statements

Re: [HACKERS] release note item

2012-05-24 Thread Bruce Momjian
On Mon, May 21, 2012 at 12:19:27PM -0400, Tom Lane wrote: > Andrew Dunstan writes: > > Regarding the item: > > * Properly handle empty arrays returned by PL/Perl functions (Andrew > > Dunstan) DETAILS? > > > This was a bug fix, not a feature, and in any case is due to Alex > > Hunsaker, no

Re: [HACKERS] Draft release notes complete

2012-05-24 Thread Peter Geoghegan
On 21 May 2012 19:10, Josh Berkus wrote: > >> For these reasons, it may be timely and appropriate, from a purely >> advocacy point-of-view, to call our new group commit "group commit" in >> release notes and documentation, and announce it as a new feature. > > First, shouldn't we be having this di

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-24 Thread Robert Haas
On Thu, May 24, 2012 at 4:46 PM, Merlin Moncure wrote: > Wait -- OP's gripe this isn't regarding standard pgbench, but multiple > large concurrent 'insert into foo select...'.  I looked in the code > and it appears that the only bulk insert strategy using operations are > copy, create table as sel

Re: [HACKERS] Backends stalled in 'startup' state: index corruption

2012-05-24 Thread Greg Sabino Mullane
> I think there are probably two independent issues here. The missing > index entries are clearly bad but it's not clear that they had anything > to do with the startup stall. On further log digging, I think you are correct, as those index warnings go back many days before the startup problems a

Re: [HACKERS] Backends stalled in 'startup' state: index corruption

2012-05-24 Thread Greg Sabino Mullane
On Thu, May 24, 2012 at 03:54:54PM -0400, Tom Lane wrote: > Did you check I/O activity? I looked again at Jeff Frost's report and > now think that what he saw was probably a lot of seqscans on bloated > system catalogs, cf > http://archives.postgresql.org/message-id/28484.1337887...@sss.pgh.pa.us

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-24 Thread Merlin Moncure
On Thu, May 24, 2012 at 3:35 PM, Robert Haas wrote: > On Thu, May 24, 2012 at 3:46 PM, Merlin Moncure wrote: >> hm, looking at the code some more, it looks like the whole point of >> the strategy system is to do that.  ISTM bulk insert type queries >> would be good candidates for a buffer strateg

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-24 Thread Robert Haas
On Thu, May 24, 2012 at 3:46 PM, Merlin Moncure wrote: > On Thu, May 24, 2012 at 2:24 PM, Merlin Moncure wrote: >>> As you can see, raw performance isn't much worse with the larger data >>> sets, but scalability at high connection counts is severely degraded >>> once the working set no longer fit

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-24 Thread Jeff Janes
On Thu, May 24, 2012 at 12:46 PM, Merlin Moncure wrote: > On Thu, May 24, 2012 at 2:24 PM, Merlin Moncure wrote: >>> As you can see, raw performance isn't much worse with the larger data >>> sets, but scalability at high connection counts is severely degraded >>> once the working set no longer fi

Re: [HACKERS] Re: [COMMITTERS] pgsql: Send new protocol keepalive messages to standby servers.

2012-05-24 Thread Robert Haas
On Thu, May 24, 2012 at 2:52 PM, Tom Lane wrote: > Robert Haas writes: >> On Wed, May 23, 2012 at 2:28 PM, Fujii Masao wrote: >>> Is there plan to implement such external functions before 9.2 release? >>> If not, keepalive protocol seems to be almost useless because there is >>> no use of it for

Re: [HACKERS] Missing optimization when filters are applied after window functions

2012-05-24 Thread Bruce Momjian
On Wed, May 16, 2012 at 09:25:13AM -0400, Tom Lane wrote: > Volker Grabsch writes: > > I propose the following general optimization: If all window > > functions are partitioned by the same first field (here: id), > > then any filter on that field should be executed before > > WindowAgg. > > I'm n

Re: [HACKERS] Backends stalled in 'startup' state: index corruption

2012-05-24 Thread Tom Lane
Greg Sabino Mullane writes: > Yesterday I had a client that experienced a sudden high load on > one of their servers (8.3.5 - yes, I know. Those of you with > clients will understand). When I checked, almost all connections > were in a "startup" state, very similar to this thread: > http://pos

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-24 Thread Sergey Koposov
On Thu, 24 May 2012, Robert Haas wrote: As you can see, raw performance isn't much worse with the larger data sets, but scalability at high connection counts is severely degraded once the working set no longer fits in shared_buffers. Actually the problem persits even when I trim the dataset si

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-24 Thread Merlin Moncure
On Thu, May 24, 2012 at 2:24 PM, Merlin Moncure wrote: >> As you can see, raw performance isn't much worse with the larger data >> sets, but scalability at high connection counts is severely degraded >> once the working set no longer fits in shared_buffers. > > Hm, wouldn't the BufFreelistLock iss

Re: [HACKERS] Archiver not exiting upon crash

2012-05-24 Thread Tom Lane
Jeff Janes writes: > On Wed, May 23, 2012 at 2:21 PM, Tom Lane wrote: >> However, I remain unsatisfied with this idea as an explanation for the >> behavior you're seeing. In the first place, that race condition window >> ought not be wide enough to allow failure probabilities as high as 10%. >>

Re: [HACKERS] Archiver not exiting upon crash

2012-05-24 Thread Jeff Janes
On Wed, May 23, 2012 at 2:21 PM, Tom Lane wrote: > I wrote: >> Jeff Janes writes: >>> But what happens if the SIGQUIT is blocked before the system(3) is >>> invoked?  Does the ignore take precedence over the block, or does the >>> block take precedence over the ignore, and so the signal is still

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-24 Thread Merlin Moncure
On Thu, May 24, 2012 at 1:43 PM, Robert Haas wrote: > On Thu, May 24, 2012 at 2:19 PM, Sergey Koposov wrote: >> On Thu, 24 May 2012, Robert Haas wrote: >>> On Thu, May 24, 2012 at 1:42 PM, Sergey Koposov >>> wrote: I guess there is nothing catastrophically wrong with that, but still I'

Re: [HACKERS] "could not open relation with OID" errors after promoting the standby to master

2012-05-24 Thread Andres Freund
On Thursday, May 24, 2012 08:46:21 PM Tom Lane wrote: > Andres Freund writes: > > On Thursday, May 24, 2012 08:32:47 PM Tom Lane wrote: > >> I'm not sure I believe that we can make a recovery resume from an > >> arbitrary point in WAL anyway, or that it would be worth the trouble. > >> Can't we ju

Re: [HACKERS] Re: [COMMITTERS] pgsql: Send new protocol keepalive messages to standby servers.

2012-05-24 Thread Tom Lane
Robert Haas writes: > On Wed, May 23, 2012 at 2:28 PM, Fujii Masao wrote: >> Is there plan to implement such external functions before 9.2 release? >> If not, keepalive protocol seems to be almost useless because there is >> no use of it for a user and the increase in the number of packets might

Re: [HACKERS] "could not open relation with OID" errors after promoting the standby to master

2012-05-24 Thread Tom Lane
Andres Freund writes: > On Thursday, May 24, 2012 08:32:47 PM Tom Lane wrote: >> I'm not sure I believe that we can make a recovery resume from an >> arbitrary point in WAL anyway, or that it would be worth the trouble. >> Can't we just resume from the last restartpoint? > Well, with a decent siz

Re: [HACKERS] Re: [COMMITTERS] pgsql: Send new protocol keepalive messages to standby servers.

2012-05-24 Thread Robert Haas
On Wed, May 23, 2012 at 2:28 PM, Fujii Masao wrote: > On Sat, Dec 31, 2011 at 10:34 PM, Simon Riggs wrote: >> Send new protocol keepalive messages to standby servers. >> Allows streaming replication users to calculate transfer latency >> and apply delay via internal functions. No external functio

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-24 Thread Robert Haas
On Thu, May 24, 2012 at 2:19 PM, Sergey Koposov wrote: > On Thu, 24 May 2012, Robert Haas wrote: >> On Thu, May 24, 2012 at 1:42 PM, Sergey Koposov >> wrote: >>> >>> I guess there is nothing catastrophically wrong with that, but still I'm >>> very suprised that you get severe locking problems (fa

Re: [HACKERS] "could not open relation with OID" errors after promoting the standby to master

2012-05-24 Thread Andres Freund
On Thursday, May 24, 2012 08:32:47 PM Tom Lane wrote: > Andres Freund writes: > > The control file currently is not a very good match because of the > > current requirement of staying below 512 bytes. If we would include the > > list of running xacts that wouldn't be enough. > > I wondered before

Re: [HACKERS] "could not open relation with OID" errors after promoting the standby to master

2012-05-24 Thread Tom Lane
Andres Freund writes: > The control file currently is not a very good match because of the current > requirement of staying below 512 bytes. If we would include the list of > running xacts that wouldn't be enough. > I wondered before if there is more to do to fix that then to do the atomic > wr

Re: [HACKERS] Stateful pointers in set-returning functions

2012-05-24 Thread Ian Pye
Fair enough -- thanks for the tip. On Thu, May 24, 2012 at 11:21 AM, Tom Lane wrote: > Ian Pye writes: >> I'm writing a set-returning function which places a file handle into >> PG's FuncCallContext's user_fctx space. My problem is that when the >> function is ran with a limit clause (SELECT * F

Re: [HACKERS] Changing the concept of a DATABASE

2012-05-24 Thread Andres Freund
On Thursday, May 24, 2012 08:12:56 PM Josh Berkus wrote: > > Yes, we do. It would be best to conclude that things I do on hackers > > relate in some way to those goals, even if it isn't immediately clear > > how. > See, now you've got me all curious. How does inter-DB queries relate to > the New R

Re: [HACKERS] Stateful pointers in set-returning functions

2012-05-24 Thread Tom Lane
Ian Pye writes: > I'm writing a set-returning function which places a file handle into > PG's FuncCallContext's user_fctx space. My problem is that when the > function is ran with a limit clause (SELECT * FROM foo() LIMIT 10) the > server will stop calling the function automatically, not giving me

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-24 Thread Sergey Koposov
On Thu, 24 May 2012, Robert Haas wrote: On Thu, May 24, 2012 at 1:42 PM, Sergey Koposov wrote: I guess there is nothing catastrophically wrong with that, but still I'm very suprised that you get severe locking problems (factor of two slow-down) when running parallel read-only transactions. M

[HACKERS] Stateful pointers in set-returning functions

2012-05-24 Thread Ian Pye
Hi, I'm writing a set-returning function which places a file handle into PG's FuncCallContext's user_fctx space. My problem is that when the function is ran with a limit clause (SELECT * FROM foo() LIMIT 10) the server will stop calling the function automatically, not giving me a chance to close t

Re: [HACKERS] Changing the concept of a DATABASE

2012-05-24 Thread Josh Berkus
> Yes, we do. It would be best to conclude that things I do on hackers > relate in some way to those goals, even if it isn't immediately clear > how. See, now you've got me all curious. How does inter-DB queries relate to the New Replication? -- Josh Berkus PostgreSQL Experts Inc. http://pgexp

Re: [HACKERS] Creating multiple indexes in one table scan.

2012-05-24 Thread Andrew Dunstan
On 05/24/2012 11:44 AM, Robert Haas wrote: On Thu, May 24, 2012 at 11:25 AM, Gurjeet Singh wrote: It'd be great if one of standard utilities like pg_restore supported this, by spawning every concurrent index build in separate backends. Just a thought. If parallel restore doesn't already take

Re: [HACKERS] plperl_helpers.h fix for clang

2012-05-24 Thread Peter Eisentraut
On tor, 2012-05-24 at 11:36 -0600, Alex Hunsaker wrote: > On Thu, May 24, 2012 at 11:31 AM, Alex Hunsaker wrote: > > On Thu, May 24, 2012 at 11:19 AM, Peter Eisentraut wrote: > >> > >> And we could use SvREFCNT_inc_simple_void(sv), since sv doesn't have any > >> side effects, but that's optional.

Re: [HACKERS] shared_preload_libraries path

2012-05-24 Thread Magnus Hagander
On Thu, May 24, 2012 at 7:58 PM, Peter Eisentraut wrote: > On tor, 2012-05-24 at 14:09 +0200, Magnus Hagander wrote: >> Would i make sense to have a postgresql.conf parameter that would add >> to LD_LIBRARY_PATH when loading libraries from >> shared_preload_libraries (and other library loads). > >

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-24 Thread Robert Haas
On Thu, May 24, 2012 at 1:42 PM, Sergey Koposov wrote: > I guess there is nothing catastrophically wrong with that, but still I'm > very suprised that you get severe locking problems (factor of two slow-down) > when running parallel read-only transactions. Me, too. How many concurrent connection

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-24 Thread Jeff Janes
On Thu, May 24, 2012 at 6:24 AM, Sergey Koposov wrote: > Hi, > > I've been running some tests on pg 9.2beta1 and in particular a set > of queries like ... > > And I noticed than when I run the query like the one shown above in parallel > (in multiple connections for ZZZ=0...8) the performance of e

Re: [HACKERS] shared_preload_libraries path

2012-05-24 Thread Peter Eisentraut
On tor, 2012-05-24 at 14:09 +0200, Magnus Hagander wrote: > Would i make sense to have a postgresql.conf parameter that would add > to LD_LIBRARY_PATH when loading libraries from > shared_preload_libraries (and other library loads). Well, you could write a library that sets it in its init function

Re: [HACKERS] [9.2] crash on regex

2012-05-24 Thread Tom Lane
Thom Brown writes: > Hmmm... curiously, lazy (non-greedy) quantifiers are stable, such as: ((a))*? I've found it. The triggering conditions are (1) more than one set of capturing parens in a substring() pattern, and (2) at least one trial midpoint failing in ccondissect() or one of its siblings.

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-24 Thread Sergey Koposov
On Thu, 24 May 2012, Merlin Moncure wrote: Are you sure? I looked at all the ReleasePredicateLocks calls and they appear to be guarded by: /* Nothing to do if this is not a serializable transaction */ if (MySerializableXact == InvalidSerializableXact) return false;

Re: [HACKERS] plperl_helpers.h fix for clang

2012-05-24 Thread Alex Hunsaker
On Thu, May 24, 2012 at 11:31 AM, Alex Hunsaker wrote: > On Thu, May 24, 2012 at 11:19 AM, Peter Eisentraut wrote: >> >> And we could use SvREFCNT_inc_simple_void(sv), since sv doesn't have any >> side effects, but that's optional. > Hrm I can't seem to find either of those functions in my copy

[HACKERS] Backends stalled in 'startup' state: index corruption

2012-05-24 Thread Greg Sabino Mullane
Yesterday I had a client that experienced a sudden high load on one of their servers (8.3.5 - yes, I know. Those of you with clients will understand). When I checked, almost all connections were in a "startup" state, very similar to this thread: http://postgresql.1045698.n5.nabble.com/9-1-3-bac

Re: [HACKERS] [RFC] Interface of Row Level Security

2012-05-24 Thread Florian Pflug
On May24, 2012, at 18:42 , Kohei KaiGai wrote: > I'd like to summarize the current design being discussed. > > syntax: > ALTER TABLE WITH ROW LEVEL SECURITY > ( ) [FOR (SELECT | UPDATE | DELETE)]; > ALTER TABLE WITHOUT ROW LEVEL SECURITY; > > I tried to patch the parser/gram.y, but here

Re: [HACKERS] plperl_helpers.h fix for clang

2012-05-24 Thread Alex Hunsaker
On Thu, May 24, 2012 at 11:19 AM, Peter Eisentraut wrote: > clang warns about that newish SvREFCNT_inc(sv) call in plperl_helpers.h > about an unused return value, because the macro expansion of > SvREFCNT_inc(sv) returns sv.  The merit of that warning might be > debatable, but it seems easy to fi

Re: [HACKERS] [RFC] Interface of Row Level Security

2012-05-24 Thread Robert Haas
On Thu, May 24, 2012 at 12:00 PM, Kohei KaiGai wrote: >> Another issue, BTW, are FOREIGN KEY constraints. Should you be allowed to >> created references to rows which are invisible to you, or should FOREIGN KEY >> constraints be exempt from security policies? I'd say they shouldn't be, i.e. >> the

[HACKERS] plperl_helpers.h fix for clang

2012-05-24 Thread Peter Eisentraut
clang warns about that newish SvREFCNT_inc(sv) call in plperl_helpers.h about an unused return value, because the macro expansion of SvREFCNT_inc(sv) returns sv. The merit of that warning might be debatable, but it seems easy to fix by using SvREFCNT_inc_void(sv) instead. And we could use SvREFCN

Re: [HACKERS] [RFC] Interface of Row Level Security

2012-05-24 Thread Robert Haas
On Thu, May 24, 2012 at 6:20 AM, Florian Pflug wrote: > But the security policy should still apply to the old rows, i.e. > you shouldn't be after to UPDATE or DELETE rows you cannot see, no? Agreed. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company --

Re: [HACKERS] [RFC] Interface of Row Level Security

2012-05-24 Thread Robert Haas
On Thu, May 24, 2012 at 6:11 AM, Kohei KaiGai wrote: >> Perhaps when we see that RLS >> applies, we should replace the reference to the original table with a >> subquery RTE that has the security_barrier flag set - essentially >> treating a table with RLS as if it were a security view. >> > I beco

Re: [HACKERS] pg_basebackup -x stream from the standby gets stuck

2012-05-24 Thread Fujii Masao
On Wed, May 23, 2012 at 9:25 PM, Magnus Hagander wrote: > While reviewing and cleaning this patch up a bit I noticed it actually > broke pg_receivexlog in the renaming. > > Here is a new version of the patch, reworked based on the above so > we're down to a single callback. I moved the "rename las

Re: [HACKERS] "could not open relation with OID" errors after promoting the standby to master

2012-05-24 Thread Andres Freund
On Thursday, May 24, 2012 06:35:06 PM Heikki Linnakangas wrote: > On 24.05.2012 18:16, Robert Haas wrote: > > On Thu, May 24, 2012 at 10:13 AM, Joachim Wieland wrote: > >> I wouldn't have assumed any corruption was possible given that I did > >> clean shutdowns on both sides... > > > > The thing

Re: [HACKERS] [RFC] Interface of Row Level Security

2012-05-24 Thread Kohei KaiGai
I'd like to summarize the current design being discussed. syntax: ALTER TABLE WITH ROW LEVEL SECURITY ( ) [FOR (SELECT | UPDATE | DELETE)]; ALTER TABLE WITHOUT ROW LEVEL SECURITY; I tried to patch the parser/gram.y, but here was syntax conflicts on ADD / DROP sub-command. And, I noti

Re: [HACKERS] "could not open relation with OID" errors after promoting the standby to master

2012-05-24 Thread Heikki Linnakangas
On 24.05.2012 18:16, Robert Haas wrote: On Thu, May 24, 2012 at 10:13 AM, Joachim Wieland wrote: I wouldn't have assumed any corruption was possible given that I did clean shutdowns on both sides... The thing that's worrying me is that there's not really any such thing as a "clean" shutdown o

Re: [HACKERS] [9.2] crash on regex

2012-05-24 Thread Tom Lane
Robert Haas writes: > On Thu, May 24, 2012 at 10:16 AM, Marko Kreen wrote: >> Following query crashes backend on 9.2: >> >>  select substring('asd TO foo' from ' TO (([a-z0-9._]+|"([^"]+|"")+")+)'); > I spent some time trying to reduce this to the simplest case that > still causes a crash, and

Re: [HACKERS] [RFC] Interface of Row Level Security

2012-05-24 Thread Kohei KaiGai
2012/5/24 Florian Pflug : > On May24, 2012, at 16:19 , Kohei KaiGai wrote: >> So, the proposed interface might be revised as follows: >>  ALTER TABLE ADD SECURITY POLICY >>      (, ...) [FOR SELECT | >>                                                         INSERT | >>                            

Re: [HACKERS] Creating multiple indexes in one table scan.

2012-05-24 Thread Robert Haas
On Thu, May 24, 2012 at 11:25 AM, Gurjeet Singh wrote: > It'd be great if one of standard utilities like pg_restore supported this, > by spawning every concurrent index build in separate backends. Just a > thought. If parallel restore doesn't already take this into account when doing job scheduli

Re: [HACKERS] pg_stat_statments queryid

2012-05-24 Thread Peter Geoghegan
On 24 May 2012 16:06, Tom Lane wrote: >>> What I'd like to be able to do is aggregate this information over time >>> and/or across standbys in a cluster, as queries are evicted and >>> subsequently re-entered into pg_stat_statement's shared hash table. > > It appears to me that the above ... > >>>

Re: [HACKERS] [9.2] crash on regex

2012-05-24 Thread Thom Brown
On 24 May 2012 16:24, Thom Brown wrote: > On 24 May 2012 16:08, Robert Haas wrote: >> On Thu, May 24, 2012 at 10:16 AM, Marko Kreen wrote: >>> Following query crashes backend on 9.2: >>> >>>  select substring('asd TO foo' from ' TO (([a-z0-9._]+|"([^"]+|"")+")+)'); >> >> I spent some time trying

Re: [HACKERS] Creating multiple indexes in one table scan.

2012-05-24 Thread Gurjeet Singh
On Thu, May 24, 2012 at 11:22 AM, Stephen Frost wrote: > * Gurjeet Singh (singh.gurj...@gmail.com) wrote: > > Bruce points out the even simpler case is to build several indexes in > > parallel over the same scan. > > > > I thought I had posted a patch to that effect long back, but upon > sear

Re: [HACKERS] [9.2] crash on regex

2012-05-24 Thread Thom Brown
On 24 May 2012 16:08, Robert Haas wrote: > On Thu, May 24, 2012 at 10:16 AM, Marko Kreen wrote: >> Following query crashes backend on 9.2: >> >>  select substring('asd TO foo' from ' TO (([a-z0-9._]+|"([^"]+|"")+")+)'); > > I spent some time trying to reduce this to the simplest case that > still

Re: [HACKERS] Creating multiple indexes in one table scan.

2012-05-24 Thread Stephen Frost
* Gurjeet Singh (singh.gurj...@gmail.com) wrote: > Bruce points out the even simpler case is to build several indexes in > parallel over the same scan. > > I thought I had posted a patch to that effect long back, but upon searching > my emails apparently I forgot about the patch. > > Attached

Re: [HACKERS] pg_stat_statments queryid

2012-05-24 Thread Peter Geoghegan
On 24 May 2012 16:06, Tom Lane wrote: > I do not want to promise that it's stable over any timeframe longer than > a server reboot. You already have though, since pg_stat_statements persistently stores statistics to disk by default, and can only ever recognise statement equivalence based on the (

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-24 Thread Merlin Moncure
On Thu, May 24, 2012 at 8:24 AM, Sergey Koposov wrote: > Hi, > > I've been running some tests on pg 9.2beta1 and in particular a set > of queries like > > create table _tmp0 as select * from ( >        select *, (select healpixid from idt_match as m where >                                        m

Re: [HACKERS] "could not open relation with OID" errors after promoting the standby to master

2012-05-24 Thread Robert Haas
On Thu, May 24, 2012 at 10:13 AM, Joachim Wieland wrote: > On Tue, May 22, 2012 at 9:50 AM, Robert Haas wrote: >> Hmm.  I think that if you do it this way, the minimum recovery point >> won't be respected, which could leave you with a corrupted database. >> Now, if all the WAL files that you need

Re: [HACKERS] [9.2] crash on regex

2012-05-24 Thread Robert Haas
On Thu, May 24, 2012 at 10:16 AM, Marko Kreen wrote: > Following query crashes backend on 9.2: > >  select substring('asd TO foo' from ' TO (([a-z0-9._]+|"([^"]+|"")+")+)'); I spent some time trying to reduce this to the simplest case that still causes a crash, and came up with this: select subs

Re: [HACKERS] pg_stat_statments queryid

2012-05-24 Thread Tom Lane
Robert Haas writes: > On Thu, May 24, 2012 at 10:26 AM, Peter Geoghegan > wrote: > But I think this explanation is enough to convince me that it might be > worthwhile: >> What I'd like to be able to do is aggregate this information over time >> and/or across standbys in a cluster, as queries ar

Re: [HACKERS] [RFC] Interface of Row Level Security

2012-05-24 Thread Florian Pflug
On May24, 2012, at 16:19 , Kohei KaiGai wrote: > So, the proposed interface might be revised as follows: > ALTER TABLE ADD SECURITY POLICY > (, ...) [FOR SELECT | > INSERT | > [BEF

Re: [HACKERS] pg_stat_statments queryid

2012-05-24 Thread Robert Haas
On Thu, May 24, 2012 at 10:26 AM, Peter Geoghegan wrote: > On 24 May 2012 11:50, Magnus Hagander wrote: >> Was there any actual reason why we didn't end up exposing queryid in >> the pg_stat_statements view? >> >> It would be highly useful when tracking changes over time. Right now I >> see peopl

Re: [HACKERS] pg_stat_statments queryid

2012-05-24 Thread Magnus Hagander
On Thu, May 24, 2012 at 4:26 PM, Peter Geoghegan wrote: > On 24 May 2012 11:50, Magnus Hagander wrote: >> Was there any actual reason why we didn't end up exposing queryid in >> the pg_stat_statements view? >> >> It would be highly useful when tracking changes over time. Right now I >> see people

Re: [HACKERS] Interrupting long external library calls

2012-05-24 Thread Florian Pflug
On May24, 2012, at 15:04 , Sandro Santilli wrote: > On Wed, May 16, 2012 at 07:30:03PM +0300, Heikki Linnakangas wrote: >> On 16.05.2012 15:42, Sandro Santilli wrote: >>> But CHECK_FOR_INTERRUPTS doesn't return, right ? >>> Is there another macro for just checking w/out yet acting upon it ? >> >>

Re: [HACKERS] shared_preload_libraries path

2012-05-24 Thread Tom Lane
Peter Geoghegan writes: > On 24 May 2012 13:09, Magnus Hagander wrote: >> Would i make sense to have a postgresql.conf parameter that would add >> to LD_LIBRARY_PATH when loading libraries from >> shared_preload_libraries (and other library loads). To make it >> possible to configure it without h

Re: [HACKERS] pg_stat_statments queryid

2012-05-24 Thread Peter Geoghegan
On 24 May 2012 11:50, Magnus Hagander wrote: > Was there any actual reason why we didn't end up exposing queryid in > the pg_stat_statements view? > > It would be highly useful when tracking changes over time. Right now I > see people doing md5(query) to do that, which is a lot more ugly (and > ob

Re: [HACKERS] [RFC] Interface of Row Level Security

2012-05-24 Thread Kohei KaiGai
2012/5/24 Florian Pflug : > On May24, 2012, at 12:43 , Kohei KaiGai wrote: >> The case of INSERT / DELETE are simple; All we need to apply is >> checks on either new or old tuples. >> >> In case of UPDATE, we need to check on the old tuple whether use can >> see, and on the new tuple whether use ca

[HACKERS] [9.2] crash on regex

2012-05-24 Thread Marko Kreen
Following query crashes backend on 9.2: select substring('asd TO foo' from ' TO (([a-z0-9._]+|"([^"]+|"")+")+)'); It is supposed to load potentially quoted table name from CREATE RULE definition. Works fine on 8.3 .. 9.1 Backtrace: Program terminated with signal 11, Segmentation fault. #0 Ex

Re: [HACKERS] Interrupting long external library calls

2012-05-24 Thread Sandro Santilli
On Thu, May 24, 2012 at 04:55:34PM +0300, Heikki Linnakangas wrote: > On 24.05.2012 16:04, Sandro Santilli wrote: > >On Wed, May 16, 2012 at 07:30:03PM +0300, Heikki Linnakangas wrote: > >>On 16.05.2012 15:42, Sandro Santilli wrote: > >>>But CHECK_FOR_INTERRUPTS doesn't return, right ? > >>>Is ther

Re: [HACKERS] "could not open relation with OID" errors after promoting the standby to master

2012-05-24 Thread Joachim Wieland
On Tue, May 22, 2012 at 9:50 AM, Robert Haas wrote: > Hmm.  I think that if you do it this way, the minimum recovery point > won't be respected, which could leave you with a corrupted database. > Now, if all the WAL files that you need are present in pg_xlog anyway, > then they ought to get replay

Re: [HACKERS] Interrupting long external library calls

2012-05-24 Thread Heikki Linnakangas
On 24.05.2012 16:04, Sandro Santilli wrote: On Wed, May 16, 2012 at 07:30:03PM +0300, Heikki Linnakangas wrote: On 16.05.2012 15:42, Sandro Santilli wrote: But CHECK_FOR_INTERRUPTS doesn't return, right ? Is there another macro for just checking w/out yet acting upon it ? Hmm, no. CHECK_FOR_I

[HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-24 Thread Sergey Koposov
Hi, I've been running some tests on pg 9.2beta1 and in particular a set of queries like create table _tmp0 as select * from ( select *, (select healpixid from idt_match as m where m.transitid=o.transitid) as x from idt_photoobservat

Re: [HACKERS] Interrupting long external library calls

2012-05-24 Thread Sandro Santilli
On Wed, May 16, 2012 at 07:30:03PM +0300, Heikki Linnakangas wrote: > On 16.05.2012 15:42, Sandro Santilli wrote: > >But CHECK_FOR_INTERRUPTS doesn't return, right ? > >Is there another macro for just checking w/out yet acting upon it ? > > Hmm, no. CHECK_FOR_INTERRUPTS() checks the InterruptPendi

Re: [HACKERS] pg_receivexlog stops upon server restart

2012-05-24 Thread Thom Brown
On 24 May 2012 13:37, Magnus Hagander wrote: > On Thu, May 24, 2012 at 2:34 PM, Thom Brown wrote: >> On 24 May 2012 13:05, Magnus Hagander wrote: >>> On Thu, Apr 19, 2012 at 1:00 PM, Thom Brown wrote: On 10 April 2012 21:07, Magnus Hagander wrote: > On Friday, April 6, 2012, Thom Brow

Re: [HACKERS] pg_receivexlog stops upon server restart

2012-05-24 Thread Magnus Hagander
On Thu, May 24, 2012 at 2:34 PM, Thom Brown wrote: > On 24 May 2012 13:05, Magnus Hagander wrote: >> On Thu, Apr 19, 2012 at 1:00 PM, Thom Brown wrote: >>> On 10 April 2012 21:07, Magnus Hagander wrote: On Friday, April 6, 2012, Thom Brown wrote: > > Hi, > > I've tried out

Re: [HACKERS] pg_receivexlog stops upon server restart

2012-05-24 Thread Thom Brown
On 24 May 2012 13:05, Magnus Hagander wrote: > On Thu, Apr 19, 2012 at 1:00 PM, Thom Brown wrote: >> On 10 April 2012 21:07, Magnus Hagander wrote: >>> On Friday, April 6, 2012, Thom Brown wrote: Hi, I've tried out pg_receivexlog and have noticed that when restarting the

Re: [HACKERS] shared_preload_libraries path

2012-05-24 Thread Peter Geoghegan
On 24 May 2012 13:22, Peter Geoghegan wrote: > I found this blog post to be insightful: > > https://blogs.oracle.com/rie/entry/tt_ld_library_path_tt This one might be more useful, and itself refers to the aforementioned, earlier post: https://blogs.oracle.com/ali/entry/avoiding_ld_library_path_t

Re: [HACKERS] shared_preload_libraries path

2012-05-24 Thread Peter Geoghegan
On 24 May 2012 13:09, Magnus Hagander wrote: > Would i make sense to have a postgresql.conf parameter that would add > to LD_LIBRARY_PATH when loading libraries from > shared_preload_libraries (and other library loads). To make it > possible to configure it without having to  mess around with the

Re: [HACKERS] pg_stat_statements temporary file

2012-05-24 Thread Magnus Hagander
On Thu, May 24, 2012 at 2:16 PM, Peter Geoghegan wrote: > On 24 May 2012 12:42, Magnus Hagander wrote: >> What actually happens if it tries to repalloc() something huge? palloc >> will throw an elog(ERROR), and since this happens during postmaster >> startup, are you sure it won't prevent the ser

Re: [HACKERS] pg_stat_statements temporary file

2012-05-24 Thread Peter Geoghegan
On 24 May 2012 12:42, Magnus Hagander wrote: > What actually happens if it tries to repalloc() something huge? palloc > will throw an elog(ERROR), and since this happens during postmaster > startup, are you sure it won't prevent the server from starting? Oh, yes, missed that. /*

[HACKERS] shared_preload_libraries path

2012-05-24 Thread Magnus Hagander
Would i make sense to have a postgresql.conf parameter that would add to LD_LIBRARY_PATH when loading libraries from shared_preload_libraries (and other library loads). To make it possible to configure it without having to mess around with the operating system configuration? Or is that too much re

Re: [HACKERS] pg_receivexlog stops upon server restart

2012-05-24 Thread Magnus Hagander
On Thu, Apr 19, 2012 at 1:00 PM, Thom Brown wrote: > On 10 April 2012 21:07, Magnus Hagander wrote: >> On Friday, April 6, 2012, Thom Brown wrote: >>> >>> Hi, >>> >>> I've tried out pg_receivexlog and have noticed that when restarting >>> the cluster, pg_receivexlog gets cut off... it doesn't kee

Re: [HACKERS] pg_stat_statements temporary file

2012-05-24 Thread Magnus Hagander
On Thu, May 24, 2012 at 1:36 PM, Peter Geoghegan wrote: > On 24 May 2012 11:43, Magnus Hagander wrote: >> In general, should a contrib module really store data in the global/ >> directory? Seems pretty ugly to me... > > I think the case could be made for moving pg_stat_statements into > core, as

Re: [HACKERS] pg_stat_statements temporary file

2012-05-24 Thread Peter Geoghegan
On 24 May 2012 11:43, Magnus Hagander wrote: > In general, should a contrib module really store data in the global/ > directory? Seems pretty ugly to me... I think the case could be made for moving pg_stat_statements into core, as an optionally enabled view, like pg_stat_user_functions, since pg_

  1   2   >