Re: [HACKERS] clog_redo causing very long recovery time

2011-05-05 Thread Joe Conway
On 05/05/2011 09:00 PM, Tom Lane wrote: > Joe Conway writes: >> Right -- I think another similar problem exists in GetNewMultiXactId >> where ExtendMultiXactOffset could succeed and write an XLOG entry and >> then ExtendMultiXactMember could fail before advancing nextMXact. The >> problem in this

[HACKERS] Debug contrib/cube code

2011-05-05 Thread Nick Raj
Hi, I am using postgresql-8.4.6. I want to debug the contrib/cube code. Can we able to debug that cube code? Because there is no .configure file to enable debug. Is there is any way to change make file to enable debug? Thanks

[HACKERS] Re: Why is RegisterPredicateLockingXid called while holding XidGenLock?

2011-05-05 Thread Kevin Grittner
Tom Lane wrote: > Yeah, I was thinking that it'd be better to pull it out of > GetNewTransactionId and put it in a higher level. As long as it is always called when an xid is assigned. Since this function appears to be on the only path to that, it should be fine. > No strong preference abou

Re: [HACKERS] Why is RegisterPredicateLockingXid called while holding XidGenLock?

2011-05-05 Thread Tom Lane
Dan Ports writes: > On Thu, May 05, 2011 at 11:12:40PM -0400, Tom Lane wrote: >> Even if it's actually necessary to set up that data structure while >> holding XidGenLock, I would *really* like the call to not be exactly >> where it is. > Good question. > I don't believe it needs to be while Xid

[HACKERS] Re: Why is RegisterPredicateLockingXid called while holding XidGenLock?

2011-05-05 Thread Kevin Grittner
Tom Lane wrote: > This seems like a pretty lousy place to do it, first because of the > contention hit from holding that high-traffic lock any longer than > necessary, and second because every added chance for error between > ExtendCLOG() and TransactionIdAdvance(ShmemVariableCache->nextXid) > g

[HACKERS] Re: Why is RegisterPredicateLockingXid called while holding XidGenLock?

2011-05-05 Thread Dan Ports
On Thu, May 05, 2011 at 11:12:40PM -0400, Tom Lane wrote: > Even if it's actually necessary to set up that data structure while > holding XidGenLock, I would *really* like the call to not be exactly > where it is. Good question. I don't believe it needs to be while XidGenLock is being held at all

Re: [HACKERS] clog_redo causing very long recovery time

2011-05-05 Thread Tom Lane
Joe Conway writes: > Right -- I think another similar problem exists in GetNewMultiXactId > where ExtendMultiXactOffset could succeed and write an XLOG entry and > then ExtendMultiXactMember could fail before advancing nextMXact. The > problem in this case is that they both write XLOG entries, so

Re: [HACKERS] clog_redo causing very long recovery time

2011-05-05 Thread Joe Conway
On 05/05/2011 08:22 PM, Tom Lane wrote: > Joseph Conway writes: >> The attached fix-clogredo diff is my proposal for a fix for this. > > That seems pretty grotty :-( > > I think a more elegant fix might be to just swap the order of the > ExtendCLOG and ExtendSUBTRANS calls in GetNewTransactionI

Re: [HACKERS] clog_redo causing very long recovery time

2011-05-05 Thread Alvaro Herrera
Excerpts from Tom Lane's message of vie may 06 00:22:43 -0300 2011: > I think a more elegant fix might be to just swap the order of the > ExtendCLOG and ExtendSUBTRANS calls in GetNewTransactionId. The > reason that would help is that pg_subtrans isn't WAL-logged, so if > we succeed doing Extend

Re: [HACKERS] clog_redo causing very long recovery time

2011-05-05 Thread Tom Lane
Joseph Conway writes: > I'm working with a client that uses Postgres on what amounts to an > appliance. > The database is therefore subject to occasional torture such as, in this > particular case, running out of disk space while performing a million > plus queries (of mixed varieties, many using

[HACKERS] Why is RegisterPredicateLockingXid called while holding XidGenLock?

2011-05-05 Thread Tom Lane
Inquiring minds want to know ... This seems like a pretty lousy place to do it, first because of the contention hit from holding that high-traffic lock any longer than necessary, and second because every added chance for error between ExtendCLOG() and TransactionIdAdvance(ShmemVariableCache->nextX

Re: [HACKERS] Backpatching of "Teach the regular expression functions to do case-insensitive matching"

2011-05-05 Thread Robert Haas
On Thu, May 5, 2011 at 5:21 AM, Andres Freund wrote: > In my opinion this is actually a bug in < 9.0. As its a (imo) low impact fix > thats constrained to two files it seems sensible to backpatch it now that the > solution has proven itself in the field? > > The issue is hard to find and has come

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-05 Thread Tom Lane
Peter Geoghegan writes: > On 5 May 2011 22:22, Tom Lane wrote: >> What that really means is that any WaitOnLatch call with a finite >> timeout ought to be viewed with a jaundiced eye.  Ideally, we want them >> all to be waiting for latch release and nothing else.  I'm concerned >> that we're goin

Re: [HACKERS] Some surprising precedence behavior in PG's grammar

2011-05-05 Thread Tom Lane
I wrote: > Andrew Dunstan writes: >> If we do need a precedence setting for NULL_P, then I think it should >> probably be on its own and not sharing one with IS. > Yeah, I was thinking that too. If we put %prec on the IS [NOT] NULL > productions then there is no need for NULL_P to have exactly

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-05 Thread Peter Geoghegan
On 5 May 2011 22:22, Tom Lane wrote: > Robert Haas writes: >> On Thu, May 5, 2011 at 4:05 PM, Tom Lane wrote: +              * The caveat about signals invalidating the timeout of +              * WaitLatch() on some platforms can be safely disregarded, > >>> Really? > >> I'm a bit con

Re: [HACKERS] Backpatching of "Teach the regular expression functions to do case-insensitive matching"

2011-05-05 Thread Tom Lane
Andres Freund writes: > In my opinion this is actually a bug in < 9.0. As its a (imo) low impact fix > thats constrained to two files it seems sensible to backpatch it now that the > solution has proven itself in the field? FWIW, I still don't trust that patch a lot (and I was the one who wrote

Re: [HACKERS] Re: [BUGS] BUG #5957: createdb with description and md5 auth forces to provide password twice

2011-05-05 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian writes: > > Prior to PG 8.2, this was necessary to put the comment on the database, > > but now that we have the shared comment/description table > > pg_shdescription, this is not necessary. > > > Do we need createdb to be able to create databases for pre-8.2 clust

Re: [HACKERS] Re: [BUGS] BUG #5957: createdb with description and md5 auth forces to provide password twice

2011-05-05 Thread Tom Lane
Bruce Momjian writes: > Prior to PG 8.2, this was necessary to put the comment on the database, > but now that we have the shared comment/description table > pg_shdescription, this is not necessary. > Do we need createdb to be able to create databases for pre-8.2 clusters? > If not, the attached

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-05 Thread Tom Lane
Robert Haas writes: > On Thu, May 5, 2011 at 4:05 PM, Tom Lane wrote: >>> + * The caveat about signals invalidating the timeout of >>> + * WaitLatch() on some platforms can be safely disregarded, >> Really? > I'm a bit confused by the phrasing of this comment as well,

[HACKERS] Re: [BUGS] BUG #5957: createdb with description and md5 auth forces to provide password twice

2011-05-05 Thread Bruce Momjian
Grzegorz Szpetkowski wrote: > > The following bug has been logged online: > > Bug reference: 5957 > Logged by: Grzegorz Szpetkowski > Email address: gszpetkow...@gmail.com > PostgreSQL version: 9.0.3 > Operating system: Ubuntu 10.10 > Description:createdb with descrip

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-05 Thread Robert Haas
On Thu, May 5, 2011 at 4:05 PM, Tom Lane wrote: >> +              * The caveat about signals invalidating the timeout of >> +              * WaitLatch() on some platforms can be safely disregarded, > > Really? I'm a bit confused by the phrasing of this comment as well, but it does seem to me that

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-05 Thread A.M.
On May 5, 2011, at 4:08 PM, Alvaro Herrera wrote: > Excerpts from Peter Geoghegan's message of jue may 05 16:49:25 -0300 2011: > >> I'll need to take a look at statistics, autovacuum and Logger >> processes too, to see if they present more subtle opportunities for >> reduced idle power consumpti

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-05 Thread Alvaro Herrera
Excerpts from Peter Geoghegan's message of jue may 05 16:49:25 -0300 2011: > I'll need to take a look at statistics, autovacuum and Logger > processes too, to see if they present more subtle opportunities for > reduced idle power consumption. More subtle? Autovacuum wakes up once per second and

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-05 Thread Tom Lane
Peter Geoghegan writes: > Attached is the first in what I hope will become a series of patches > for reducing power consumption when idle. Cool. This has been on my personal to-do list for awhile, but it keeps on failing to get to the top, so I'm glad to see somebody else putting time into it.

Re: [HACKERS] new clang report

2011-05-05 Thread Tom Lane
Peter Eisentraut writes: > Btw., when you build a simple test program in the default mode, pow() > indeed returns Inf on overflow. There appear to be some code generation > or optimization problems when it builds the postgres code, because the > problem goes away with either -O0 or by inserting a

[HACKERS] Process wakeups when idle and power consumption

2011-05-05 Thread Peter Geoghegan
There is a general need to have Postgres consume fewer CPU cycles and less power when idle. Until something is done about this, shared hosting providers, particularly those who want to deploy many VM instances with databases, will continue to choose MySQL out of hand. I have quantified the differe

Re: [HACKERS] FDW table hints

2011-05-05 Thread Magnus Hagander
On Thu, May 5, 2011 at 19:26, Magnus Hagander wrote: > On Thu, May 5, 2011 at 19:22, Tom Lane wrote: >> Magnus Hagander writes: >>> Since I brought it up - a patch along this line? >> >> Please don't capitalize "foreign table" there. > > Yeah, I was a bit split about that myself. Will change. >

Re: [HACKERS] Visibility map and hint bits

2011-05-05 Thread Merlin Moncure
On Thu, May 5, 2011 at 2:00 PM, Kevin Grittner wrote: > Merlin Moncure wrote: > >> a small cache that remembers the commit/cancel status of recently >> seen transactions. > > How is that different from the head of the clog SLRU? several things: *) any slru access requires lock (besides the lock

Re: [HACKERS] new clang report

2011-05-05 Thread Peter Eisentraut
On mån, 2011-05-02 at 14:45 -0400, Tom Lane wrote: > > So issue here is actually that clang has an option > > >-fmath-errno > >Indicate that math functions should be treated as > updating errno. > > Really? It looks to me like the issue is that pow() is returning NaN > instea

Re: [HACKERS] Visibility map and hint bits

2011-05-05 Thread Kevin Grittner
Merlin Moncure wrote: > a small cache that remembers the commit/cancel status of recently > seen transactions. How is that different from the head of the clog SLRU? -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www

Re: [HACKERS] Visibility map and hint bits

2011-05-05 Thread Merlin Moncure
On Thu, May 5, 2011 at 1:34 PM, Bruce Momjian wrote: > Merlin Moncure wrote: >> On Thu, May 5, 2011 at 11:59 AM, Bruce Momjian wrote: >> > There has been a lot of recent discussion about the visibility map (for >> > index-only scans) and hint bits (trying to avoid double-writing a >> > table). >>

Re: [HACKERS] Visibility map and hint bits

2011-05-05 Thread Robert Haas
On Thu, May 5, 2011 at 12:59 PM, Bruce Momjian wrote: > I wonder if we could fix both of these at the same time.  Once the > visibility map is reliable, can we use that to avoid updating the hint > bits on all rows on a page? I don't think so. There are two problems: 1. If there is a long-runni

Re: [HACKERS] Comments on system tables and columns

2011-05-05 Thread Bruce Momjian
Alvaro Herrera wrote: > Excerpts from Thom Brown's message of lun mar 28 08:14:07 -0300 2011: > > Hi, > > > > I notice that none of the system tables or columns thereof bear any > > comments. Is this intentional, or an oversight? I would have thought > > comments would be useful since the column

Re: [HACKERS] Visibility map and hint bits

2011-05-05 Thread Bruce Momjian
Merlin Moncure wrote: > On Thu, May 5, 2011 at 11:59 AM, Bruce Momjian wrote: > > There has been a lot of recent discussion about the visibility map (for > > index-only scans) and hint bits (trying to avoid double-writing a > > table). > > I still think a small tqual.c maintained cache of hint bi

Re: [HACKERS] Visibility map and hint bits

2011-05-05 Thread Merlin Moncure
On Thu, May 5, 2011 at 11:59 AM, Bruce Momjian wrote: > There has been a lot of recent discussion about the visibility map (for > index-only scans) and hint bits (trying to avoid double-writing a > table). I still think a small tqual.c maintained cache of hint bits will effectively eliminate hint

Re: [HACKERS] patch: fix race in SSI's CheckTargetForConflictsIn

2011-05-05 Thread Kevin Grittner
Dan Ports wrote: > While running some benchmarks to test SSI performance, I found a > race condition that's capable of causing a segfault. A patch is > attached. > > The bug is in CheckTargetForConflictsIn, which scans the list of > SIREAD locks on a lock target when it's modified. There's an > o

Re: [HACKERS] Patch to improve style of generate_history.pl perl script

2011-05-05 Thread Bruce Momjian
Andrew Dunstan wrote: > > > On 05/05/2011 12:49 PM, Bruce Momjian wrote: > > I have applied the attached patch to improve the style of our > > generate_history.pl perl script. > > > > Wow, really? There's nothing wrong with the changes but they seem pretty > trivial and pointless to me. Maybe I

Re: [HACKERS] FDW table hints

2011-05-05 Thread Magnus Hagander
On Thu, May 5, 2011 at 19:22, Tom Lane wrote: > Magnus Hagander writes: >> Since I brought it up - a patch along this line? > > Please don't capitalize "foreign table" there. Yeah, I was a bit split about that myself. Will change. > Also, I think an "else" before the other ereport would make t

Re: [HACKERS] FDW table hints

2011-05-05 Thread Tom Lane
Magnus Hagander writes: > Since I brought it up - a patch along this line? Please don't capitalize "foreign table" there. Also, I think an "else" before the other ereport would make the code clearer, even though it's not strictly necessary. regards, tom lane -- Sent vi

Re: [HACKERS] VARIANT / ANYTYPE datatype

2011-05-05 Thread Andrew Dunstan
On 05/05/2011 01:00 PM, Jim Nasby wrote: On May 4, 2011, at 6:24 PM, Andrew Dunstan wrote: I'm far from convinced that storing deltas per column rather than per record is a win anyway. I don't have hard numbers to hand, but my vague recollection is that my tests showed it to be a design that

Re: [HACKERS] FDW table hints

2011-05-05 Thread Magnus Hagander
On Tue, May 3, 2011 at 16:19, Tom Lane wrote: > Dave Page writes: >> On Tue, May 3, 2011 at 10:33 AM, Susanne Ebrecht >> wrote: >>> When we make such a hint for foreign tables then we should make a similar >>> hint for views. > >> A view really isn't a table, unlike a foreign table, so I don't t

Re: [HACKERS] Patch to improve style of generate_history.pl perl script

2011-05-05 Thread Andrew Dunstan
On 05/05/2011 12:49 PM, Bruce Momjian wrote: I have applied the attached patch to improve the style of our generate_history.pl perl script. Wow, really? There's nothing wrong with the changes but they seem pretty trivial and pointless to me. Maybe I've drunk the perl TIMTOWTDI koolaid too

Re: [HACKERS] VARIANT / ANYTYPE datatype

2011-05-05 Thread Jim Nasby
On May 4, 2011, at 6:24 PM, Andrew Dunstan wrote: > I'm far from convinced that storing deltas per column rather than per record > is a win anyway. I don't have hard numbers to hand, but my vague recollection > is that my tests showed it to be a design that used more space. It depends on how man

[HACKERS] Visibility map and hint bits

2011-05-05 Thread Bruce Momjian
There has been a lot of recent discussion about the visibility map (for index-only scans) and hint bits (trying to avoid double-writing a table). I wonder if we could fix both of these at the same time. Once the visibility map is reliable, can we use that to avoid updating the hint bits on all ro

Re: [HACKERS] alpha5

2011-05-05 Thread Bruce Momjian
Robert Haas wrote: > On Mon, Mar 28, 2011 at 9:26 AM, Tom Lane wrote: > > Robert Haas writes: > >> On Sat, Mar 26, 2011 at 9:41 PM, Robert Haas wrote: > >>> Per previous discussion, I'm going to wrap alpha5 Monday morning > >>> Eastern time, barring objections. > > > >> It seems that the 'make d

[HACKERS] Patch to improve style of generate_history.pl perl script

2011-05-05 Thread Bruce Momjian
I have applied the attached patch to improve the style of our generate_history.pl perl script. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + diff --git a/doc/src/sgml/generate_hist

Re: [HACKERS] adding a new column in IDENTIFY_SYSTEM

2011-05-05 Thread Tom Lane
Magnus Hagander writes: >> So even if people don't believe in the rationale behind the patch, >> would allowing it harm anything at this point? > Adding it for the sake of upgrades seems very far fetched. > Adding it for the sake of giving a better error message seems like a > very good idea. Bu

Re: [HACKERS] adding a new column in IDENTIFY_SYSTEM

2011-05-05 Thread Magnus Hagander
On Wed, May 4, 2011 at 22:42, Simon Riggs wrote: > On Wed, May 4, 2011 at 3:47 AM, Tom Lane wrote: >> Jaime Casanova writes: >>> I want to propose the addition of a new field in IDENTIFY_SYSTEM: >>> xlogversion, which will carry XLOG_PAGE_MAGIC from primary. >>> The idea of sending that info is

Re: [HACKERS] VARIANT / ANYTYPE datatype

2011-05-05 Thread Merlin Moncure
On Wed, May 4, 2011 at 8:03 PM, Tom Lane wrote: > Alvaro Herrera writes: >> As a followup idea there exists the desire to store records as records >> and not text representation of same (given differing record types, of >> course), for which it'd be more worthwhile. > > Maybe.  The conventional w

Re: [HACKERS] GSoC 2011: Fast GiST index build

2011-05-05 Thread Alexander Korotkov
2011/5/5 Teodor Sigaev > > See several lines above: >if (parent->blkno == InvalidBlockNumber) > >/* > * end of chain and still didn't found parent, It's > very-very > * rare situation when root splited > */ >

Re: [HACKERS] patch for new feature: Buffer Cache Hibernation

2011-05-05 Thread Cédric Villemain
2011/5/5 Mitsuru IWASAKI : > Hi, > >> I think that PgFincore (http://pgfoundry.org/projects/pgfincore/) >> provides similar functionality.  Are you familiar with that?  If so, >> could you contrast your approach with that one? > > I'm not familiar with PgFincore at all sorry, but I got source code

Re: [HACKERS] GSoC 2011: Fast GiST index build

2011-05-05 Thread Teodor Sigaev
gistFindCorrectParent function have branch with following comment: /* * awful!!, we need search tree to find parent ... , but before we * should release all old parent */ Can you provide me an example of case when this branch works? See several lines above: if (parent->blkno == Inval

Re: [HACKERS] patch for new feature: Buffer Cache Hibernation

2011-05-05 Thread Mitsuru IWASAKI
Hi, > I think that PgFincore (http://pgfoundry.org/projects/pgfincore/) > provides similar functionality. Are you familiar with that? If so, > could you contrast your approach with that one? I'm not familiar with PgFincore at all sorry, but I got source code and documents and read through them

Re: [HACKERS] patch for new feature: Buffer Cache Hibernation

2011-05-05 Thread Mitsuru IWASAKI
Hi, thanks for good suggestions. > > Postgres usually starts with ZERO buffer cache. By saving the buffer > > cache data structure into hibernation files just before shutdown, and > > loading them at startup, postgres can start operations with the saved > > buffer cache as the same condition as j

[HACKERS] Backpatching of "Teach the regular expression functions to do case-insensitive matching"

2011-05-05 Thread Andres Freund
Hi, In my opinion this is actually a bug in < 9.0. As its a (imo) low impact fix thats constrained to two files it seems sensible to backpatch it now that the solution has proven itself in the field? The issue is hard to find and has come up several times in the field. And it has been slightl

Re: [HACKERS] Pull up aggregate subquery

2011-05-05 Thread Hitoshi Harada
2011/5/5 Tom Lane : > Hitoshi Harada writes: >> I sometimes wonder if we could pull up aggregate query in the optimizer. > > I don't have time to look at this right now, but please add to the > upcoming commitfest. Done. https://commitfest.postgresql.org/action/patch_view?id=548 I'll work furthe

Re: [HACKERS] patch for new feature: Buffer Cache Hibernation

2011-05-05 Thread Cédric Villemain
2011/5/4 Josh Berkus : > All, > > I thought that Dimitri had already implemented this using Fincore.  It's > linux-only, but that should work well enough to test the general concept. Harald provided me some pointers at pgday in Stuttgart to make it work with windows but ... hum I have not windows

Re: [HACKERS] patch for new feature: Buffer Cache Hibernation

2011-05-05 Thread Dimitri Fontaine
Josh Berkus writes: > I thought that Dimitri had already implemented this using Fincore. It's > linux-only, but that should work well enough to test the general concept. Actually, Cédric did, and I have a clone of his repository where I did some debian packaging of it. http://villemain.org/pr