Re: [HACKERS] canceling autovacuum task woes

2012-07-24 Thread Tom Lane
Robert Haas writes: > On Jul 24, 2012, at 4:31 PM, Tom Lane wrote: >> ... This means that with respect to (a), the connection >> from the process doing the kill to the AV proc may be inadequately >> documented by this patch, and with respect to (b), there might well be >> cases where we found an

Re: [HACKERS] [BUGS] BUG #6748: sequence value may be conflict in some cases

2012-07-24 Thread Tom Lane
I wrote: > The tuple in buffers has log_cnt = 1, is_called = false, but the initial > XLOG_SEQ_LOG record shows log_cnt = 0, is_called = true. So if we crash > at this point, after recovery it looks like one nextval() has already > been done. However, AlterSequence generates another XLOG_SEQ_LOG

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tuesday, July 24, 2012, Marko Kreen wrote: > On Wed, Jul 25, 2012 at 1:29 AM, Merlin Moncure wrote: >> On Tue, Jul 24, 2012 at 4:59 PM, Marko Kreen wrote: >>> So if we give only PQgetResult() in 9.2, I do not see that we >>> are locked out from any interesting optimizations. >> >> Well, you a

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Marko Kreen
On Wed, Jul 25, 2012 at 1:29 AM, Merlin Moncure wrote: > On Tue, Jul 24, 2012 at 4:59 PM, Marko Kreen wrote: >> So if we give only PQgetResult() in 9.2, I do not see that we >> are locked out from any interesting optimizations. > > Well, you are locked out of having PQgetResult reuse the conn's r

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 5:29 PM, Merlin Moncure wrote: > so #2 seems like the lowest common > denominator (it would permanently preclude #3 and would require #4 to > introduce two new functions instead of just one). #1 of course would > bolt on to #2. oops, got #1 and #2 backwards there. merlin

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 4:59 PM, Marko Kreen wrote: > So if we give only PQgetResult() in 9.2, I do not see that we > are locked out from any interesting optimizations. Well, you are locked out of having PQgetResult reuse the conn's result since that would then introduce potentially breaking chan

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Marko Kreen
On Wed, Jul 25, 2012 at 12:35 AM, Merlin Moncure wrote: > On Tue, Jul 24, 2012 at 3:52 PM, Marko Kreen wrote: >> * Is there better API than PQsetSingleRowMode()? New PQsend* >> functions is my alternative. > > I would prefer to have PQsetSingleRowMode() over new flavor of PQsend. > > To consol

Re: [HACKERS] canceling autovacuum task woes

2012-07-24 Thread Robert Haas
On Jul 24, 2012, at 4:31 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Jul 24, 2012 at 4:03 PM, Alvaro Herrera >> wrote: >>> Looks great. Are you considering backpatching this? > >> Well, that would certainly make MY life easier. I am not sure whether >> it would be in line with projec

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 3:52 PM, Marko Kreen wrote: > * Is there better API than PQsetSingleRowMode()? New PQsend* > functions is my alternative. I would prefer to have PQsetSingleRowMode() over new flavor of PQsend. To consolidate my argument above: since you're throwing PQgetResult in the m

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Marko Kreen
On Tue, Jul 24, 2012 at 10:37 PM, Tom Lane wrote: > Merlin Moncure writes: >> Either way, it looks like there's plausible paths to optimizing >> repeated result fetch without having expose an alternate data-fetching >> API and that the proposed implementation doesn't appear to be a wall >> in ter

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Marko Kreen
On Tue, Jul 24, 2012 at 11:34 PM, Merlin Moncure wrote: > On Tue, Jul 24, 2012 at 2:37 PM, Tom Lane wrote: >> In particular I agree that PQsetSingleRowMode is a bit ugly, so I'm >> wondering about your thoughts on providing PQgetSingleRowResult instead. >> I don't see how to make that work in asy

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 2:37 PM, Tom Lane wrote: > Merlin Moncure writes: >> Either way, it looks like there's plausible paths to optimizing >> repeated result fetch without having expose an alternate data-fetching >> API and that the proposed implementation doesn't appear to be a wall >> in term

Re: [HACKERS] CHECK NO INHERIT syntax

2012-07-24 Thread Alvaro Herrera
Excerpts from Alvaro Herrera's message of sáb jul 21 00:20:57 -0400 2012: > Here's a (hopefully) complete patch. Pushed. I hope people like this one better (third time's the charm, and all that). -- Álvaro Herrera The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulti

Re: [HACKERS] canceling autovacuum task woes

2012-07-24 Thread Tom Lane
Robert Haas writes: > On Tue, Jul 24, 2012 at 4:03 PM, Alvaro Herrera > wrote: >> Looks great. Are you considering backpatching this? > Well, that would certainly make MY life easier. I am not sure whether > it would be in line with project policy, however. +1 for a backpatch. Otherwise it'l

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Marko Kreen
> Hm, I think it's possible to rig the test to do dummy > copy of pgresult, thus it's possible to see what kind > of speed is possible.. Will try. I added a new method (-x) to rowdump where it asks for row with PQgetRowData() and then tries to emulate super-efficient PGresult copy, then loads dat

Re: [HACKERS] canceling autovacuum task woes

2012-07-24 Thread Robert Haas
On Tue, Jul 24, 2012 at 4:09 PM, Tom Lane wrote: > Robert Haas writes: >> Yeah, you're right. So you do get the table name. But you don't get >> the cause, which is what you really need to understand why it's >> happening. Attached is a patch that adds some more detail. > > Uh, what's the adde

Re: [HACKERS] canceling autovacuum task woes

2012-07-24 Thread Tom Lane
Robert Haas writes: > Yeah, you're right. So you do get the table name. But you don't get > the cause, which is what you really need to understand why it's > happening. Attached is a patch that adds some more detail. Uh, what's the added dependency on pgstat.h for? Looks sane to the eyeball o

Re: [HACKERS] canceling autovacuum task woes

2012-07-24 Thread Robert Haas
On Tue, Jul 24, 2012 at 4:03 PM, Alvaro Herrera wrote: > Excerpts from Robert Haas's message of mar jul 24 15:52:23 -0400 2012: >> On Tue, Jul 24, 2012 at 3:35 PM, Alvaro Herrera >> wrote: >> > Yep, it says: >> > >> > ERROR: canceling autovacuum task >> > CONTEXT: automatic vacuum of table "alv

Re: [HACKERS] canceling autovacuum task woes

2012-07-24 Thread Alvaro Herrera
Excerpts from Robert Haas's message of mar jul 24 15:52:23 -0400 2012: > On Tue, Jul 24, 2012 at 3:35 PM, Alvaro Herrera > wrote: > > Yep, it says: > > > > ERROR: canceling autovacuum task > > CONTEXT: automatic vacuum of table "alvherre.public.foo" > > > > So at least that part seems pilot err

Re: [HACKERS] canceling autovacuum task woes

2012-07-24 Thread Robert Haas
On Tue, Jul 24, 2012 at 3:35 PM, Alvaro Herrera wrote: > Yep, it says: > > ERROR: canceling autovacuum task > CONTEXT: automatic vacuum of table "alvherre.public.foo" > > So at least that part seems pilot error more than anything else. Yeah, you're right. So you do get the table name. But you

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Tom Lane
Merlin Moncure writes: > Either way, it looks like there's plausible paths to optimizing > repeated result fetch without having expose an alternate data-fetching > API and that the proposed implementation doesn't appear to be a wall > in terms of getting there. So I'm firmly in the non-exposed-row

Re: [HACKERS] canceling autovacuum task woes

2012-07-24 Thread Alvaro Herrera
Excerpts from Alvaro Herrera's message of mar jul 24 15:30:49 -0400 2012: > > > On Tuesday, July 24, 2012 07:48:27 PM Robert Haas wrote: > > > I am running into a lot of customer situations where the customer > > > reports that "canceling autovacuum task" shows up in the logs, and > > > it's uncl

Re: [HACKERS] canceling autovacuum task woes

2012-07-24 Thread Alvaro Herrera
> On Tuesday, July 24, 2012 07:48:27 PM Robert Haas wrote: > > I am running into a lot of customer situations where the customer > > reports that "canceling autovacuum task" shows up in the logs, and > > it's unclear whether this is happening often enough to matter, and > > even more unclear what

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 1:49 PM, Merlin Moncure wrote: > On Tue, Jul 24, 2012 at 1:33 PM, Merlin Moncure wrote: >> The 'source' result (or source data that would be copied into the >> destination result) would be stored in the PGconn, right? So, the idea >> is that when you set up single row mode

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 1:33 PM, Merlin Moncure wrote: > The 'source' result (or source data that would be copied into the > destination result) would be stored in the PGconn, right? So, the idea > is that when you set up single row mode the connection generates a > template PGconn which is then c

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 11:57 AM, Marko Kreen wrote: > On Tue, Jul 24, 2012 at 7:52 PM, Merlin Moncure wrote: >> But, the faster rowbuf method is a generally incompatible way of >> dealing with data vs current libpq -- this is bad. If it's truly >> impossible to get those benefits without bypass

Re: [HACKERS] canceling autovacuum task woes

2012-07-24 Thread Andrew Dunstan
On 07/24/2012 01:48 PM, Robert Haas wrote: I am running into a lot of customer situations where the customer reports that "canceling autovacuum task" shows up in the logs, and it's unclear whether this is happening often enough to matter, and even more unclear what's causing it. Me: So, do you

Re: [HACKERS] canceling autovacuum task woes

2012-07-24 Thread Robert Haas
On Tue, Jul 24, 2012 at 2:11 PM, Steve Singer wrote: > On 12-07-24 01:48 PM, Robert Haas wrote: >> I am running into a lot of customer situations where the customer >> reports that "canceling autovacuum task" shows up in the logs, and >> it's unclear whether this is happening often enough to matte

Re: [HACKERS] canceling autovacuum task woes

2012-07-24 Thread Steve Singer
On 12-07-24 01:48 PM, Robert Haas wrote: I am running into a lot of customer situations where the customer reports that "canceling autovacuum task" shows up in the logs, and it's unclear whether this is happening often enough to matter, and even more unclear what's causing it. Could autovacuum

Re: [HACKERS] canceling autovacuum task woes

2012-07-24 Thread Andres Freund
Hi, On Tuesday, July 24, 2012 07:48:27 PM Robert Haas wrote: > I am running into a lot of customer situations where the customer > reports that "canceling autovacuum task" shows up in the logs, and > it's unclear whether this is happening often enough to matter, and > even more unclear what's caus

Re: [HACKERS] 9.2 release schedule

2012-07-24 Thread Fujii Masao
On Tue, Jul 24, 2012 at 4:10 AM, Noah Misch wrote: > On Mon, Jul 23, 2012 at 10:29:06AM -0400, Tom Lane wrote: >> Robert Haas writes: >> > Seems OK, but I think we need to work a little harder on evicting some >> > things from the list of open items. I don't think all of the things >> > listed i

[HACKERS] canceling autovacuum task woes

2012-07-24 Thread Robert Haas
I am running into a lot of customer situations where the customer reports that "canceling autovacuum task" shows up in the logs, and it's unclear whether this is happening often enough to matter, and even more unclear what's causing it. Me: So, do you know what table it's getting cancelled on? Cus

Re: [HACKERS] Recent absence

2012-07-24 Thread Atri Sharma
On Tue, Jul 24, 2012 at 11:05 PM, Joshua D. Drake wrote: > > On 07/24/2012 10:26 AM, Kevin Grittner wrote: >> >> >> I have recently been laid up with a MRSA infection, which left me >> suddenly unable to deal with even reading my email. (I took a shot >> at catching up a week ago, and it was too

Re: [HACKERS] Recent absence

2012-07-24 Thread Joshua D. Drake
On 07/24/2012 10:26 AM, Kevin Grittner wrote: I have recently been laid up with a MRSA infection, which left me suddenly unable to deal with even reading my email. (I took a shot at catching up a week ago, and it was too taxing, resulting in a relapse.) I'm trying to fight though the backlog

[HACKERS] Recent absence

2012-07-24 Thread Kevin Grittner
I have recently been laid up with a MRSA infection, which left me suddenly unable to deal with even reading my email. (I took a shot at catching up a week ago, and it was too taxing, resulting in a relapse.) I'm trying to fight though the backlog and get caught up, without pushing it too hard too

Re: [HACKERS] Use of rsync for data directory copying

2012-07-24 Thread Kevin Grittner
Bruce Momjian wrote: > if a write happens in both the first and second half of a second, While I'm not sure whether I believe that granularity is really to the nanosecond, a stat of a table in a production database on xfs shows this: Modify: 2012-07-24 10:15:44.096415501 -0500 So presumabl

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-24 Thread karavelov
- Цитат от David Fetter (da...@fetter.org), на 23.07.2012 в 15:41 - I'm not sure how you automate testing a pull-the-plug scenario. I have a dim memory of how the FreeBSD project was alleged to have done it, namely by rigging a serial port (yes, it was that long ago) to the pow

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Tom Lane
Merlin Moncure writes: > I think the dummy copy of PGresult is plausible (if by that you mean > optimizing PQgetResult when in single row mode). That would be even > better: you'd remove the need for the rowbuf mode. I haven't spent any time looking at this, but my gut tells me that a big chunk

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Marko Kreen
On Tue, Jul 24, 2012 at 7:52 PM, Merlin Moncure wrote: > But, the faster rowbuf method is a generally incompatible way of > dealing with data vs current libpq -- this is bad. If it's truly > impossible to get those benefits without bypassing result API that > then I remove my objection on the gro

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 11:39 AM, Marko Kreen wrote: > On Tue, Jul 24, 2012 at 7:25 PM, Merlin Moncure wrote: >> On Tue, Jul 24, 2012 at 11:08 AM, Marko Kreen wrote: I'm arguing that *all* data getting must continue to do so through the result object, and bypassing the result to get at

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Marko Kreen
On Tue, Jul 24, 2012 at 7:25 PM, Merlin Moncure wrote: > On Tue, Jul 24, 2012 at 11:08 AM, Marko Kreen wrote: >>> I'm arguing that *all* data getting must continue to do so through the >>> result object, and bypassing the result to get at data is breaking the >>> result abstraction in the libpq a

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Marko Kreen
On Tue, Jul 24, 2012 at 7:08 PM, Marko Kreen wrote: > The PQgetRowData() is unimportant as it just exposes > the rowBuf to user and thats all. So to get back to the more interesting PQgetRowData() discussion... Did you notice that it's up to app to decide whether it calls PQgetResult() or PQgetR

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 11:08 AM, Marko Kreen wrote: >> I'm arguing that *all* data getting must continue to do so through the >> result object, and bypassing the result to get at data is breaking the >> result abstraction in the libpq api. I bet you can still maintain >> data access through resu

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Marko Kreen
On Tue, Jul 24, 2012 at 6:18 PM, Merlin Moncure wrote: > On Mon, Jul 23, 2012 at 5:07 PM, Marko Kreen wrote: >>> Does PQgetRowData() break the ability to call PQgetvalue() against the >>> result as well as other functions like PQgetisnull()? If so, I >>> object. >> >> I don't get what are you ob

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 10:49 AM, Tom Lane wrote: > Merlin Moncure writes: >> I'm arguing that *all* data getting must continue to do so through the >> result object, and bypassing the result to get at data is breaking the >> result abstraction in the libpq api. > > That's a fair point, but the s

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Tom Lane
Merlin Moncure writes: > I'm arguing that *all* data getting must continue to do so through the > result object, and bypassing the result to get at data is breaking the > result abstraction in the libpq api. That's a fair point, but the single-row mode without PQgetRowData still fits that model,

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Mon, Jul 23, 2012 at 5:07 PM, Marko Kreen wrote: >> Does PQgetRowData() break the ability to call PQgetvalue() against the >> result as well as other functions like PQgetisnull()? If so, I >> object. > > I don't get what are you objecting to. The PQgetRowData() > is called instead of PQgetRes

Re: [HACKERS] isolation check takes a long time

2012-07-24 Thread Alvaro Herrera
Excerpts from Noah Misch's message of dom jul 22 17:11:53 -0400 2012: > I was pondering something like this: > > setting "i-rc" "isolation" = "READ COMMITTED" > setting "i-rr" "isolation" = "REPEATABLE READ" > > session "s1" > setup{ BEGIN TRANSACTION ISOLATION LEVEL :is

[HACKERS] Re: Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-24 Thread Greg Stark
On Wed, Jul 18, 2012 at 1:13 AM, Craig Ringer wrote: > That makes me wonder if on top of the buildfarm, extending some buildfarm > machines into a "crashfarm" is needed: > > - Keep kvm instances with copy-on-write snapshot disks and the build env > on them > - Fire up the VM, do a build, and star