Re: [HACKERS] strange buildfarm failures

2007-04-25 Thread Stefan Kaltenbrunner
Alvaro Herrera wrote: > Tom Lane wrote: >> Stefan Kaltenbrunner <[EMAIL PROTECTED]> writes: >>> Stefan Kaltenbrunner wrote: two of my buildfarm members had different but pretty weird looking failures lately: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=quagga&dt=2007-04-25%2002:

Re: [HACKERS] strange buildfarm failures

2007-04-25 Thread Alvaro Herrera
Tom Lane wrote: > Stefan Kaltenbrunner <[EMAIL PROTECTED]> writes: > > Stefan Kaltenbrunner wrote: > >> two of my buildfarm members had different but pretty weird looking > >> failures lately: > >> http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=quagga&dt=2007-04-25%2002:03:03 > >> and > >> > >>

Re: [HACKERS] autovacuum does not start in HEAD

2007-04-25 Thread ITAGAKI Takahiro
I wrote: > I found that autovacuum launcher does not launch any workers in HEAD. The attached autovacuum-fix.patch could fix the problem. I changed to use 'greater or equal' instead of 'greater' at the decision of next autovacuum target. The point was in the resolution of timer; There is a platfo

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-25 Thread Koichi Suzuki
Hi, Zeugswetter Andreas ADI SD wrote: I don't insist the name and the default of the GUC parameter. I'm afraid wal_fullpage_optimization = on (default) makes some confusion because the default behavior becomes a bit different on WAL itself. Seems my wal_fullpage_optimization is not a good n

Re: [HACKERS] Fragmentation project

2007-04-25 Thread Josh Berkus
Gustavo, The pgpool is an interesting approach to this, but I think that the funcionality of inserting a record at a backend which will be "redirectioned" to other and verifying deadlocks under network demands in acquiring locks on the referenced records/tables in several hosts. Then, IMO, this

Re: [HACKERS] database size estimates

2007-04-25 Thread Gregory Stark
"Heikki Linnakangas" <[EMAIL PROTECTED]> writes: > Francois Deliege wrote: >> Hi, >> >> I am trying to estimate the size of a table composed of 51754000 rows. >> Each row has 31 attributes: 16 x bit(24) and 15 x bit(8) >> >> So, the payload should be: >> 51754000 x ( 16 x 24 + 15 x 24 ) bits = 31

Re: [HACKERS] strange buildfarm failures

2007-04-25 Thread Tom Lane
Stefan Kaltenbrunner <[EMAIL PROTECTED]> writes: > Stefan Kaltenbrunner wrote: >> two of my buildfarm members had different but pretty weird looking >> failures lately: >> http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=quagga&dt=2007-04-25%2002:03:03 >> and >> >> http://www.pgbuildfarm.org/cgi-

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-25 Thread Tom Lane
Josh Berkus <[EMAIL PROTECTED]> writes: > Andreas, >> So imho pg_compresslog is the correct path forward. The current >> discussion is only about whether we want a more complex pg_compresslog >> and no change to current WAL, or an increased WAL size for a less >> complex implementation. >> Both wou

Re: [HACKERS] Avoiding unnecessary reads in recovery

2007-04-25 Thread Tom Lane
Heikki Linnakangas <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> but it'd make it safe to use in non-WAL contexts (I think there are >> other places where we know we are going to init the page and so a >> physical read is a waste of time). > Is there? I can't think of any. Extending a relation

Re: [HACKERS] [BUGS] BUG #3245: PANIC: failed to re-find shared loc k o b j ect

2007-04-25 Thread Tom Lane
Heikki Linnakangas <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> I still don't want to introduce more checking overhead into >> hash_seq_search, though, so what I'm now thinking about is a new >> dynahash primitive named something like "hash_freeze", which'd mark a >> hashtable as disallowing in

Re: [HACKERS] Avoiding unnecessary reads in recovery

2007-04-25 Thread Heikki Linnakangas
Tom Lane wrote: "Simon Riggs" <[EMAIL PROTECTED]> writes: As regards the zero_damaged_pages question, I raised that some time ago but we didn't arrive at an explicit answer. All I would say is we can't allow invalid pages in the buffer manager at any time, whatever options we have requested, oth

[HACKERS] Re: [BUGS] BUG #3245: PANIC: failed to re-find shared loc k o b j ect

2007-04-25 Thread Heikki Linnakangas
Tom Lane wrote: I wrote: Heikki Linnakangas <[EMAIL PROTECTED]> writes: We could have two kinds of seq scans, with and without support for concurrent inserts. Yeah, I considered that too, but it just seems too error-prone. We could maybe make it trustworthy by having hash_seq_search complai

Re: [HACKERS] Fragmentation project

2007-04-25 Thread Gustavo Tonini
Marko, On 4/24/07, Marko Kreen <[EMAIL PROTECTED]> wrote: On 4/23/07, Heikki Linnakangas <[EMAIL PROTECTED]> wrote: > Oh, you're talking about distributing partitions across different nodes > and parallelizing queries. No, we don't do that today. PL/Proxy actually works like that, only in small

Re: [HACKERS] Fragmentation project

2007-04-25 Thread Gustavo Tonini
Josh, On 4/23/07, Josh Berkus <[EMAIL PROTECTED]> wrote: Gustavo, > > Oh, you're talking about distributing partitions across different nodes > > and parallelizing queries. No, we don't do that today. > > Yes.This is the goal. Well, I will try it. I'll send the project > reports to this list. C

Re: [HACKERS] ECPG failure on BF member Vaquita (Windows Vista)

2007-04-25 Thread Tom Lane
Andrew Dunstan <[EMAIL PROTECTED]> writes: > But I also see that my amd64/FC6 machine does pass these tests with gcc. Yeah, but the typedef represented by va_list can and probably does vary between amd64 and ppc64. I haven't an easy way to check, but I wonder whether it's not an array type on ppc

[HACKERS] My upcoming travel

2007-04-25 Thread Bruce Momjian
I am leaving Friday for a two week trip to Sydney and Melbourne, Australia, Mumbai and Pune, India, and London, England. I will be attending Open Cebit in Sydney, and a Sydney PostgreSQL Users Group meeting. London is planning to put together a user group meeting. I expect both events to appear o

Re: [HACKERS] ECPG failure on BF member Vaquita (Windows Vista)

2007-04-25 Thread Andrew Dunstan
Tom Lane wrote: Hmm, and I don't have to look far to find a smoking gun: #if defined(__GNUC__) && (defined (__powerpc__) || defined(__amd64__) || defined(__x86_64__)) if (create_statement(lineno, compat, force_indicator, con, &stmt, query, args) == false) #else if (create_stat

Re: [HACKERS] ECPG failure on BF member Vaquita (Windows Vista)

2007-04-25 Thread Mark Wong
On 4/25/07, Tom Lane <[EMAIL PROTECTED]> wrote: "Mark Wong" <[EMAIL PROTECTED]> writes: > Does this help? > (gdb) p var->ind_pointer > $8 = (void *) 0x0 Well, that seems to be the reason why it's failing to indirect through ind_pointer ... but why is it only failing on your machine and not ever

Re: [HACKERS] ECPG failure on BF member Vaquita (Windows Vista)

2007-04-25 Thread Tom Lane
"Mark Wong" <[EMAIL PROTECTED]> writes: > Does this help? > (gdb) p var->ind_pointer > $8 = (void *) 0x0 Well, that seems to be the reason why it's failing to indirect through ind_pointer ... but why is it only failing on your machine and not everyone else's? I think this indicates something unp

Re: [HACKERS] strange buildfarm failures

2007-04-25 Thread Stefan Kaltenbrunner
Stefan Kaltenbrunner wrote: > two of my buildfarm members had different but pretty weird looking > failures lately: > > http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=quagga&dt=2007-04-25%2002:03:03 > > and > > http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=emu&dt=2007-04-24%2014:35:02 >

Re: [HACKERS] [BUGS] BUG #3245: PANIC: failed to re-find shared loc k o b j ect

2007-04-25 Thread Tom Lane
I wrote: > Heikki Linnakangas <[EMAIL PROTECTED]> writes: >> We could have two kinds of seq scans, with and without support for >> concurrent inserts. > Yeah, I considered that too, but it just seems too error-prone. We > could maybe make it trustworthy by having hash_seq_search complain if > it

Re: [HACKERS] ECPG failure on BF member Vaquita (Windows Vista)

2007-04-25 Thread Mark Wong
On 4/25/07, Tom Lane <[EMAIL PROTECTED]> wrote: Andrew Dunstan <[EMAIL PROTECTED]> writes: > I think you'll need to compile with optimisation turned off and then try > running the test under debugger control, putting a breakpoint in > ECPGget_variable() and then stepping through it. I wonder what

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-25 Thread Josh Berkus
Andreas, > Writing to a different area was considered in pg, but there were more > negative issues than positive. > So imho pg_compresslog is the correct path forward. The current > discussion is only about whether we want a more complex pg_compresslog > and no change to current WAL, or an increas

Re: [HACKERS] ECPG failure on BF member Vaquita (Windows Vista)

2007-04-25 Thread Tom Lane
Andrew Dunstan <[EMAIL PROTECTED]> writes: > I think you'll need to compile with optimisation turned off and then try > running the test under debugger control, putting a breakpoint in > ECPGget_variable() and then stepping through it. I wonder what value of > var->ind_pointer it is getting? Yo

Re: [HACKERS] Avoiding unnecessary reads in recovery

2007-04-25 Thread Tom Lane
"Simon Riggs" <[EMAIL PROTECTED]> writes: > As regards the zero_damaged_pages question, I raised that some time ago > but we didn't arrive at an explicit answer. All I would say is we can't > allow invalid pages in the buffer manager at any time, whatever options > we have requested, otherwise othe

Re: [HACKERS] ECPG failure on BF member Vaquita (Windows Vista)

2007-04-25 Thread Andrew Dunstan
Mark Wong wrote: On 4/25/07, Michael Meskes <[EMAIL PROTECTED]> wrote: I also saw that wombat is segfaulting in ecpg tests but not only with CVS HEAD but also trying to test 8.2. Any idea what's going on with this machine? I generated a stack trace for REL8_2_STABLE, but I'm not sure how helpf

Re: [HACKERS] ECPG failure on BF member Vaquita (Windows Vista)

2007-04-25 Thread Mark Wong
On 4/25/07, Michael Meskes <[EMAIL PROTECTED]> wrote: I also saw that wombat is segfaulting in ecpg tests but not only with CVS HEAD but also trying to test 8.2. Any idea what's going on with this machine? I generated a stack trace for REL8_2_STABLE, but I'm not sure how helpful it is. Let me

Re: [HACKERS] temporal variants of generate_series()

2007-04-25 Thread Neil Conway
On Thu, 2007-04-12 at 14:56 -0700, Andrew Hammond wrote: > I've written the following function definitions to extend > generate_series to support some temporal types (timestamptz, date and > time). Please include them if there's sufficient perceived need or > value. I could see these being useful,

Re: [HACKERS] Avoiding unnecessary reads in recovery

2007-04-25 Thread Simon Riggs
On Wed, 2007-04-25 at 13:48 +0100, Heikki Linnakangas wrote: > I was surprised how big a difference it makes, but when you think about > it it's logical. Without the patch, it's doing roughly the same I/O as > the test itself, reading in pages, modifying them, and writing them > back. With the

Re: [HACKERS] ECPG failure on BF member Vaquita (Windows Vista)

2007-04-25 Thread Andrew Dunstan
Dave Page wrote: Andrew Dunstan wrote: Please don't do that on your buildfarm repo copy (if that's what you did). You should not touch *anything* inside it. If need to you do this, make a copy (see later) and alter that. If you did do this to the buildfarm repo copy, please blow it away so t

Re: [HACKERS] ECPG failure on BF member Vaquita (Windows Vista)

2007-04-25 Thread Dave Page
Andrew Dunstan wrote: > Please don't do that on your buildfarm repo copy (if that's what you > did). You should not touch *anything* inside it. If need to you do this, > make a copy (see later) and alter that. > > If you did do this to the buildfarm repo copy, please blow it away so > that buildfa

Re: [HACKERS] Avoiding unnecessary reads in recovery

2007-04-25 Thread Heikki Linnakangas
Gregory Stark wrote: So in short I think with your patch this piece of code no longer has a role. Either your patch kicks in and we never even look at the damaged page at all, or we should be treating it as corrupt data and just check zero_damaged_pages alone and not do anything special in recove

Re: [HACKERS] ECPG failure on BF member Vaquita (Windows Vista)

2007-04-25 Thread Andrew Dunstan
Dave Page wrote: Michael Meskes wrote: On Wed, Apr 25, 2007 at 10:47:57AM +0100, Dave Page wrote: I'm seeing an ECPG-Check failure on Windows Vista - any ideas what might be causing this? Hmm, first glance suggests some permission problems. Yes, that was my thought as wel

Re: [HACKERS] Kill a Long Running Query

2007-04-25 Thread Heikki Linnakangas
Please don't cross-post to multiple mailing lists. And pgsql-hackers is not the correct list for basic usage questions. And long end-of-mail disclaimers are not generally appreciated. Mageshwaran wrote: Any body tell me how to kill a long running query in postgresql, is there any statement to

Re: [HACKERS] Kill a Long Running Query

2007-04-25 Thread Andrew Dunstan
Mageshwaran wrote: Hi , Any body tell me how to kill a long running query in postgresql, is there any statement to kill a query, and also tell me how to log slow queries to a log file. First. please do not cross-post like this. Pick the correct list and use it. Second, this query definite

Re: [HACKERS] [ADMIN] Kill a Long Running Query

2007-04-25 Thread Aaron Bono
On 4/25/07, Mageshwaran <[EMAIL PROTECTED]> wrote: Hi , Any body tell me how to kill a long running query in postgresql, is there any statement to kill a query, and also tell me how to log slow queries to a log file. Regards J Mageshwaran See if this helps: http://archives.postgresql.org/pg

[HACKERS] Kill a Long Running Query

2007-04-25 Thread Mageshwaran
Hi , Any body tell me how to kill a long running query in postgresql, is there any statement to kill a query, and also tell me how to log slow queries to a log file. Regards J Mageshwaran ** DISCLAIMER ** Information contained and transmitted by this E-MAIL is proprietary to

Re: [HACKERS] database size estimates

2007-04-25 Thread Heikki Linnakangas
Francois Deliege wrote: Hi, I am trying to estimate the size of a table composed of 51754000 rows. Each row has 31 attributes: 16 x bit(24) and 15 x bit(8) So, the payload should be: 51754000 x ( 16 x 24 + 15 x 24 ) bits = 3115 MB What data types are those exactly? If those 24-bit fields are

Re: [HACKERS] ECPG failure on BF member Vaquita (Windows Vista)

2007-04-25 Thread Dave Page
Michael Meskes wrote: > On Wed, Apr 25, 2007 at 10:47:57AM +0100, Dave Page wrote: >> I'm seeing an ECPG-Check failure on Windows Vista - any ideas what might >> be causing this? > > Hmm, first glance suggests some permission problems. Yes, that was my thought as well, however I ran cacls down t

[HACKERS] database size estimates

2007-04-25 Thread Francois Deliege
Hi, I am trying to estimate the size of a table composed of 51754000 rows. Each row has 31 attributes: 16 x bit(24) and 15 x bit(8) So, the payload should be: 51754000 x ( 16 x 24 + 15 x 24 ) bits = 3115 MB Now, from what I understand from postgresql manual is that the overhead is composed of

Re: [HACKERS] ECPG failure on BF member Vaquita (Windows Vista)

2007-04-25 Thread Michael Meskes
On Wed, Apr 25, 2007 at 10:47:57AM +0100, Dave Page wrote: > I'm seeing an ECPG-Check failure on Windows Vista - any ideas what might > be causing this? Hmm, first glance suggests some permission problems. I never touched a Vista system so far, so I'm at a loss as far as details are concerned.

Re: [HACKERS] Avoiding unnecessary reads in recovery

2007-04-25 Thread Gregory Stark
"Heikki Linnakangas" <[EMAIL PROTECTED]> writes: > While working on this, this comment in ReadBuffer caught my eye: > >> /* >> * During WAL recovery, the first access to any data page should >> * overwrite the whole page from the WAL; so a clobbered page >> * header is not r

Re: [HACKERS] Avoiding unnecessary reads in recovery

2007-04-25 Thread Heikki Linnakangas
Heikki Linnakangas wrote: While working on this, this comment in ReadBuffer caught my eye: /* * During WAL recovery, the first access to any data page should * overwrite the whole page from the WAL; so a clobbered page * header is not reason to fail. Hence, when InRecovery w

Re: [HACKERS] autovacuum does not start in HEAD

2007-04-25 Thread Alvaro Herrera
ITAGAKI Takahiro wrote: > I found that autovacuum launcher does not launch any workers in HEAD. > > AFAICS, we track the time to be vaccumed of each database in the following > way: > > 1. In rebuild_database_list(), we initialize avl_dbase->adl_next_worker >with (current_time + autovacuum_n

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-25 Thread Kenneth Marshall
On Wed, Apr 25, 2007 at 10:00:16AM +0200, Zeugswetter Andreas ADI SD wrote: > > > > 1) To deal with partial/inconsisitent write to the data file at > crash > > > recovery, we need full page writes at the first modification to > pages > > > after each checkpoint. It consumes much of WAL space. >

[HACKERS] Avoiding unnecessary reads in recovery

2007-04-25 Thread Heikki Linnakangas
In recovery, with full_pages_writes=on, we read in each page only to overwrite the contents with a full page image. That's a waste of time, and can have a surprisingly large effect on recovery time. As a quick test on my laptop, I initialized a DBT-2 test with 5 warehouses, and let it run for

Re: [HACKERS] Buildfarm: Stage logs not available for MSVC builds

2007-04-25 Thread Dave Page
Andrew Dunstan wrote: > The problem not beacuse of MSVC, but because of member misconfiguration, > by the look of it. The tar command string will need to be set in the > config file and tar installed. I found that I needed bsdtar for Windows > for this to work. See Ah, OK, thanks - there was a typ

[HACKERS] ECPG failure on BF member Vaquita (Windows Vista)

2007-04-25 Thread Dave Page
I'm seeing an ECPG-Check failure on Windows Vista - any ideas what might be causing this? http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=vaquita&dt=2007-04-24%2020:00:05 The only other Vista buildfarm member (baiji, on the same physical box) is running MSVC builds which don't yet test ECPG fro

[HACKERS] autovacuum does not start in HEAD

2007-04-25 Thread ITAGAKI Takahiro
I found that autovacuum launcher does not launch any workers in HEAD. AFAICS, we track the time to be vaccumed of each database in the following way: 1. In rebuild_database_list(), we initialize avl_dbase->adl_next_worker with (current_time + autovacuum_naptime / nDBs). 2. In do_start_worker()

[HACKERS] Buildfarm: Stage logs not available for MSVC builds

2007-04-25 Thread Dave Page
I just noticed that the stage logs aren't displayed against MSVC build hosts as they are for regular hosts, eg: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=mastodon&dt=2007-04-25%2001:00:02 vs. http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=narwhal&dt=2007-04-25%2002:00:03 Is this WIP,

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-25 Thread Zeugswetter Andreas ADI SD
> > 1) To deal with partial/inconsisitent write to the data file at crash > > recovery, we need full page writes at the first modification to pages > > after each checkpoint. It consumes much of WAL space. > > We need to find a way around this someday. Other DBs don't > do this; it may be be