Re: [HACKERS] syslogging oddity

2007-07-22 Thread Tom Lane
Andrew Dunstan <[EMAIL PROTECTED]> writes: > Somewhere along the way we seem to have made the syslogger's shutdown > message go to stderr, even if we have redirected it: I'm pretty sure it has done that all along; at least the design intention is that messages generated by syslogger itself should

[HACKERS] syslogging oddity

2007-07-22 Thread Andrew Dunstan
Somewhere along the way we seem to have made the syslogger's shutdown message go to stderr, even if we have redirected it: [EMAIL PROTECTED] inst.test.5703]$ bin/pg_ctl -D data/ -w stop waiting for server to shut downLOG: logger shutting down done server stopped Not sure if this is somet

Re: [HACKERS] Full page images in WAL & Cache Invalidation

2007-07-22 Thread Gregory Stark
"Tom Lane" <[EMAIL PROTECTED]> writes: > 2. WAL-reading process receives and executes DROP TABLE X. > > (It doesn't even have to be a DROP; most varieties of ALTER are enough > to create problems for a concurrently-running query.) > > It's really hard to see how to defend against that without a f

Re: [HACKERS] Full page images in WAL & Cache Invalidation

2007-07-22 Thread Simon Riggs
On Sun, 2007-07-22 at 17:54 +0200, Florian G. Pflug wrote: > 1) It seems that the btree code sends out relcache > invalidation msgs during normal operation > (No DDL statements are executed). This lets any > simple flush-all-caches-if-ddl-was-execute scheme > fail. Cache invalidat

Re: [HACKERS] Full page images in WAL & Cache Invalidation

2007-07-22 Thread Simon Riggs
On Sun, 2007-07-22 at 19:58 +0200, Florian G. Pflug wrote: > Tom Lane wrote: > > "Florian G. Pflug" <[EMAIL PROTECTED]> writes: > >> I'm currently working on correctly flushing the > >> catalog/relation/sgmr caches on a readonly PITR > >> slave during recovery. > > > > I don't believe there is any

Re: [HACKERS] Full page images in WAL & Cache Invalidation

2007-07-22 Thread Tom Lane
"Florian G. Pflug" <[EMAIL PROTECTED]> writes: >> Anyway, if you believe that DDL is infrequent, why are you resistant >> to the idea of WAL-logging cache flushes? > First, cache invalidations are not the only problem caused by replaying > system-table updates. The whole SnapshotNow > business do

Re: [HACKERS] Full page images in WAL & Cache Invalidation

2007-07-22 Thread Tom Lane
"Florian G. Pflug" <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> The real problem with the scheme you propose is that it turns a >> cache flush on one table into a system-wide cache flush. > Yes.. It really builds on the idea that those flushes happen not > too frequently. The problem is that

Re: [HACKERS] Full page images in WAL & Cache Invalidation

2007-07-22 Thread Florian G. Pflug
Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: My basic assumption is that DDL is something quite uncommon on a production system. I'm not sure I believe that, because of temp tables. There's also the problem that plain VACUUM (or ANALYZE) causes a relcache flush to update the

Re: [HACKERS] 8.2 is 30% better in pgbench than 8.3

2007-07-22 Thread Greg Smith
On Sun, 22 Jul 2007, Tom Lane wrote: There seems to be also some of the "good average but bad worst case" behavior that Josh and others have pointed out in bigger benchmarks. I've always assumed this was due to checkpointing At lot of variation is from checkpoints, some comes from the index f

Re: [HACKERS] Full page images in WAL & Cache Invalidation

2007-07-22 Thread Tom Lane
"Florian G. Pflug" <[EMAIL PROTECTED]> writes: > My basic assumption is that DDL is something quite uncommon on a > production system. I'm not sure I believe that, because of temp tables. There's also the problem that plain VACUUM (or ANALYZE) causes a relcache flush to update the relation-size s

Re: [HACKERS] Full page images in WAL & Cache Invalidation

2007-07-22 Thread Florian G. Pflug
Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: Leaving aside the btree issues, are you worried about performance problems, or can aggressive cache flushing hurt correctness? It shouldn't hurt correctness, but I don't think you entirely grasp the magnitude of the performance hit

Re: [HACKERS] Full page images in WAL & Cache Invalidation

2007-07-22 Thread Tom Lane
"Florian G. Pflug" <[EMAIL PROTECTED]> writes: > Leaving aside the btree issues, are you worried about performance > problems, or can aggressive cache flushing hurt correctness? It shouldn't hurt correctness, but I don't think you entirely grasp the magnitude of the performance hit you'll take. T

Re: [HACKERS] Full page images in WAL & Cache Invalidation

2007-07-22 Thread Florian G. Pflug
Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: I'm currently working on correctly flushing the catalog/relation/sgmr caches on a readonly PITR slave during recovery. I don't believe there is any workable solution to that short of logging cache-flush operations in WAL. I still

Re: [HACKERS] 8.2 is 30% better in pgbench than 8.3

2007-07-22 Thread Tom Lane
Greg Smith <[EMAIL PROTECTED]> writes: > I'd want a set of 3 at each configuration because even with longer runs, > you occasionally get really odd results. Until you have 3 it can be > unclear which is the weird one. Yeah, pgbench results are notoriously unrepeatable. One issue is that the fi

Re: [HACKERS] Full page images in WAL & Cache Invalidation

2007-07-22 Thread Tom Lane
"Florian G. Pflug" <[EMAIL PROTECTED]> writes: > I'm currently working on correctly flushing the > catalog/relation/sgmr caches on a readonly PITR > slave during recovery. I don't believe there is any workable solution to that short of logging cache-flush operations in WAL.

Re: [HACKERS] Full page images in WAL & Cache Invalidation

2007-07-22 Thread Tom Lane
"Florian G. Pflug" <[EMAIL PROTECTED]> writes: > I'm currently working on correctly flushing the > catalog/relation/sgmr caches on a readonly PITR > slave during recovery. I don't believe there is any workable solution to that short of logging cache-flush operations in WAL.

[HACKERS] Full page images in WAL & Cache Invalidation

2007-07-22 Thread Florian G. Pflug
Hi I'm currently working on correctly flushing the catalog/relation/sgmr caches on a readonly PITR slave during recovery. These are the things that currently cause me headache. 1) It seems that the btree code sends out relcache invalidation msgs during normal operation (No DDL statements a

Re: [HACKERS] 8.2 is 30% better in pgbench than 8.3

2007-07-22 Thread Josh Berkus
Tom, Note to all: we ***HAVE TO*** settle on some reasonable default vacuum_cost_delay settings before we can ship 8.3. With no cost delay and two or three workers active, 8.3's autovac does indeed send performance into the tank. I've been using 20ms for most of my setups. That's aimed at re

Re: [HACKERS] 8.2 is 30% better in pgbench than 8.3

2007-07-22 Thread Josh Berkus
Pavel Stehule wrote: Hello, I checked my tests again I have different results. Now I tested PostgreSQL on dedicated server. Now 8.3 is about 20% faster. I didn't see strong impression of autovacuum. All numbers are approximate only. I did pgbench 3x for folowing configuration: (autovacuum on,

Re: [HACKERS] 8.2 is 30% better in pgbench than 8.3

2007-07-22 Thread Pavel Stehule
Hello, I checked my tests again I have different results. Now I tested PostgreSQL on dedicated server. Now 8.3 is about 20% faster. I didn't see strong impression of autovacuum. All numbers are approximate only. I did pgbench 3x for folowing configuration: (autovacuum on, autovacuum off, statis