Re: [PERFORM] strange pgbench results (as if blocked at the end)

2011-08-12 Thread Greg Smith
On 08/12/2011 07:37 PM, Tomas Vondra wrote: I've run nearly 200 of these, and in about 10 cases I got something that looks like this: http://www.fuzzy.cz/tmp/pgbench/tps.png http://www.fuzzy.cz/tmp/pgbench/latency.png i.e. it runs just fine for about 3:40 and then something goes wrong. The benc

Re: [PERFORM] strange pgbench results (as if blocked at the end)

2011-08-12 Thread Craig Ringer
On 13/08/2011 7:37 AM, Tomas Vondra wrote: I'd understand a slowdown, but why does it block? My first guess is that you're having checkpoint issues. Try enabling logging of checkpoints and checkpoint timings and see if anything there lines up with the pause you encounter. -- Craig Ringer

[PERFORM] How to see memory usage using explain analyze ?

2011-08-12 Thread hyelluas
Hello, I need to compare quiery execution : I have 2 tables partitioned by Datex ( daily): summary_daily ( counter | bigint datasource_id| integer application_id | integer action | character(1) srcreporter_id | integer destreporter_id | integer b

[PERFORM] strange pgbench results (as if blocked at the end)

2011-08-12 Thread Tomas Vondra
Hi, I've run a lot of pgbench tests recently (trying to compare various fs, block sizes etc.), and I've noticed several really strange results. Eeach benchmark consists of three simple steps: 1) set-up the database 2) read-only run (10 clients, 5 minutes) 3) read-write run (10 clients, 5 minutes

Re: [PERFORM] Recommended optimisations slows down PostgreSQL 8.4

2011-08-12 Thread Stephen Frost
* Waldo Nell (pwn...@telkomsa.net) wrote: > The fsync = off was because the production system runs on a uber expensive > SAN system with multipathing over Fibre Channel, it is on UPS and backup > generators in a secure datacenter, and we have daily backups we can fall back > to. So, two points:

Re: [PERFORM] Recommended optimisations slows down PostgreSQL 8.4

2011-08-12 Thread Tom Lane
Waldo Nell writes: > I have PostgreSQL 8.4.8 on Ubuntu Linux x64. Server is a Core i7 950 > with 6GB of RAM. 2GB of RAM us used by Java, some small amount by the > kernel / services and the rest is available to PostgreSQL. [ and the DB is 74GB, and things get slower when raising shared_buffers

Re: [PERFORM] Recommended optimisations slows down PostgreSQL 8.4

2011-08-12 Thread Waldo Nell
On 2011-08-12, at 10:10 , Kevin Grittner wrote: > Turning fsync off in production may be OK as long as those daily > backups aren't in the same building as the uber expensive SAN, and > it's really OK to fall back on a daily backup if the database server > crashes or locks up. By the way, I neve

Re: [PERFORM] Recommended optimisations slows down PostgreSQL 8.4

2011-08-12 Thread Greg Smith
On 08/12/2011 12:28 PM, Waldo Nell wrote: I guess that means the OS cache is better for this particular use case than the postgresql cache? There you go. It's not magic; the database cache has some properties that work very well for some workloads. And for others, you might as well let the

Re: [PERFORM] Recommended optimisations slows down PostgreSQL 8.4

2011-08-12 Thread Kevin Grittner
Waldo Nell wrote: > The fsync = off was because the production system runs on a uber > expensive SAN system with multipathing over Fibre Channel, it is > on UPS and backup generators in a secure datacenter, and we have > daily backups we can fall back to. Turning fsync off in production may be

Re: [PERFORM] Recommended optimisations slows down PostgreSQL 8.4

2011-08-12 Thread Waldo Nell
On 2011-08-12, at 09:32 , Merlin Moncure wrote: > In my opinion before looking at postgresql.conf you need to make sure > your queries and their plans are good. fire up pgfouine and see where > those 60 minutes are gettings spent. maybe you have a problem query > that demands optimization. Tha

Re: [PERFORM] Streaming replication performance

2011-08-12 Thread Merlin Moncure
On Thu, Aug 11, 2011 at 9:46 AM, Antonin Faltynek wrote: > Hi all, > > I'm testing Streamin replication with one hot standby node and I'm > experiencing high delay of hot standby node. > > When I reach aprox. 50 transactions per second where every transaction > includes only simple "UPDATE status

Re: [PERFORM] pgpool master or slave goes down java access error

2011-08-12 Thread Waldo Nell
On 2011-08-12, at 03:58 , jenopob wrote: > > I do some db falt tests > 1)the test code run always connect pgpool, > 2)test master or slave go down > > but when mster or slave go down ,java code throws exception : > > org.postgresql.util.PSQLException: An I

Re: [PERFORM] Recommended optimisations slows down PostgreSQL 8.4

2011-08-12 Thread Merlin Moncure
On Thu, Aug 11, 2011 at 7:27 PM, Waldo Nell wrote: > > On 2011-08-11, at 17:18 , k...@rice.edu wrote: > >> One guess is that you are using the defaults for other costing parameters >> and they >> do not accurately reflect your system. This means that it will be a crap >> shoot as >> to whether a

Re: [PERFORM] Recommended optimisations slows down PostgreSQL 8.4

2011-08-12 Thread Waldo Nell
On 2011-08-11, at 18:17 , Greg Smith wrote: > shared_buffers=512MB > wal_buffers=16MB > checkpoint_segments=64 Thanks for the advice. I tried these values... And it is even worse - went up to 63 minutes (from 60 minutes). Like I said this load is read mostly. My 80 / 20% might be a bit inac

Re: [PERFORM] poor pefrormance with regexp searches on large tables

2011-08-12 Thread Kevin Grittner
Grzegorz Blinowski wrote: > 2) changing long attribute storage to EXTERNAL gave 30% better > search time (but only on the first search - i.e. before data is > cached) That suggests that all of the following are true: (1) The long value was previously being compressed and stored in-line. (

Re: [PERFORM] poor pefrormance with regexp searches on large tables

2011-08-12 Thread Grzegorz Blinowski
To summarize this thread: We have tried most of the suggestions and found two of them effective: 1) collapsing OR expressions in the WHERE clause into one '(...)|(...)' regexp resulted in about 60% better search time 2) changing long attribute storage to EXTERNAL gave 30% better search time (but

[PERFORM] pgpool master or slave goes down java access error

2011-08-12 Thread jenopob
hi pgpool Expert my architecture as follows: Master/Slave with Streaming Replication and pgpool-II version of pgpool-II is pgpool-II.3.0.4 version of PostgreSQL is 9.0.2 I am using pgpool works as master/slave sub mode stream and pgpool key configuration is: