Re: [PERFORM] NOW vs CURRENT_DATE

2008-08-24 Thread Tino Wildenhain
Hi, dforum wrote: Hello every body, I just discover a big not only big huge difference between NOW() and CURRENT_DATE. Did you already know about it and do you know why ? DELETE FROM blacklist where bl_date < (NOW() - interval '2 DAY'); on 6 000 000 of records 699 ms DELETE FROM blacklist

Re: [PERFORM] Identifying the nature of blocking I/O

2008-08-24 Thread Alvaro Herrera
Tom Lane wrote: > "Scott Carey" <[EMAIL PROTECTED]> writes: > > DTrace is available now on MacOSX, Solaris 10, OpenSolaris, and FreeBSD. > > Linux however is still in the dark ages when it comes to system monitoring, > > especially with I/O. > > Oh, after poking around a bit, I should note that so

Re: [PERFORM] Large number of tables slow insert

2008-08-24 Thread Loic Petit
That's not a bad idea, at least for historical data. But actually one of the most common thing in sensor network monitoring is last readings monitoring. With indexes what I can do is : SELECT * FROM measures_xx ORDER BY timestamp DESC LIMIT 1 => And I got the very last reading in a blink (one page

Re: [PERFORM] Identifying the nature of blocking I/O

2008-08-24 Thread Tom Lane
"Scott Carey" <[EMAIL PROTECTED]> writes: > DTrace is available now on MacOSX, Solaris 10, OpenSolaris, and FreeBSD. > Linux however is still in the dark ages when it comes to system monitoring, > especially with I/O. Oh, after poking around a bit, I should note that some of my Red Hat compatriots

Re: [PERFORM] Identifying the nature of blocking I/O

2008-08-24 Thread Tom Lane
Craig Ringer <[EMAIL PROTECTED]> writes: > Peter Schuller wrote: >> But in general, it would be very interesting to see, at any given >> moment, what PostgreSQL backends are actually blocking on from the >> perspective of PostgreSQL. > The recent work on DTrace support for PostgreSQL will probably

Re: [PERFORM] Identifying the nature of blocking I/O

2008-08-24 Thread Scott Carey
More info/notes on DTrace -- DTrace is available now on MacOSX, Solaris 10, OpenSolaris, and FreeBSD. Linux however is still in the dark ages when it comes to system monitoring, especially with I/O. You can write some custom DTrace scripts to map any of the basic Postgres operations or processes

Re: [PERFORM] Large number of tables slow insert

2008-08-24 Thread Scott Carey
I don't know if the max_fsm_relations issue will solve your problem or not. I do know that you definitely want to increase it to a number larger than the sum of all your tables and indexes -- preferably with room to grow. Additionally the max_fsm_pages value will likely need to be increased as your

Re: [PERFORM] Identifying the nature of blocking I/O

2008-08-24 Thread Craig Ringer
Peter Schuller wrote: > But in general, it would be very interesting to see, at any given > moment, what PostgreSQL backends are actually blocking on from the > perspective of PostgreSQL. The recent work on DTrace support for PostgreSQL will probably give you the easiest path to useful results. Y

Re: [PERFORM] Large number of tables slow insert

2008-08-24 Thread Loic Petit
Quite a lot of answers ! > Does all INSERTs are made by one application? Maybe PREPARED STATEMENTS will help you? What about optimization on application level? Yes it's on only one application (through JDBC), optimizations (grouped transaction, prepared statement) will be done but that our very fi

Re: [PERFORM] Large number of tables slow insert

2008-08-24 Thread Peter Schuller
> I use Postgresql 8.3.1-1 to store a lot of data coming from a large amount > of sensors. In order to have good performances on querying by timestamp on > each sensor, I partitionned my measures table for each sensor. Thus I create > a lot of tables. > I simulated a large sensor network with 3000

Re: [PERFORM] NOW vs CURRENT_DATE

2008-08-24 Thread Peter Schuller
> I just discover a big not only big huge difference between NOW() and > CURRENT_DATE. > > Did you already know about it and do you know why ? > > DELETE FROM blacklist where bl_date < (NOW() - interval '2 DAY'); > on 6 000 000 of records > 699 ms > > DELETE FROM blacklist where bl_date < (CURR

Re: [PERFORM] Large number of tables slow insert

2008-08-24 Thread H. Hall
Loic Petit wrote: Hi, I use Postgresql 8.3.1-1 to store a lot of data coming from a large amount of sensors. In order to have good performances on querying by timestamp on each sensor, I partitionned my measures table for each sensor. Thus I create a lot of tables. I simulated a large sensor

Re: [PERFORM] Large number of tables slow insert

2008-08-24 Thread Scott Carey
Just a guess, but have you tried increasing max_fsm_relations ? This probably shouldn't matter but you'll want this to be larger than the sum of all your tables and indexes and it doesn't take that much memory to increase it. My next suggestion would be to log in as the superuser and 'vacuum anal

[PERFORM] NOW vs CURRENT_DATE

2008-08-24 Thread dforum
Hello every body, I just discover a big not only big huge difference between NOW() and CURRENT_DATE. Did you already know about it and do you know why ? DELETE FROM blacklist where bl_date < (NOW() - interval '2 DAY'); on 6 000 000 of records 699 ms DELETE FROM blacklist where bl_date < (CUR

Re: [PERFORM] Large number of tables slow insert

2008-08-24 Thread DiezelMax
Does all INSERTs are made by one application? Maybe PREPARED STATEMENTS will help you? What about optimization on application level? -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-perfor

Re: [PERFORM] Large number of tables slow insert

2008-08-24 Thread dforum
hello to all, I've a question regarding the folowing comments. How to estimate vacuum aggressiveness ? It's for me very deficulte to setup the autovaccum setting correctly. It seems for me that it is not enough aggressive, but when I change the settings the autovacuum process is almost always