Re: [PERFORM] Periodically slow inserts

2010-10-22 Thread Gael Le Mignot
Hello, Thanks to everyone who gave me hints and feedbacks. I managed to solve the problem. My understanding of what was happening is the following : - The gin index (as explained on [1]), stores temporary list, and when they grow big enough, those are dispatched into the real index. Vacuum

Re: [PERFORM] Periodically slow inserts

2010-10-22 Thread Leonardo Francalanci
>>> I think this may be caused by GIN's habit of queuing index insertions >>> until it's accumulated a reasonable-size batch: So the fact that it takes 21s to sync 365 buffers in this case is normal? -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make chan

Re: [PERFORM] Periodically slow inserts

2010-10-21 Thread Tom Lane
Gael Le Mignot writes: > Thu, 21 Oct 2010 10:55:48 -0400, you wrote: >>> I think this may be caused by GIN's habit of queuing index insertions >>> until it's accumulated a reasonable-size batch: >>> http://www.postgresql.org/docs/9.0/static/gin-implementation.html#GIN-FAST-UPDATE > So a big work

Re: [PERFORM] Periodically slow inserts

2010-10-21 Thread Mladen Gogala
Gael Le Mignot wrote: Hello, We are using PostgreSQL for storing data and full-text search indexes for the webiste of a daily newspaper. We are very happy overall with the results, but we have one "weird" behaviour that we would like to solve. The problem is when we index objects into the

Re: [PERFORM] Periodically slow inserts

2010-10-21 Thread Gael Le Mignot
Hello Tom! Thu, 21 Oct 2010 10:55:48 -0400, you wrote: > Gael Le Mignot writes: >> The problem is when we index objects into the full-text search part of >> the database (which a DELETE and then an INSERT into a specific table), >> the INSERT sometimes take a long time (from 10s to 20s

Re: [PERFORM] Periodically slow inserts

2010-10-21 Thread Greg Smith
Gael Le Mignot wrote: The delay is the delay of the "sync" part of the checkpoints : 2010-10-21 16:39:15 CEST LOG: checkpoint complete: wrote 365 buffers (11.9%); 0 transaction log file(s) added, 0 removed, 3 recycled; write=0.403 s, sync=21.312 s, total=21.829 s Maybe there is something I m

Re: [PERFORM] Periodically slow inserts

2010-10-21 Thread Cédric Villemain
2010/10/21 Leonardo Francalanci : >> 2010-10-21 16:39:15 CEST  LOG:  checkpoint complete: wrote 365 buffers >> (11.9%); 0 transaction log  file(s) added, 0 removed, 3 recycled; >> write=0.403 s, sync=21.312 s,  total=21.829 s > > > I'm no expert, but isn't 21s to sync 365 buffers a big amount of ti

Re: [PERFORM] Periodically slow inserts

2010-10-21 Thread Leonardo Francalanci
> 2010-10-21 16:39:15 CEST LOG: checkpoint complete: wrote 365 buffers > (11.9%); 0 transaction log file(s) added, 0 removed, 3 recycled; > write=0.403 s, sync=21.312 s, total=21.829 s I'm no expert, but isn't 21s to sync 365 buffers a big amount of time? -- Sent via pgsql-performance m

Re: [PERFORM] Periodically slow inserts

2010-10-21 Thread Leonardo Francalanci
> > does it use a BBU?) Sorry, this was supposed to read "do you have cache on the controller", of course a battery can't change the performance... but you got it anyway... -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: h

Re: [PERFORM] Periodically slow inserts

2010-10-21 Thread Gael Le Mignot
Hello Leonardo! Thu, 21 Oct 2010 14:15:40 +0100 (BST), you wrote: >> We are using PostgreSQL for storing data and full-text search indexes >> for the webiste of a daily newspaper. We are very happy overall with the >> results, but we have one "weird" behaviour that we would like to solv

Re: [PERFORM] Periodically slow inserts

2010-10-21 Thread Tom Lane
Gael Le Mignot writes: > The problem is when we index objects into the full-text search part of > the database (which a DELETE and then an INSERT into a specific table), > the INSERT sometimes take a long time (from 10s to 20s), but the same > insert (and many other similar ones) are fast (b

Re: [PERFORM] Periodically slow inserts

2010-10-21 Thread Leonardo Francalanci
> We are using PostgreSQL for storing data and full-text search indexes > for the webiste of a daily newspaper. We are very happy overall with the > results, but we have one "weird" behaviour that we would like to solve. I think there's a lot of missing info worth knowing: 1) checkpoints

Re: [PERFORM] Periodically slow inserts

2010-10-21 Thread Kenneth Marshall
Hi, There are a lot of details missing about your system: http://wiki.postgresql.org/wiki/SlowQueryQuestions Cheers, Ken On Thu, Oct 21, 2010 at 02:25:44PM +0200, Gael Le Mignot wrote: > > Hello, > > We are using PostgreSQL for storing data and full-text search indexes > for the webiste of

[PERFORM] Periodically slow inserts

2010-10-21 Thread Gael Le Mignot
Hello, We are using PostgreSQL for storing data and full-text search indexes for the webiste of a daily newspaper. We are very happy overall with the results, but we have one "weird" behaviour that we would like to solve. The problem is when we index objects into the full-text search part o