Re: [GENERAL] Vacuuming strategy

2014-05-02 Thread Jeff Janes
On Wed, Apr 30, 2014 at 10:49 AM, Elanchezhiyan Elango wrote: > >> Each table has 4 indices. The updates are to the indexed columns. >> >> Here is the schema of a table: http://pastebin.com/ffu0dUjS All tables >> have this same schema except that some tables don't have a port column and >> so wil

Re: [GENERAL] Vacuuming strategy

2014-05-02 Thread Jeff Janes
On Wed, Apr 30, 2014 at 10:45 AM, Elanchezhiyan Elango wrote: > Missed to answer this one: > >> Is the 5GB for the table plus indexes, or just the table itself? > > No it's not including the the indices. Including indices it's actually > 17GB!! > Is it one particular index that is bloated? Che

Re: [GENERAL] Vacuuming strategy

2014-05-02 Thread Jeff Janes
On Wed, Apr 30, 2014 at 10:40 AM, Elanchezhiyan Elango wrote: > > > Why do you have a 4 minute timeout? That seems counter-productive. > > Oh, Is it less or more? > I would not have timeouts on maintenance operations at all. To me a statement timeout is a last ditch method to deal with a rec

Re: [GENERAL] Vacuuming strategy

2014-05-01 Thread Venkata Balaji Nagothi
On Wed, Apr 30, 2014 at 9:59 AM, Elanchezhiyan Elango wrote: > Hi, > > I need help on deciding my vacuuming strategy. I need to know if I ever > need to do 'vacuum full' for my tables. > > Important and critical configuration is "fillfactor". "fillfactor" will have a greater impact on VACUUMING s

Re: [GENERAL] Vacuuming strategy

2014-05-01 Thread Francisco Olarte
Hi: On Wed, Apr 30, 2014 at 7:40 PM, Elanchezhiyan Elango wrote: > Francisco, > Thanks for the partitioning idea. I used to have the tables partitioned. But > now that I have moved to a schema where data is split across about ~90 > tables I have moved away from partitioning. But it's something I

Re: [GENERAL] Vacuuming strategy

2014-04-30 Thread Sergey Konoplev
On Wed, Apr 30, 2014 at 10:40 AM, Elanchezhiyan Elango wrote: > log_autovacuum_min_duration = 0 > > autovacuum_vacuum_scale_factor = 0 > autovacuum_vacuum_threshold = 4 > autovacuum_analyze_scale_factor = 0 > autovacuum_analyze_threshold = 4 I don't think it is a good idea to set scale fa

Re: [GENERAL] Vacuuming strategy

2014-04-30 Thread Elanchezhiyan Elango
> > > Each table has 4 indices. The updates are to the indexed columns. > > Here is the schema of a table: http://pastebin.com/ffu0dUjS All tables > have this same schema except that some tables don't have a port column and > so will have one less index What indexes exist? Are the updates to inde

Re: [GENERAL] Vacuuming strategy

2014-04-30 Thread Elanchezhiyan Elango
Missed to answer this one: > > Is the 5GB for the table plus indexes, or just the table itself? No it's not including the the indices. Including indices it's actually 17GB!! On Wed, Apr 30, 2014 at 10:40 AM, Elanchezhiyan Elango wrote: > Sergey, > Thanks for the aggressive settings. I have li

Re: [GENERAL] Vacuuming strategy

2014-04-30 Thread Elanchezhiyan Elango
Sergey, Thanks for the aggressive settings. I have listed some settings I am planning to try below. Please review and let me know your feedback. Francisco, Thanks for the partitioning idea. I used to have the tables partitioned. But now that I have moved to a schema where data is split across abou

Re: [GENERAL] Vacuuming strategy

2014-04-30 Thread Jeff Janes
On Tue, Apr 29, 2014 at 4:59 PM, Elanchezhiyan Elango wrote: > Hi, > > I need help on deciding my vacuuming strategy. I need to know if I ever > need to do 'vacuum full' for my tables. > > Tables1: Following is the query patterns on 4 high traffic table in my > database: > 1. Every 5 minutes about

Re: [GENERAL] Vacuuming strategy

2014-04-30 Thread Francisco Olarte
Hi: On Wed, Apr 30, 2014 at 1:59 AM, Elanchezhiyan Elango wrote: > I need help on deciding my vacuuming strategy. I need to know if I ever need > to do 'vacuum full' for my tables. > > Tables1: Following is the query patterns on 4 high traffic table in my > database: > 1. Every 5 minutes about 50

Re: [GENERAL] Vacuuming strategy

2014-04-29 Thread Sergey Konoplev
On Tue, Apr 29, 2014 at 4:59 PM, Elanchezhiyan Elango wrote: [...] > With the above query pattern with intensive updates and deletes, I need to > do some aggressive vacuuming. > > Current strategy:I am running with default autovacuum settings (postgres > 9.1.9) and I tried doing a 'vacuum full' fo

[GENERAL] Vacuuming strategy

2014-04-29 Thread Elanchezhiyan Elango
Hi, I need help on deciding my vacuuming strategy. I need to know if I ever need to do 'vacuum full' for my tables. Tables1: Following is the query patterns on 4 high traffic table in my database: 1. Every 5 minutes about 5 rows in the table are updated. And for a given clock hour the same 50