Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2014-01-18 Thread Tom Lane
Amit Kapila writes: > On Sun, Jan 19, 2014 at 5:59 AM, Tom Lane wrote: >> It strikes me that there may be an obvious way to improve the number >> further, based on the observation in this thread that nkeep doesn't need >> to be scaled up because VACUUM should have scanned every page that could >>

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2014-01-18 Thread Amit Kapila
On Sun, Jan 19, 2014 at 5:59 AM, Tom Lane wrote: > Amit Kapila writes: >> I am marking this (based on patch vacuum_fix_v7_nkeep.patch) as Ready >> For Committer. > > I've reviewed and committed this patch, with one significant change. > If you look at the way that vacuumlazy.c computes new_rel_tu

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2014-01-18 Thread Tom Lane
Amit Kapila writes: > I am marking this (based on patch vacuum_fix_v7_nkeep.patch) as Ready > For Committer. I've reviewed and committed this patch, with one significant change. If you look at the way that vacuumlazy.c computes new_rel_tuples, it's based on scanned_tuples (lazy_scan_heap's num_tu

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2014-01-16 Thread Amit Kapila
On Thu, Jan 16, 2014 at 10:24 PM, tirtho wrote: > Is this now fixed? If so, where do I find the patch for postgres 9.2.2. This is still not fixed, the patch is in Ready For Committer stage. You can confirm the status here: https://commitfest.postgresql.org/action/patch_view?id=1258

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2014-01-16 Thread tirtho
Is this now fixed? If so, where do I find the patch for postgres 9.2.2. Thanks - Tirthankar -- View this message in context: http://postgresql.1045698.n5.nabble.com/Heavily-modified-big-table-bloat-even-in-auto-vacuum-is-running-tp5774274p5787477.html Sent from the PostgreSQL - hackers mailin

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-12-16 Thread Amit Kapila
On Thu, Dec 12, 2013 at 12:24 PM, Haribabu kommi wrote: > On 06 December 2013 11:57 Amit Kapila wrote: > > A simplified test and updated patch by taking care the above comment are > attached in the mail. > I am not able to reduce the test duration but changed as the test > automatically exists a

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-12-16 Thread Amit Kapila
On Mon, Dec 16, 2013 at 6:51 PM, Peter Eisentraut wrote: > Amit, if you think this patch is ready now, please mark it as Ready for > Committer. Otherwise, I encourage your to continue the discussion and > possibly resubmit the patch to the next commitfest. Only one test is pending from myside to

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-12-16 Thread Peter Eisentraut
Amit, if you think this patch is ready now, please mark it as Ready for Committer. Otherwise, I encourage your to continue the discussion and possibly resubmit the patch to the next commitfest. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscri

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-12-11 Thread Haribabu kommi
On 06 December 2013 11:57 Amit Kapila wrote: > On Fri, Nov 29, 2013 at 6:55 PM, Haribabu kommi > wrote: > > On 29 November 2013 12:00 Amit Kapila wrote: > >> On Tue, Nov 26, 2013 at 7:26 PM, Haribabu kommi > >> wrote: > >> Few questions about your latest patch: > >> a. Is there any reason why you

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-12-05 Thread Amit Kapila
On Fri, Nov 29, 2013 at 6:55 PM, Haribabu kommi wrote: > On 29 November 2013 12:00 Amit Kapila wrote: >> On Tue, Nov 26, 2013 at 7:26 PM, Haribabu kommi >> wrote: >> Few questions about your latest patch: >> a. Is there any reason why you are doing estimation of dead tuples only >> for Autovacuum

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-11-29 Thread Haribabu kommi
On 29 November 2013 12:00 Amit Kapila wrote: > On Tue, Nov 26, 2013 at 7:26 PM, Haribabu kommi > wrote: > > On 25 November 2013 10:43 Amit Kapila wrote: > >> On Fri, Nov 22, 2013 at 12:12 PM, Haribabu kommi > >> wrote: > >> > On 19 November 2013 10:33 Amit Kapila wrote: > >> >> If I understood c

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-11-28 Thread Amit Kapila
On Tue, Nov 26, 2013 at 7:26 PM, Haribabu kommi wrote: > On 25 November 2013 10:43 Amit Kapila wrote: >> On Fri, Nov 22, 2013 at 12:12 PM, Haribabu kommi >> wrote: >> > On 19 November 2013 10:33 Amit Kapila wrote: >> >> If I understood correctly, then your patch's main intention is to >> >> corre

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-11-26 Thread Haribabu kommi
On 25 November 2013 10:43 Amit Kapila wrote: > On Fri, Nov 22, 2013 at 12:12 PM, Haribabu kommi > wrote: > > On 19 November 2013 10:33 Amit Kapila wrote: > >> If I understood correctly, then your patch's main intention is to > >> correct the estimate of dead tuples, so that it can lead to Vacuum >

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-11-24 Thread Amit Kapila
On Fri, Nov 22, 2013 at 12:12 PM, Haribabu kommi wrote: > On 19 November 2013 10:33 Amit Kapila wrote: >> If I understood correctly, then your patch's main intention is to >> correct the estimate of dead tuples, so that it can lead to Vacuum >> cleaning the table/index which otherwise is not happe

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-11-21 Thread Haribabu kommi
On 19 November 2013 10:33 Amit Kapila wrote: > If I understood correctly, then your patch's main intention is to > correct the estimate of dead tuples, so that it can lead to Vacuum > cleaning the table/index which otherwise is not happening as per > configuration value (autovacuum_vacuum_threshold

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-11-18 Thread Amit Kapila
On Fri, Nov 15, 2013 at 10:52 AM, Haribabu kommi wrote: > On 15 November 2013 10:00 Amit Kapila wrote: >> On Wed, Nov 13, 2013 at 12:02 PM, Haribabu kommi >> wrote: >> > >> > With the simulated bloat test run for 1 hour the bloat occurred as >> > below, >> > >> > Unpatched - 1532MB >> > Patched

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-11-14 Thread Haribabu kommi
On 15 November 2013 10:00 Amit Kapila wrote: > On Wed, Nov 13, 2013 at 12:02 PM, Haribabu kommi > wrote: > > On 12 November 2013 08:47 Amit Kapila wrote: > >> On Mon, Nov 11, 2013 at 3:14 PM, Haribabu kommi > >> wrote: > >> > On 08 November 2013 18:35 Amit Kapila wrote: > >> >> On Fri, Nov 8, 201

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-11-14 Thread Amit Kapila
On Wed, Nov 13, 2013 at 12:02 PM, Haribabu kommi wrote: > On 12 November 2013 08:47 Amit Kapila wrote: >> On Mon, Nov 11, 2013 at 3:14 PM, Haribabu kommi >> wrote: >> > On 08 November 2013 18:35 Amit Kapila wrote: >> >> On Fri, Nov 8, 2013 at 10:56 AM, Haribabu kommi >> >> wrote: >> >> > On 07 N

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-11-12 Thread Haribabu kommi
On 12 November 2013 08:47 Amit Kapila wrote: > On Mon, Nov 11, 2013 at 3:14 PM, Haribabu kommi > wrote: > > On 08 November 2013 18:35 Amit Kapila wrote: > >> On Fri, Nov 8, 2013 at 10:56 AM, Haribabu kommi > >> wrote: > >> > On 07 November 2013 09:42 Amit Kapila wrote: > >> > 1. Taking a copy of

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-11-11 Thread Amit Kapila
On Mon, Nov 11, 2013 at 3:14 PM, Haribabu kommi wrote: > On 08 November 2013 18:35 Amit Kapila wrote: >> On Fri, Nov 8, 2013 at 10:56 AM, Haribabu kommi >> wrote: >> > On 07 November 2013 09:42 Amit Kapila wrote: >> >> I am not sure whether the same calculation as done for >> new_rel_tuples >> >>

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-11-11 Thread Haribabu kommi
On 08 November 2013 18:35 Amit Kapila wrote: > On Fri, Nov 8, 2013 at 10:56 AM, Haribabu kommi > wrote: > > On 07 November 2013 09:42 Amit Kapila wrote: > >> I am not sure whether the same calculation as done for > new_rel_tuples > >> works for new_dead_tuples, you can once check it. > > > > I did

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-11-08 Thread Amit Kapila
On Fri, Nov 8, 2013 at 10:56 AM, Haribabu kommi wrote: > On 07 November 2013 09:42 Amit Kapila wrote: >> I am not sure whether the same calculation as done for new_rel_tuples >> works for new_dead_tuples, you can once check it. > > I didn't find any way to calculate new_dead_tuples like new_rel_tu

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-11-07 Thread Haribabu kommi
On 07 November 2013 09:42 Amit Kapila wrote: > On Tue, Oct 22, 2013 at 2:09 PM, Haribabu kommi > wrote: > > On 22 October 2013 10:15 Amit Kapila wrote: > >>>On Mon, Oct 21, 2013 at 10:54 AM, Haribabu kommi > wrote: > >>> > >>Actually what I had in mind was to use nkeep to estimate > n_dead_tuples

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-11-06 Thread Amit Kapila
On Tue, Oct 22, 2013 at 2:09 PM, Haribabu kommi wrote: > On 22 October 2013 10:15 Amit Kapila wrote: >>>On Mon, Oct 21, 2013 at 10:54 AM, Haribabu kommi >>>wrote: >>> >>Actually what I had in mind was to use nkeep to estimate n_dead_tuples >>similar to how num_tuples is used to estimate n_live_

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-10-22 Thread Haribabu kommi
On 22 October 2013 10:15 Amit Kapila wrote: >>On Mon, Oct 21, 2013 at 10:54 AM, Haribabu kommi >>wrote: >> >> Yes, it's correct. "nkeep" counter have the dead tuples which are recently >> dead and are not vacuumed. The removal of tuples vacuumed from dead tuples >> should be the same as "nkeep"

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-10-21 Thread Amit Kapila
On Mon, Oct 21, 2013 at 10:54 AM, Haribabu kommi wrote: > On 20 October 2013 12:06 Amit Kapila wrote: >>On Tue, Oct 15, 2013 at 3:37 PM, Haribabu kommi >>wrote: >>> On 12 October 2013 11:30 Tom Lane wrote: Haribabu kommi writes: >>> Another way to look at it is that we want to keep any

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-10-20 Thread Haribabu kommi
On 20 October 2013 12:06 Amit Kapila wrote: >On Tue, Oct 15, 2013 at 3:37 PM, Haribabu kommi >wrote: >> On 12 October 2013 11:30 Tom Lane wrote: >>>Haribabu kommi writes: To handle the above case instead of directly resetting the dead tuples as zero, how if the exact dead tuples are r

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-10-19 Thread Amit Kapila
On Tue, Oct 15, 2013 at 3:37 PM, Haribabu kommi wrote: > On 12 October 2013 11:30 Tom Lane wrote: >>Haribabu kommi writes: >>> To handle the above case instead of directly resetting the dead tuples >>> as zero, how if the exact dead tuples are removed from the table stats. >>> With this approach

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-10-15 Thread Haribabu kommi
On 12 October 2013 11:30 Tom Lane wrote: >Haribabu kommi writes: >> To handle the above case instead of directly resetting the dead tuples >> as zero, how if the exact dead tuples are removed from the table stats. With >> this approach vacuum gets triggered frequently thus it reduces the bloat.

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-10-11 Thread Tom Lane
Haribabu kommi writes: > To handle the above case instead of directly resetting the dead tuples as > zero, how if the exact dead tuples > are removed from the table stats. With this approach vacuum gets triggered > frequently thus it reduces the bloat. This does not seem like a very good idea a

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-10-11 Thread Kevin Grittner
Haribabu kommi wrote: > To handle the above case instead of directly resetting the dead > tuples as zero, how if the exact dead tuples are removed from the > table stats. With this approach vacuum gets triggered frequently > thus it reduces the bloat. > Patch for the same is attached in the mail.

[HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-10-11 Thread Haribabu kommi
vacuum is not happening on a heavily modified big table even if the dead tuples are more than configured threshold. This is because during at the end of vacuum, the number of dead tuples of the table is reset as zero, because of this reason the dead tuples which are occurred during the vacuum op