RE: how to slow down parts of Pg

2020-04-23 Thread Kevin Brannen
>From: Ron > >What you need is async replication instead of synchronous replication. The only way I can think of to do that in our present situation would be to buy DRBD-Proxy, which becomes a single-point-of-failure and goes against the idea of HA (it seems like a good product for disaster reco

RE: how to slow down parts of Pg

2020-04-23 Thread Kevin Brannen
>From: Peter J. Holzer >On 2020-04-21 21:16:57 +, Kevin Brannen wrote: >> From: Michael Loftis >> > drbdsetup allows you to control the sync rates. >> >> I was hoping not to have to do that, but the more I think about this >> I'm realizing that it won't hurt because the network cap is >> effe

Re: how to slow down parts of Pg

2020-04-22 Thread Peter J. Holzer
On 2020-04-21 21:16:57 +, Kevin Brannen wrote: > From: Michael Loftis > > drbdsetup allows you to control the sync rates. > > I was hoping not to have to do that, but the more I think about this I'm > realizing that it won't hurt because the network cap is effectively limiting > me > anyway.

Re: how to slow down parts of Pg

2020-04-22 Thread Ron
What you need is async replication instead of synchronous replication. On 4/21/20 3:30 PM, Kevin Brannen wrote: I have an unusual need:  I need Pg to slow down. I know, we all want our DB to go faster, but in this case it's speed is working against me in 1 area. We have systems that are geo-

Re: how to slow down parts of Pg

2020-04-22 Thread Adrian Klaver
On 4/21/20 7:43 PM, Virendra Kumar wrote: Hi Adrian, Here is test case, basically when autovacuum runs it did release the space to disk since it had may be continuous blocks which can be released to disk but the space used by index is still being held until I ran the reindex on the table (I a

RE: how to slow down parts of Pg

2020-04-22 Thread Kevin Brannen
>From: Laurenz Albe > >>On Tue, 2020-04-21 at 20:30 +, Kevin Brannen wrote: >> I have an unusual need: I need Pg to slow down. I know, we all want >> our DB to go faster, but in this case it's speed is working against me in 1 >> area. >> >> [...] The part that hurts so bad is when we do main

Re: how to slow down parts of Pg

2020-04-22 Thread Laurenz Albe
On Tue, 2020-04-21 at 20:30 +, Kevin Brannen wrote: > I have an unusual need: I need Pg to slow down. I know, we all want our DB > to go faster, > but in this case it's speed is working against me in 1 area. > > We have systems that are geo-redundant for HA, with the redundancy being > hand

Re: how to slow down parts of Pg

2020-04-21 Thread Virendra Kumar
Hi Adrian, Here is test case, basically when autovacuum runs it did release the space to disk since it had may be continuous blocks which can be released to disk but the space used by index is still being held until I ran the reindex on the table (I assume reindex for index would work as well).

Re: how to slow down parts of Pg

2020-04-21 Thread Adrian Klaver
On 4/21/20 2:32 PM, Virendra Kumar wrote: Autovacuum does takes care of dead tuples and return space to table's allocated size and can be re-used by fresh incoming rows or any updates. Index bloat is still not being taken care of by autovacuum process. You should use pg_repack to do index rebu

Re: how to slow down parts of Pg

2020-04-21 Thread Michael Lewis
Reviewing pg_stat_user_tables will give you an idea of how often autovacuum is cleaning up those tables that "need" that vacuum full on a quarterly basis. You can tune individual tables to have a lower threshold ratio of dead tuples so the system isn't waiting until you have 20% dead rows before va

RE: how to slow down parts of Pg

2020-04-21 Thread Kevin Brannen
From: Virendra Kumar >Autovacuum does takes care of dead tuples and return space to table's >allocated size and can be re-used by fresh incoming rows or any updates. > >Index bloat is still not being taken care of by autovacuum process. You should >use pg_repack to do index rebuild. Keep in min

Re: how to slow down parts of Pg

2020-04-21 Thread David G. Johnston
On Tue, Apr 21, 2020 at 2:25 PM Kevin Brannen wrote: > Sometimes I need the disk space back. It also makes me feel better. (OK, > this may not a good reason but there is a hint of truth in this.) What this > probably means is that I need to get a better understanding of vacuuming. > Imagine you

Re: how to slow down parts of Pg

2020-04-21 Thread Virendra Kumar
Autovacuum does takes care of dead tuples and return space to table's allocated size and can be re-used by fresh incoming rows or any updates. Index bloat is still not being taken care of by autovacuum process. You should use pg_repack to do index rebuild. Keep in mind that pg_repack requires d

RE: how to slow down parts of Pg

2020-04-21 Thread Kevin Brannen
From: Michael Loftis >>From: Kevn Brannen >> I don't particularly like doing the vacuum full, but when it will release >> 20-50% of disk space for a large table, then it's something we live with. As >> I understand, a normal vacuum won't release all the old pages that a "full" >> does, hence w

RE: how to slow down parts of Pg

2020-04-21 Thread Kevin Brannen
From: Michael Loftis >>From: Kevn Brannen >>I have an unusual need: I need Pg to slow down. I know, we all want our DB >>to go faster, but in this case it's speed is working against me in 1 area. >> >>We have systems that are geo-redundant for HA, with the redundancy being >>handled by DRBD to

Re: how to slow down parts of Pg

2020-04-21 Thread Michael Loftis
On Tue, Apr 21, 2020 at 15:05 Kevin Brannen wrote: > *From:* Michael Lewis > > > You say 12.2 is in testing but what are you using now? Have you tuned > configs much? Would you be able to implement partitioning such that your > deletes become truncates or simply a detaching of the old partition?

RE: how to slow down parts of Pg

2020-04-21 Thread Kevin Brannen
From: Michael Lewis > You say 12.2 is in testing but what are you using now? Have you tuned configs > much? Would you be able to implement partitioning such that your deletes > become truncates or simply a detaching of the old partition? Generally if you > are doing a vacuum full, you perhaps

Re: how to slow down parts of Pg

2020-04-21 Thread Michael Loftis
drbdsetup allows you to control the sync rates. On Tue, Apr 21, 2020 at 14:30 Kevin Brannen wrote: > I have an unusual need: I need Pg to slow down. I know, we all want our > DB to go faster, but in this case it's speed is working against me in 1 > area. > > > > We have systems that are geo-red

Re: how to slow down parts of Pg

2020-04-21 Thread Michael Lewis
You say 12.2 is in testing but what are you using now? Have you tuned configs much? Would you be able to implement partitioning such that your deletes become truncates or simply a detaching of the old partition? Generally if you are doing a vacuum full, you perhaps need to tune autovacuum to be mor

how to slow down parts of Pg

2020-04-21 Thread Kevin Brannen
I have an unusual need: I need Pg to slow down. I know, we all want our DB to go faster, but in this case it's speed is working against me in 1 area. We have systems that are geo-redundant for HA, with the redundancy being handled by DRBD to keep the disks in sync, which it does at the block le