Greetings,
* Ants Aasma (ants.aa...@eesti.ee) wrote:
> On Thu, Feb 21, 2019 at 12:50 PM Stephen Frost wrote:
>
> > > Rate limit in front of WAL insertion would allow for allocating the
> > > throughput between foreground and background tasks, and even allow for
> > > priority inheritance to alle
On Thu, Feb 21, 2019 at 12:50 PM Stephen Frost wrote:
> > Rate limit in front of WAL insertion would allow for allocating the
> > throughput between foreground and background tasks, and even allow for
> > priority inheritance to alleviate priority inversion due to locks.
>
> I'm not sure how much
Greetings,
* Ants Aasma (ants.aa...@eesti.ee) wrote:
> On Thu, Feb 21, 2019 at 2:20 AM Stephen Frost wrote:
> > The issue with hitting those bandwidth limits is that you end up with
> > queues outside of your control and therefore are unable to prioritize
> > the data going through them. I agree
On Thu, Feb 21, 2019 at 2:20 AM Stephen Frost wrote:
> * Andres Freund (and...@anarazel.de) wrote:
> > On 2019-02-20 18:46:09 -0500, Stephen Frost wrote:
> > > * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote:
> > > > On 2/20/19 10:43 PM, Stephen Frost wrote:
> > > > > Just to share a few addi
Greetings,
* Andres Freund (and...@anarazel.de) wrote:
> On 2019-02-20 18:46:09 -0500, Stephen Frost wrote:
> > * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote:
> > > On 2/20/19 10:43 PM, Stephen Frost wrote:
> > > > Just to share a few additional thoughts after pondering this for a
> > > > wh
Hi,
On 2019-02-20 18:46:09 -0500, Stephen Frost wrote:
> * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote:
> > On 2/20/19 10:43 PM, Stephen Frost wrote:
> > > Just to share a few additional thoughts after pondering this for a
> > > while, but the comment Andres made up-thread really struck a ch
Greetings,
* Tomas Vondra (tomas.von...@2ndquadrant.com) wrote:
> On 2/20/19 10:43 PM, Stephen Frost wrote:
> > Just to share a few additional thoughts after pondering this for a
> > while, but the comment Andres made up-thread really struck a chord- we
> > don't necessairly want to throttle anyth
On 2/20/19 10:43 PM, Stephen Frost wrote:
> Greetings,
> ...
>>
>> So maybe doing it for WAL first makes sense. But I still think we need
>> to have at least a rough idea how it interacts with the existing
>> throttling and how it will work in the end.
>
> Well, it seems like Andres explained how
Greetings,
* Tomas Vondra (tomas.von...@2ndquadrant.com) wrote:
> On 2/19/19 8:40 PM, Andres Freund wrote:
> > On 2019-02-19 20:34:25 +0100, Tomas Vondra wrote:
> >> On 2/19/19 8:22 PM, Andres Freund wrote:
> >>> And my main point is that even if you implement a proper bytes/sec limit
> >>> ONLY f
On 2/19/19 8:40 PM, Andres Freund wrote:
> Hi,
>
> On 2019-02-19 20:34:25 +0100, Tomas Vondra wrote:
>> On 2/19/19 8:22 PM, Andres Freund wrote:
>>> And my main point is that even if you implement a proper bytes/sec limit
>>> ONLY for WAL, the behaviour of VACUUM rate limiting doesn't get
>>> m
On Wed, 20 Feb 2019 at 07:28, Robert Haas wrote:
> Or maybe we should just blow up the current vacuum cost delay stuff
> and replace it with something that is easier to tune. For example, we
> could just have one parameter that sets the maximum read rate in kB/s
> and another that sets the maximu
Hi,
On 2019-02-19 20:34:25 +0100, Tomas Vondra wrote:
> On 2/19/19 8:22 PM, Andres Freund wrote:
> > And my main point is that even if you implement a proper bytes/sec limit
> > ONLY for WAL, the behaviour of VACUUM rate limiting doesn't get
> > meaningfully more confusing than right now.
> >
>
On 2/19/19 8:22 PM, Andres Freund wrote:
> On 2019-02-19 20:02:32 +0100, Tomas Vondra wrote:
>> Let's do a short example. Assume the default vacuum costing parameters
>>
>> vacuum_cost_limit = 200
>> vacuum_cost_delay = 20ms
>> cost_page_dirty = 20
>>
>> and for simplicity we only do
On 2019-02-19 20:02:32 +0100, Tomas Vondra wrote:
> Let's do a short example. Assume the default vacuum costing parameters
>
> vacuum_cost_limit = 200
> vacuum_cost_delay = 20ms
> cost_page_dirty = 20
>
> and for simplicity we only do writes. So vacuum can do ~8MB/s of writes.
>
> Now,
On 2/19/19 7:28 PM, Robert Haas wrote:
> On Fri, Feb 15, 2019 at 1:42 PM Andres Freund wrote:
>> I think it'd not be insane to add two things:
>> - WAL write rate limiting, independent of the vacuum stuff. It'd also be
>> used by lots of other bulk commands (CREATE INDEX, ALTER TABLE
>> rew
On 2/19/19 7:50 PM, Andres Freund wrote:
> On 2019-02-19 19:43:14 +0100, Tomas Vondra wrote:
>>
>>
>> On 2/19/19 7:35 PM, Andres Freund wrote:
>>> Hi,
>>>
>>> On 2019-02-19 13:28:00 -0500, Robert Haas wrote:
On Fri, Feb 15, 2019 at 1:42 PM Andres Freund wrote:
> I think it'd not be insane
On 2019-02-19 19:43:14 +0100, Tomas Vondra wrote:
>
>
> On 2/19/19 7:35 PM, Andres Freund wrote:
> > Hi,
> >
> > On 2019-02-19 13:28:00 -0500, Robert Haas wrote:
> >> On Fri, Feb 15, 2019 at 1:42 PM Andres Freund wrote:
> >>> I think it'd not be insane to add two things:
> >>> - WAL write rate
On Tue, Feb 19, 2019 at 1:35 PM Andres Freund wrote:
> I still don't *AT ALL* buy Stephen and Tomas' argument that it'd be
> confusing that when both VACUUM and WAL cost limiting are active, the
> lower limit takes effect.
I think you guys may all be in vigorous -- not to say mortal -- agreement.
On 2/19/19 7:35 PM, Andres Freund wrote:
> Hi,
>
> On 2019-02-19 13:28:00 -0500, Robert Haas wrote:
>> On Fri, Feb 15, 2019 at 1:42 PM Andres Freund wrote:
>>> I think it'd not be insane to add two things:
>>> - WAL write rate limiting, independent of the vacuum stuff. It'd also be
>>> used
Hi,
On 2019-02-19 13:28:00 -0500, Robert Haas wrote:
> On Fri, Feb 15, 2019 at 1:42 PM Andres Freund wrote:
> > I think it'd not be insane to add two things:
> > - WAL write rate limiting, independent of the vacuum stuff. It'd also be
> > used by lots of other bulk commands (CREATE INDEX, ALTER
On Fri, Feb 15, 2019 at 1:42 PM Andres Freund wrote:
> I think it'd not be insane to add two things:
> - WAL write rate limiting, independent of the vacuum stuff. It'd also be
> used by lots of other bulk commands (CREATE INDEX, ALTER TABLE
> rewrites, ...)
> - Account for WAL writes in the cu
Greetings,
* Tomas Vondra (tomas.von...@2ndquadrant.com) wrote:
> On 2/15/19 7:41 PM, Andres Freund wrote:
> > On 2019-02-15 08:50:03 -0500, Stephen Frost wrote:
> >> * Andres Freund (and...@anarazel.de) wrote:
> >>> On 2019-02-14 11:02:24 -0500, Stephen Frost wrote:
> On Thu, Feb 14, 2019 at
On 2/15/19 7:41 PM, Andres Freund wrote:
> Hi,
>
> On 2019-02-15 08:50:03 -0500, Stephen Frost wrote:
>> * Andres Freund (and...@anarazel.de) wrote:
>>> On 2019-02-14 11:02:24 -0500, Stephen Frost wrote:
On Thu, Feb 14, 2019 at 10:15 Peter Eisentraut <
peter.eisentr...@2ndquadrant.com>
Hi,
On 2019-02-15 08:50:03 -0500, Stephen Frost wrote:
> * Andres Freund (and...@anarazel.de) wrote:
> > On 2019-02-14 11:02:24 -0500, Stephen Frost wrote:
> > > On Thu, Feb 14, 2019 at 10:15 Peter Eisentraut <
> > > peter.eisentr...@2ndquadrant.com> wrote:
> > > > On 14/02/2019 11:03, Tomas Vondr
Greetings,
* Andres Freund (and...@anarazel.de) wrote:
> On 2019-02-14 11:02:24 -0500, Stephen Frost wrote:
> > On Thu, Feb 14, 2019 at 10:15 Peter Eisentraut <
> > peter.eisentr...@2ndquadrant.com> wrote:
> > > On 14/02/2019 11:03, Tomas Vondra wrote:
> > > > But if you add extra sleep() calls so
On 2019-02-14 11:02:24 -0500, Stephen Frost wrote:
> Greetings,
>
> On Thu, Feb 14, 2019 at 10:15 Peter Eisentraut <
> peter.eisentr...@2ndquadrant.com> wrote:
>
> > On 14/02/2019 11:03, Tomas Vondra wrote:
> > > But if you add extra sleep() calls somewhere (say because there's also
> > > limit o
Hi,
On 2019-02-14 16:16:05 +0100, Peter Eisentraut wrote:
> On 13/02/2019 16:40, Andres Freund wrote:
> > On February 13, 2019 4:39:21 PM GMT+01:00, Peter Eisentraut
> > wrote:
> >> On 13/02/2019 13:18, Andres Freund wrote:
> >>> But I don't think the way you did it is acceptable - we can't just
Greetings,
On Thu, Feb 14, 2019 at 10:15 Peter Eisentraut <
peter.eisentr...@2ndquadrant.com> wrote:
> On 14/02/2019 11:03, Tomas Vondra wrote:
> > But if you add extra sleep() calls somewhere (say because there's also
> > limit on WAL throughput), it will affect how fast VACUUM works in
> > gene
On 14/02/2019 11:03, Tomas Vondra wrote:
> But if you add extra sleep() calls somewhere (say because there's also
> limit on WAL throughput), it will affect how fast VACUUM works in
> general. Yet it'll continue with the cost-based throttling, but it will
> never reach the limits. Say you do anothe
On 13/02/2019 16:40, Andres Freund wrote:
> On February 13, 2019 4:39:21 PM GMT+01:00, Peter Eisentraut
> wrote:
>> On 13/02/2019 13:18, Andres Freund wrote:
>>> But I don't think the way you did it is acceptable - we can't just
>> delay while holding buffer locks, in critical sections, while not
On 2/14/19 10:36 AM, Andres Freund wrote:
>
>
> On February 14, 2019 10:31:57 AM GMT+01:00, Tomas Vondra
> wrote:
>>
>>
>> On 2/14/19 10:06 AM, Andres Freund wrote:
>>> Hi,
>>>
>>> On 2019-02-14 10:00:38 +0100, Tomas Vondra wrote:
On 2/13/19 4:31 PM, Stephen Frost wrote:
> Greetings
On February 14, 2019 10:31:57 AM GMT+01:00, Tomas Vondra
wrote:
>
>
>On 2/14/19 10:06 AM, Andres Freund wrote:
>> Hi,
>>
>> On 2019-02-14 10:00:38 +0100, Tomas Vondra wrote:
>>> On 2/13/19 4:31 PM, Stephen Frost wrote:
Greetings,
* Peter Eisentraut (peter.eisentr...@2ndquadrant
On 2/14/19 10:06 AM, Andres Freund wrote:
> Hi,
>
> On 2019-02-14 10:00:38 +0100, Tomas Vondra wrote:
>> On 2/13/19 4:31 PM, Stephen Frost wrote:
>>> Greetings,
>>>
>>> * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote:
Bulk operations like CREATE INDEX, ALTER TABLE, or bulk load
Hi,
On 2019-02-14 10:00:38 +0100, Tomas Vondra wrote:
> On 2/13/19 4:31 PM, Stephen Frost wrote:
> > Greetings,
> >
> > * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote:
> >> Bulk operations like CREATE INDEX, ALTER TABLE, or bulk loads can create
> >> a lot of WAL. A lot of WAL at on
On Thu, Feb 14, 2019 at 12:53 AM Peter Geoghegan wrote:
> I didn't mention that the utility they used would send SIGSTOP and
> SIGCONT in close succession. (Yeah, I know.)
Actually, it SIGSTOP'd backends, not the WAL writer or background writer.
--
Peter Geoghegan
On 2/13/19 4:31 PM, Stephen Frost wrote:
> Greetings,
>
> * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote:
>> Bulk operations like CREATE INDEX, ALTER TABLE, or bulk loads can create
>> a lot of WAL. A lot of WAL at once can cause delays in replication.
>
> Agreed, though I think
On Thu, Feb 14, 2019 at 12:52 AM Peter Geoghegan wrote:
> I imagine that it held the critical locks briefly.
I didn't mention that the utility they used would send SIGSTOP and
SIGCONT in close succession. (Yeah, I know.)
--
Peter Geoghegan
On Thu, Feb 14, 2019 at 12:42 AM Andres Freund wrote:
> That can't have been the workaround - either you'd interrupt it while
> holding critical locks (in which case nobody could write WAL anymore),
> or you'd just move all the writing to backends, no?
I imagine that it held the critical locks br
Hi,
On 2019-02-13 23:21:39 -0800, Peter Geoghegan wrote:
> There would occasionally be cases where ops
> would find it useful to throttle WAL writing using their own terrible
> kludge (it involved sending SIGSTOP to the WAL writer).
That can't have been the workaround - either you'd interrupt it
On Wed, Feb 13, 2019 at 4:16 AM Peter Eisentraut
wrote:
> Bulk operations like CREATE INDEX, ALTER TABLE, or bulk loads can create
> a lot of WAL. A lot of WAL at once can cause delays in replication.
> For synchronous replication, this can make seemingly unrelated sessions
> hang. But also for
On 13.02.2019 19:57, Tom Lane wrote:
Andres Freund writes:
On February 13, 2019 1:16:07 PM GMT+01:00, Peter Eisentraut
wrote:
One idea to address this is to slow down WAL-generating maintenance
operations. This is similar to the vacuum delay. Where the vacuum
delay counts notional I/O c
On February 13, 2019 4:39:21 PM GMT+01:00, Peter Eisentraut
wrote:
>On 13/02/2019 13:18, Andres Freund wrote:
>> But I don't think the way you did it is acceptable - we can't just
>delay while holding buffer locks, in critical sections, while not
>interruptible.
>
>The code I added to XLogInse
On 13/02/2019 13:18, Andres Freund wrote:
> But I don't think the way you did it is acceptable - we can't just delay
> while holding buffer locks, in critical sections, while not interruptible.
The code I added to XLogInsertRecord() is not inside the critical section.
--
Peter Eisentraut
Greetings,
* Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote:
> Bulk operations like CREATE INDEX, ALTER TABLE, or bulk loads can create
> a lot of WAL. A lot of WAL at once can cause delays in replication.
Agreed, though I think VACUUM should certainly be included in this.
I'm all fo
Andres Freund writes:
> On February 13, 2019 1:16:07 PM GMT+01:00, Peter Eisentraut
> wrote:
>> One idea to address this is to slow down WAL-generating maintenance
>> operations. This is similar to the vacuum delay. Where the vacuum
>> delay counts notional I/O cost before sleeping, here we wo
Hi
13.02.2019 17:16, Peter Eisentraut пишет:
Bulk operations like CREATE INDEX, ALTER TABLE, or bulk loads can create
a lot of WAL. A lot of WAL at once can cause delays in replication.
For synchronous replication, this can make seemingly unrelated sessions
hang. But also for asynchronous repl
Hi,
On February 13, 2019 1:16:07 PM GMT+01:00, Peter Eisentraut
wrote:
>Bulk operations like CREATE INDEX, ALTER TABLE, or bulk loads can
>create
>a lot of WAL. A lot of WAL at once can cause delays in replication.
>For synchronous replication, this can make seemingly unrelated sessions
>hang.
47 matches
Mail list logo
