Greetings,
* Ants Aasma (ants.aa...@eesti.ee) wrote:
> On Thu, Feb 21, 2019 at 12:50 PM Stephen Frost wrote:
>
> > > Rate limit in front of WAL insertion would allow for allocating the
> > > throughput between foreground and background tasks, and even allow for
> > > priority inheritance to alle
On Thu, Feb 21, 2019 at 12:50 PM Stephen Frost wrote:
> > Rate limit in front of WAL insertion would allow for allocating the
> > throughput between foreground and background tasks, and even allow for
> > priority inheritance to alleviate priority inversion due to locks.
>
> I'm not sure how much
Greetings,
* Ants Aasma (ants.aa...@eesti.ee) wrote:
> On Thu, Feb 21, 2019 at 2:20 AM Stephen Frost wrote:
> > The issue with hitting those bandwidth limits is that you end up with
> > queues outside of your control and therefore are unable to prioritize
> > the data going through them. I agree
On Thu, Feb 21, 2019 at 2:20 AM Stephen Frost wrote:
> * Andres Freund (and...@anarazel.de) wrote:
> > On 2019-02-20 18:46:09 -0500, Stephen Frost wrote:
> > > * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote:
> > > > On 2/20/19 10:43 PM, Stephen Frost wrote:
> > > > > Just to share a few addi
Greetings,
* Andres Freund (and...@anarazel.de) wrote:
> On 2019-02-20 18:46:09 -0500, Stephen Frost wrote:
> > * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote:
> > > On 2/20/19 10:43 PM, Stephen Frost wrote:
> > > > Just to share a few additional thoughts after pondering this for a
> > > > wh
Hi,
On 2019-02-20 18:46:09 -0500, Stephen Frost wrote:
> * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote:
> > On 2/20/19 10:43 PM, Stephen Frost wrote:
> > > Just to share a few additional thoughts after pondering this for a
> > > while, but the comment Andres made up-thread really struck a ch
Greetings,
* Tomas Vondra (tomas.von...@2ndquadrant.com) wrote:
> On 2/20/19 10:43 PM, Stephen Frost wrote:
> > Just to share a few additional thoughts after pondering this for a
> > while, but the comment Andres made up-thread really struck a chord- we
> > don't necessairly want to throttle anyth
On 2/20/19 10:43 PM, Stephen Frost wrote:
> Greetings,
> ...
>>
>> So maybe doing it for WAL first makes sense. But I still think we need
>> to have at least a rough idea how it interacts with the existing
>> throttling and how it will work in the end.
>
> Well, it seems like Andres explained how
Greetings,
* Tomas Vondra (tomas.von...@2ndquadrant.com) wrote:
> On 2/19/19 8:40 PM, Andres Freund wrote:
> > On 2019-02-19 20:34:25 +0100, Tomas Vondra wrote:
> >> On 2/19/19 8:22 PM, Andres Freund wrote:
> >>> And my main point is that even if you implement a proper bytes/sec limit
> >>> ONLY f
On 2/19/19 8:40 PM, Andres Freund wrote:
> Hi,
>
> On 2019-02-19 20:34:25 +0100, Tomas Vondra wrote:
>> On 2/19/19 8:22 PM, Andres Freund wrote:
>>> And my main point is that even if you implement a proper bytes/sec limit
>>> ONLY for WAL, the behaviour of VACUUM rate limiting doesn't get
>>> m
On Wed, 20 Feb 2019 at 07:28, Robert Haas wrote:
> Or maybe we should just blow up the current vacuum cost delay stuff
> and replace it with something that is easier to tune. For example, we
> could just have one parameter that sets the maximum read rate in kB/s
> and another that sets the maximu
Hi,
On 2019-02-19 20:34:25 +0100, Tomas Vondra wrote:
> On 2/19/19 8:22 PM, Andres Freund wrote:
> > And my main point is that even if you implement a proper bytes/sec limit
> > ONLY for WAL, the behaviour of VACUUM rate limiting doesn't get
> > meaningfully more confusing than right now.
> >
>
On 2/19/19 8:22 PM, Andres Freund wrote:
> On 2019-02-19 20:02:32 +0100, Tomas Vondra wrote:
>> Let's do a short example. Assume the default vacuum costing parameters
>>
>> vacuum_cost_limit = 200
>> vacuum_cost_delay = 20ms
>> cost_page_dirty = 20
>>
>> and for simplicity we only do
On 2019-02-19 20:02:32 +0100, Tomas Vondra wrote:
> Let's do a short example. Assume the default vacuum costing parameters
>
> vacuum_cost_limit = 200
> vacuum_cost_delay = 20ms
> cost_page_dirty = 20
>
> and for simplicity we only do writes. So vacuum can do ~8MB/s of writes.
>
> Now,
On 2/19/19 7:28 PM, Robert Haas wrote:
> On Fri, Feb 15, 2019 at 1:42 PM Andres Freund wrote:
>> I think it'd not be insane to add two things:
>> - WAL write rate limiting, independent of the vacuum stuff. It'd also be
>> used by lots of other bulk commands (CREATE INDEX, ALTER TABLE
>> rew
On 2/19/19 7:50 PM, Andres Freund wrote:
> On 2019-02-19 19:43:14 +0100, Tomas Vondra wrote:
>>
>>
>> On 2/19/19 7:35 PM, Andres Freund wrote:
>>> Hi,
>>>
>>> On 2019-02-19 13:28:00 -0500, Robert Haas wrote:
On Fri, Feb 15, 2019 at 1:42 PM Andres Freund wrote:
> I think it'd not be insane
On 2019-02-19 19:43:14 +0100, Tomas Vondra wrote:
>
>
> On 2/19/19 7:35 PM, Andres Freund wrote:
> > Hi,
> >
> > On 2019-02-19 13:28:00 -0500, Robert Haas wrote:
> >> On Fri, Feb 15, 2019 at 1:42 PM Andres Freund wrote:
> >>> I think it'd not be insane to add two things:
> >>> - WAL write rate
On Tue, Feb 19, 2019 at 1:35 PM Andres Freund wrote:
> I still don't *AT ALL* buy Stephen and Tomas' argument that it'd be
> confusing that when both VACUUM and WAL cost limiting are active, the
> lower limit takes effect.
I think you guys may all be in vigorous -- not to say mortal -- agreement.
On 2/19/19 7:35 PM, Andres Freund wrote:
> Hi,
>
> On 2019-02-19 13:28:00 -0500, Robert Haas wrote:
>> On Fri, Feb 15, 2019 at 1:42 PM Andres Freund wrote:
>>> I think it'd not be insane to add two things:
>>> - WAL write rate limiting, independent of the vacuum stuff. It'd also be
>>> used
Hi,
On 2019-02-19 13:28:00 -0500, Robert Haas wrote:
> On Fri, Feb 15, 2019 at 1:42 PM Andres Freund wrote:
> > I think it'd not be insane to add two things:
> > - WAL write rate limiting, independent of the vacuum stuff. It'd also be
> > used by lots of other bulk commands (CREATE INDEX, ALTER
On Fri, Feb 15, 2019 at 1:42 PM Andres Freund wrote:
> I think it'd not be insane to add two things:
> - WAL write rate limiting, independent of the vacuum stuff. It'd also be
> used by lots of other bulk commands (CREATE INDEX, ALTER TABLE
> rewrites, ...)
> - Account for WAL writes in the cu
Greetings,
* Tomas Vondra (tomas.von...@2ndquadrant.com) wrote:
> On 2/15/19 7:41 PM, Andres Freund wrote:
> > On 2019-02-15 08:50:03 -0500, Stephen Frost wrote:
> >> * Andres Freund (and...@anarazel.de) wrote:
> >>> On 2019-02-14 11:02:24 -0500, Stephen Frost wrote:
> On Thu, Feb 14, 2019 at
On 2/15/19 7:41 PM, Andres Freund wrote:
> Hi,
>
> On 2019-02-15 08:50:03 -0500, Stephen Frost wrote:
>> * Andres Freund (and...@anarazel.de) wrote:
>>> On 2019-02-14 11:02:24 -0500, Stephen Frost wrote:
On Thu, Feb 14, 2019 at 10:15 Peter Eisentraut <
peter.eisentr...@2ndquadrant.com>
Hi,
On 2019-02-15 08:50:03 -0500, Stephen Frost wrote:
> * Andres Freund (and...@anarazel.de) wrote:
> > On 2019-02-14 11:02:24 -0500, Stephen Frost wrote:
> > > On Thu, Feb 14, 2019 at 10:15 Peter Eisentraut <
> > > peter.eisentr...@2ndquadrant.com> wrote:
> > > > On 14/02/2019 11:03, Tomas Vondr
Greetings,
* Andres Freund (and...@anarazel.de) wrote:
> On 2019-02-14 11:02:24 -0500, Stephen Frost wrote:
> > On Thu, Feb 14, 2019 at 10:15 Peter Eisentraut <
> > peter.eisentr...@2ndquadrant.com> wrote:
> > > On 14/02/2019 11:03, Tomas Vondra wrote:
> > > > But if you add extra sleep() calls so
On 2019-02-14 11:02:24 -0500, Stephen Frost wrote:
> Greetings,
>
> On Thu, Feb 14, 2019 at 10:15 Peter Eisentraut <
> peter.eisentr...@2ndquadrant.com> wrote:
>
> > On 14/02/2019 11:03, Tomas Vondra wrote:
> > > But if you add extra sleep() calls somewhere (say because there's also
> > > limit o
Hi,
On 2019-02-14 16:16:05 +0100, Peter Eisentraut wrote:
> On 13/02/2019 16:40, Andres Freund wrote:
> > On February 13, 2019 4:39:21 PM GMT+01:00, Peter Eisentraut
> > wrote:
> >> On 13/02/2019 13:18, Andres Freund wrote:
> >>> But I don't think the way you did it is acceptable - we can't just
Greetings,
On Thu, Feb 14, 2019 at 10:15 Peter Eisentraut <
peter.eisentr...@2ndquadrant.com> wrote:
> On 14/02/2019 11:03, Tomas Vondra wrote:
> > But if you add extra sleep() calls somewhere (say because there's also
> > limit on WAL throughput), it will affect how fast VACUUM works in
> > gene
On 14/02/2019 11:03, Tomas Vondra wrote:
> But if you add extra sleep() calls somewhere (say because there's also
> limit on WAL throughput), it will affect how fast VACUUM works in
> general. Yet it'll continue with the cost-based throttling, but it will
> never reach the limits. Say you do anothe
On 13/02/2019 16:40, Andres Freund wrote:
> On February 13, 2019 4:39:21 PM GMT+01:00, Peter Eisentraut
> wrote:
>> On 13/02/2019 13:18, Andres Freund wrote:
>>> But I don't think the way you did it is acceptable - we can't just
>> delay while holding buffer locks, in critical sections, while not
On 2/14/19 10:36 AM, Andres Freund wrote:
>
>
> On February 14, 2019 10:31:57 AM GMT+01:00, Tomas Vondra
> wrote:
>>
>>
>> On 2/14/19 10:06 AM, Andres Freund wrote:
>>> Hi,
>>>
>>> On 2019-02-14 10:00:38 +0100, Tomas Vondra wrote:
On 2/13/19 4:31 PM, Stephen Frost wrote:
> Greetings
On February 14, 2019 10:31:57 AM GMT+01:00, Tomas Vondra
wrote:
>
>
>On 2/14/19 10:06 AM, Andres Freund wrote:
>> Hi,
>>
>> On 2019-02-14 10:00:38 +0100, Tomas Vondra wrote:
>>> On 2/13/19 4:31 PM, Stephen Frost wrote:
Greetings,
* Peter Eisentraut (peter.eisentr...@2ndquadrant
On 2/14/19 10:06 AM, Andres Freund wrote:
> Hi,
>
> On 2019-02-14 10:00:38 +0100, Tomas Vondra wrote:
>> On 2/13/19 4:31 PM, Stephen Frost wrote:
>>> Greetings,
>>>
>>> * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote:
Bulk operations like CREATE INDEX, ALTER TABLE, or bulk load
Hi,
On 2019-02-14 10:00:38 +0100, Tomas Vondra wrote:
> On 2/13/19 4:31 PM, Stephen Frost wrote:
> > Greetings,
> >
> > * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote:
> >> Bulk operations like CREATE INDEX, ALTER TABLE, or bulk loads can create
> >> a lot of WAL. A lot of WAL at on
On Thu, Feb 14, 2019 at 12:53 AM Peter Geoghegan wrote:
> I didn't mention that the utility they used would send SIGSTOP and
> SIGCONT in close succession. (Yeah, I know.)
Actually, it SIGSTOP'd backends, not the WAL writer or background writer.
--
Peter Geoghegan
On 2/13/19 4:31 PM, Stephen Frost wrote:
> Greetings,
>
> * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote:
>> Bulk operations like CREATE INDEX, ALTER TABLE, or bulk loads can create
>> a lot of WAL. A lot of WAL at once can cause delays in replication.
>
> Agreed, though I think
On Thu, Feb 14, 2019 at 12:52 AM Peter Geoghegan wrote:
> I imagine that it held the critical locks briefly.
I didn't mention that the utility they used would send SIGSTOP and
SIGCONT in close succession. (Yeah, I know.)
--
Peter Geoghegan
On Thu, Feb 14, 2019 at 12:42 AM Andres Freund wrote:
> That can't have been the workaround - either you'd interrupt it while
> holding critical locks (in which case nobody could write WAL anymore),
> or you'd just move all the writing to backends, no?
I imagine that it held the critical locks br
Hi,
On 2019-02-13 23:21:39 -0800, Peter Geoghegan wrote:
> There would occasionally be cases where ops
> would find it useful to throttle WAL writing using their own terrible
> kludge (it involved sending SIGSTOP to the WAL writer).
That can't have been the workaround - either you'd interrupt it
On Wed, Feb 13, 2019 at 4:16 AM Peter Eisentraut
wrote:
> Bulk operations like CREATE INDEX, ALTER TABLE, or bulk loads can create
> a lot of WAL. A lot of WAL at once can cause delays in replication.
> For synchronous replication, this can make seemingly unrelated sessions
> hang. But also for
On 13.02.2019 19:57, Tom Lane wrote:
Andres Freund writes:
On February 13, 2019 1:16:07 PM GMT+01:00, Peter Eisentraut
wrote:
One idea to address this is to slow down WAL-generating maintenance
operations. This is similar to the vacuum delay. Where the vacuum
delay counts notional I/O c
On February 13, 2019 4:39:21 PM GMT+01:00, Peter Eisentraut
wrote:
>On 13/02/2019 13:18, Andres Freund wrote:
>> But I don't think the way you did it is acceptable - we can't just
>delay while holding buffer locks, in critical sections, while not
>interruptible.
>
>The code I added to XLogInse
On 13/02/2019 13:18, Andres Freund wrote:
> But I don't think the way you did it is acceptable - we can't just delay
> while holding buffer locks, in critical sections, while not interruptible.
The code I added to XLogInsertRecord() is not inside the critical section.
--
Peter Eisentraut
Greetings,
* Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote:
> Bulk operations like CREATE INDEX, ALTER TABLE, or bulk loads can create
> a lot of WAL. A lot of WAL at once can cause delays in replication.
Agreed, though I think VACUUM should certainly be included in this.
I'm all fo
Andres Freund writes:
> On February 13, 2019 1:16:07 PM GMT+01:00, Peter Eisentraut
> wrote:
>> One idea to address this is to slow down WAL-generating maintenance
>> operations. This is similar to the vacuum delay. Where the vacuum
>> delay counts notional I/O cost before sleeping, here we wo
Hi
13.02.2019 17:16, Peter Eisentraut пишет:
Bulk operations like CREATE INDEX, ALTER TABLE, or bulk loads can create
a lot of WAL. A lot of WAL at once can cause delays in replication.
For synchronous replication, this can make seemingly unrelated sessions
hang. But also for asynchronous repl
Hi,
On February 13, 2019 1:16:07 PM GMT+01:00, Peter Eisentraut
wrote:
>Bulk operations like CREATE INDEX, ALTER TABLE, or bulk loads can
>create
>a lot of WAL. A lot of WAL at once can cause delays in replication.
>For synchronous replication, this can make seemingly unrelated sessions
>hang.
Bulk operations like CREATE INDEX, ALTER TABLE, or bulk loads can create
a lot of WAL. A lot of WAL at once can cause delays in replication.
For synchronous replication, this can make seemingly unrelated sessions
hang. But also for asynchronous replication, it will increase latency.
One idea to
48 matches
Mail list logo
