Greetings, * Ants Aasma (ants.aa...@eesti.ee) wrote: > On Thu, Feb 21, 2019 at 12:50 PM Stephen Frost <sfr...@snowman.net> wrote: > > > > Rate limit in front of WAL insertion would allow for allocating the > > > throughput between foreground and background tasks, and even allow for > > > priority inheritance to alleviate priority inversion due to locks. > > > > I'm not sure how much we have to worry about priority inversion here as > > you need to have conflicts for that and if there's actually a conflict, > > then it seems like we should just press on. > > > > That is, a non-concurrent REINDEX is going to prevent an UPDATE from > > modifying anything in the table, which if the UPDATE is a higher > > priority than the REINDEX would be priority inversion, but that doesn't > > mean we should slow down the REINDEX to allow the UPDATE to happen > > because the UPDATE simply can't happen until the REINDEX is complete. > > Now, we might slow down the REINDEX because there's UPDATEs against > > *other* tables that aren't conflicting and we want those UPDATEs to be > > prioritized over the REINDEX but then that isn't priority inversion. > > I was thinking along the lines that each backend gets a budget of WAL > insertion credits per time interval, and when the credits run out the > process sleeps. With this type of scheme it would be reasonably > straightforward to let UPDATEs being blocked by REINDEX to transfer their > WAL insertion budgets to the REINDEX, making it get a larger piece of the > total throughput pie.
Sure, that could possibly be done if it's a per-backend throttle mechanism, but that's got more-or-less the same issue as a per-command mechanism and work_mem as discussed up-thread. Also seems like if we've solved for a way to do this transferring and delay and such that we could come up with a way to prioritize (or 'give more credits') to foreground and less to background (there was another point made elsewhere in the thread that background processes should still be given *some* amount of credits, always, so that they don't end up starving completely, and I agree with that). There's actually a lot of similarity or parallels between this and basic network traffic shaping, it seems to me anyway, where you have a pipe of a certain size and you want to prioritize some things (like interactive SSH) while de-prioritizing other things (bulk SCP) but also using the pipe fully. Thanks! Stephen
signature.asc
Description: PGP signature
