On 08/22/2013 03:33 PM, Craig Ringer wrote: > On 08/22/2013 01:39 PM, PostgreSQL - Hans-Jürgen Schönig wrote: > >> what would be a reasonable scenario where limiting streaming would make >> sense? i cannot think of any to be honest. > > I tend to agree. If anything we're likely to want the reverse - the > ability to throttle WAL *generation* on the master so streaming can keep up.
(I assume you mean WAL *sending* rather than *generation*.) I don't think we want to throttle WAL sending/receiving at all. The WAL senders should always keep up with the amount of WAL generated. If the rate of WAL sending is restricted, then the pg_basebackup (or another client application) might never catch up. Also, IMO, pg_basebackup starts at the current WAL segment. Thus the unthrottled WAL transfer shouldn't cause significant additional load, such as reading many old WAL segments from the master's disk. > I see a lot of value in throttling base backup transfer rates. It's > something PgBarman does per-tablespace using rsync at the moment, but > it'd be nice if it available as an option possible over the streaming > replication protocol via pg_basebackup so it was easier for people to > use ad-hoc and without all the shell access wrangling. (Possibly huge) DATA directory (tablespaces, etc.) is the actual purpose of this patch. This is the additional load that pg_basebackup imposes on the master. As pointed out elsewhere in the thread, the client-side throttling was chosen because it seemed to be less invasive. But the discussion (including this your comment) keeps me no longer convinced that it's the best way. I'll reconsider things. // Antonin Houska (Tony) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
