On 12 October 2017 at 00:57, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote:
> The reason of such behavior is obvious: wal sender has to decode huge > transaction generate by insert although it has no relation to this > publication. It does. Though I wouldn't expect anywhere near the kind of drop you report, and haven't observed it here. Is the CREATE TABLE and INSERT done in the same transaction? Because that's a known pathological case for logical replication, it has to do a LOT of extra work when it's in a transaction that has done DDL. I'm sure there's room for optimisation there, but the general recommendation for now is "don't do that". > Filtering of insert records of this transaction is done only inside output > plug-in. Only partly true. The output plugin can register a transaction origin filter and use that to say it's entirely uninterested in a transaction. But this only works based on filtering by origins. Not tables. I imagine we could call another hook in output plugins, "do you care about this table", and use it to skip some more work for tuples that particular decoding session isn't interested in. Skip adding them to the reorder buffer, etc. No such hook currently exists, but it'd be an interesting patch for Pg11 if you feel like working on it. > Unfortunately it is not quite clear how to make wal-sender smarter and let > him skip transaction not affecting its publication. As noted, it already can do so by origin. Mostly. We cannot totally skip over WAL, since we need to process various invalidations etc. See ReorderBufferSkip. It's not so simple by table since we don't know early enough whether the xact affects tables of interest or not. But you could definitely do some selective skipping. Making it efficient could be the challenge. > Once of the possible solutions is to let backend inform wal-sender about > smallest LSN it should wait for (backend knows which table is affected by > current operation, > so which publications are interested in this operation and so can point wal > -sender to the proper LSN without decoding huge part of WAL. > But it seems to be not so easy to implement. Sounds like confusing layering violations to me. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
