Hi. It has already passed a few months but patch still have required review state. Can I help to speed up the review, or should i wait commitfest? I plane complete changes in pgjdbc drive inside PR https://github.com/pgjdbc/pgjdbc/pull/550 but PR blocked current problem with not available stop logical replication.
2016-05-11 7:25 GMT+03:00 Craig Ringer <cr...@2ndquadrant.com>: > On 11 May 2016 at 06:47, Vladimir Gordiychuk <fol...@gmail.com> wrote: > >> Same thread, I just think these are two somewhat separate changes. One is >>> just in the walsender and allows return to command mode during waiting for >>> WAL. The other is more intrusive into the reorder buffer etc and allows >>> aborting decoding during commit processing. So two separate patches make >>> sense here IMO, one on top of the other. >> >> >> About the second part of the patch. What the reason decode and send whole >> transaction? Why we can't process logical decoding via WalSndLoop LSN by >> LSN as it work in physycal replication? For example if transaction contains >> in more them one LSN, first we decode and send "begin", "part data from >> current LSN" and then returns to WalSndLoop on the next iteration we send >> "another part data", "commit". I don't research in this way, because I >> think it will be big changes in comparison callback that stop sending. >> > > There are two parts to that. First, why do we reorder at all, accumulating > a whole transaction in a reorder buffer until we see a commit then sending > it all at once? Second, when sending, why don't we return control to the > walsender between messages? > > For the first: reordering xacts server-side lets the client not worry > about replay order. It just applies them as it receives them. It means the > server can omit uncommitted transactions from the stream entirely and > clients can be kept simple(r). IIRC there are also some issues around > relcache invalidation handling and time travel that make it desirable to > wait until commit before building a snapshot and decoding, but I haven't > dug into the details. Andres is the person who knows that area best. > > As for why we don't return control to the walsender between change > callbacks when processing a reorder buffer at commit time, I'm not really > sure but suspect it's mostly down to easy API and development. If control > returned to the walsender between each change we'd need an async api for > the reorder buffer where you test to see if there's more unprocessed work > and call back into the reorder buffer again if there is. So the reorder > buffer has to keep state for the progress of replaying a commit in a > separate struct, handle repeated calls to process work, etc. Also, since > many individual changes are very small that could lead to a fair bit of > overhead; it'd probably want to process a batch of changes then return. > Which makes it even more complicated. > > If it returned control to the caller between changes then each caller > would also need to have the logic to test for more work and call back into > the reorder buffer. Both the walsender and SQL interface would need it. The > way it is, the loop is just in one place. > > It probably makes more sense to have a callback that can test state and > abort processing, like you've introduced. The callback could probably even > periodically check to see if there's client input to process and consume it. > > -- > Craig Ringer http://www.2ndQuadrant.com/ > PostgreSQL Development, 24x7 Support, Training & Services >
