If there's no objection, I'd like to move this to the next CF for consideration in PG16.
On Mon, Jan 17, 2022 at 10:39:19PM -0600, Justin Pryzby wrote: > On Tue, Jan 18, 2022 at 02:06:32AM +0500, Daniil Zakhlystov wrote: > > > => Since March, errmsg doesn't need extra parenthesis around it (e3a87b4). > > > I’ve resolved the stuck tests and added zlib support for CI Windows builds > > to patch 0003-*. Thanks > > for the suggestion, all tests seem to be OK now, except the macOS one. It > > won't schedule in Cirrus > > CI for some reason, but I guess it happens because of my GitHub account > > limitation. > > I don't know about your github account, but it works for cfbot, which is now > green. > > Thanks for implementing zlib for windows. Did you try this with default > compressions set to lz4 and zstd ? > > The thread from 2019 is very long, and starts off with the guidance that > compression had been implemented at the wrong layer. It looks like this > hasn't > changed since then. secure_read/write are passed as function pointers to the > ZPQ interface, which then calls back to them to read and flush its compression > buffers. As I understand, the suggestion was to leave the socket reads and > writes alone. And then conditionally de/compress buffers after reading / > before writing from the socket if compression was negotiated. > > It's currently done like this > pq_recvbuf() => secure_read() - when compression is disabled > pq_recvbuf() => ZPQ => secure_read() - when compression is enabled > > Dmitri sent a partial, POC patch which changes the de/compression to happen in > secure_read/write, which is changed to call ZPQ: > https://www.postgresql.org/message-id/ca+q6zcuprssnars+fyobsd-f2stk1roa-4sahfofajowlzi...@mail.gmail.com > pq_recvbuf() => secure_read() => ZPQ > > The same thing is true of the frontend: function pointers to > pqsecure_read/write are being passed to zpq_create, and then the ZPQ interface > called instead of the original functions. Those are the functions which read > using SSL, so they should also handle compression. > > That's where SSL is handled, and it seems like the right place to handle > compression. Have you evaluated that way to do things ? > > Konstantin said he put ZPQ at that layer seems to 1) avoid code duplication > between client/server; and, 2) to allow compression to happen before SSL, to > allow both (if the admin decides it's okay). But I don't see why compression > can't happen before sending to SSL, or after reading from it?
