Ok, thanks
> On 3 Mar 2022, at 02:33, Justin Pryzby wrote:
>
> If there's no objection, I'd like to move this to the next CF for
> consideration
> in PG16.
>
> On Mon, Jan 17, 2022 at 10:39:19PM -0600, Justin Pryzby wrote:
>> On Tue, Jan 18, 2022 at 02:0
change anything fundamentally. So I would
appreciate someone doing
a review of the current patch version.
The original thread is quite huge so I guess that it makes it hard to catch up
with the current patch status.
I can make a new one with a detailed summary if that would help.
Thanks,
Daniil Zakhlystov
Hi, thanks for your fix! I am currently working on implementing the lz4 support for libpq compression. Looking forward to post an update soon.—Daniil ZakhlystovOn 2 Oct 2021, at 02:20, Daniel Gustafsson wrote:On 2 Sep 2021, at 00:29, Daniel Gustafsson wrote:On 29 Jul 2021, at 16:57, Daniil
yzby wrote:
> Also, I'm not sure, but I think you may find that the zstd configure.ac should
> use pkgconfig. This allowed the CIs to compile these patches. Without
> pkg-config, the macos CI couldn't find (at least) LZ4.k
> https://commitfest.postgresql.org/32/2813/
> https://commitfest.postgresql.org/32/3015/
Now --with-zstd uses pkg-config to link the ZSTD library and works correctly on
macos.
I would appreciate hearing your thoughts on the new version of the patch,
Daniil Zakhlystov
introduces more flexibility and contains almost no overhead compared to
permanent compression.
Also, later we may introduce some setting to control should we reset the
compression context in each message without
breaking the backward compatibility.
--
Daniil Zakhlystov
The new status of this patc
Hi, thanks for your review,
> On Feb 22, 2021, at 10:38 AM, Craig Ringer
> wrote:
>
>
> On Thu, 11 Feb 2021, 21:09 Daniil Zakhlystov,
> wrote::
>
> 3. Chunked compression allows to compress only well compressible messages and
> save the CPU cycles by not
need to be reached in this
patch to make it committable.
Thanks,
Daniil Zakhlystov
more than X bytes),
I am not sure if the current compression criteria provides the best results.
Thanks,
Daniil Zakhlystov
A-1682-4EF2-B835-C5F46615EC49%40yandex-team.ru
So maybe it is worthwhile to use similar restrictions in this patch.
—
Daniil Zakhlystov
higher than 19, decompressor returned the appropriate error
(excerpt from PostgreSQL server log):
LOG: failed to decompress data: Frame requires too much memory for decoding
Full benchmark report:
https://docs.google.com/document/d/1LI8hPzMkzkdQLf7pTN-LXPjIJdjN33bEAqVJj0PLnHA
Pull request with max window size limit:
https://github.com/postgrespro/libpq_compression/pull/5
This should fix the possible attack vectors related to high ZSTD compression
levels.
—
Daniil Zakhlystov
CPU/Memory/Network consumption graph is available here:
https://docs.google.com/document/d/1gI7c3_YvcL5-PzeK65P0pIY-4BI9KBDwlfPpGhYxrNg
To sum up, there is actually almost no difference when decompressing the
different compression levels, except the Committed_AS size.
—
Daniil Zakhlystov
intermediate component between the secure_read and
the decompressor will do the following:
1. Read the next 5 bytes (type and length) from the buffer
2.1 If the message type is other than CompressedData, forward it straight to
the PqRecvBuffer / conn->inBuffer.
2.2 If the message type is CompressedData, forward its contents to the current
decompressor.
What do you think of this approach?
—
Daniil Zakhlystov
aining “alg_idx=0”
which means that the client chose the uncompressed
client-to-server messaging. Actually, the client had no other options, because
the “uncompressed” was the only option left after the intersection of
compression algorithms from the connection string and algorithms received from
the server in the CompressionAck message.
Every next message from the client is now being sent uncompressed.
—
Daniil Zakhlystov
ms = ‘...’ // default value is ‘uncompressed’
decompress_algorithms = ‘...’ // default value is ‘uncompressed’
Please, let me know if I was unclear somewhere in the protocol description so I
can clarify the things that I might have missed. I would appreciate hearing
your opinion on the proposed protocol.
Thanks,
Daniil Zakhlystov
ol for 'switchable' and 'direction-specific'. It may be
added in the future.
Basically, this is what I’ve meant in my previous message about the future
expansion of the current design, I hope that I managed to clarify it.
Thanks,
Daniil Zakhlystov
> On Nov 24, 2020,
hat it is a good idea to differentiate these two cases. Maybe define
the following behavior somewhere in docs:
“When connecting to an older backend, which does not support compression, or in
the case when the backend support compression but for some reason wants to
disable it, the backend will just ignore the _pq_.compression parameter and
won’t send the compressionAck message to the frontend.”
To sum up, I think that the current implementation already introduces good
benefits. As I proposed in the Usability review, we may introduce the new
approaches later as separate compression 'algorithms'.
Thanks,
Daniil Zakhlystov
for future implementation?
Thanks,
Daniil Zakhlystov
06.11.2020, 11:58, "Andrey Borodin" :
>> 6 нояб. 2020 г., в 00:22, Peter Eisentraut
>> :
>>
>> On 2020-11-02 20:50, Andres Freund wrote:
>>> On 2020-10-31 22:25:36 +0500, Andrey Borodin wrote:
&g
pport this approach in the future, maybe we should add something like the compression mode to protocol and name the current approach as “permanent” while reserving the “switchable” compression type for future implementation? Thanks, Daniil Zakhlystov 06.11.2020, 11:58, "Andrey Borodin" : 6 но
re is any buffered rx data before calling pqWait() to avoid such hangs.--Daniil Zakhlystov
rx_func() but haven't been decompressed yet, in any other case zpq_buffered_rx() returns zero. In this case, I think that we may also rename the existing zpq_buffered() to zpq_buffered_tx() for clarity.--Daniil Zakhlystov
It looks there was a different version of pq_getbyte_if_available on my side,
my bad.
I’ve reapplied the patch and compiler is happy now, thanks!
—
Daniil Zakhlystov
> On Nov 1, 2020, at 3:01 PM, Konstantin Knizhnik
> wrote:
>
> Hi
>
> On 01.11.2020 12:37, Daniil Zakhl
do not use Zlib
--with-zstd do not use zstd // is this correct?
Thanks
--
Daniil Zakhlystov
> On Nov 1, 2020, at 12:08 AM, Konstantin Knizhnik
> wrote:
>
> Hi
>
> On 31.10.2020 00:03, Andres Freund wrote:
>> Hi,
>>
>> On 2020-10-29 16:45:58 +0300,
q_rx_func)secure_read, MyProcPort);
>> +}
>> +return 0;
>> +}
>
> Why is zpq a wrapper around secure_write/read? I'm a bit worried this
> will reduce the other places we could use zpq.
Maybe we can just split the PqStream into PqCompressStream and
PqDecompressStream? It looks like that they can work independently.
—
Daniil Zakhlystov
psql: error: could not connect to server: expected authentication request from
server, but received x // FAIL
--
Daniil Zakhlystov
24 matches
Mail list logo
