Hi!
> On Jan 4, 2021, at 11:04 AM, Andrey Borodin <x4...@yandex-team.ru> wrote:
>
> Daniil, is levels definition compatible with libpq compression patch?
> +typedef struct Compress {
> + CompressionAlgorithm alg;
> + int level;
> + /* Is a nondefault level set ? This is useful since different
> compression
> + * methods have different "default" levels. For now we assume the
> levels
> + * are all integer, though.
> + */
> + bool level_set;
> +} Compress;
Similarly to this patch, it is also possible to define the compression level at
the initialization stage in libpq compression patch.
The difference is that in libpq compression patch the default compression level
always equal to 1, independently of the chosen compression algorithm.
> On Jan 4, 2021, at 11:04 AM, Andrey Borodin <x4...@yandex-team.ru> wrote:
>
> Libpq compression encountered some problems with memory consumption which
> required some extra config efforts.
> On Jan 4, 2021, at 12:06 PM, Justin Pryzby <pry...@telsasoft.com> wrote:
>
> RAM use is not significantly different from zlib, except that zstd --long adds
> more memory.
Regarding ZSTD memory usage:
Recently I’ve made a couple of tests of libpq compression with different
ZLIB/ZSTD compression levels which shown that compressing/decompressing ZSTD w/
high compression levels
require to allocate more virtual (Commited_AS) memory, which may be exploited
by malicious clients:
https://www.postgresql.org/message-id/62527092-16BD-479F-B503-FA527AF3B0C2%40yandex-team.ru
We can avoid high memory usage by limiting the max window size to 8MB. This
should effectively disable the support of compression levels above 19:
https://www.postgresql.org/message-id/6A45DFAA-1682-4EF2-B835-C5F46615EC49%40yandex-team.ru
So maybe it is worthwhile to use similar restrictions in this patch.
—
Daniil Zakhlystov
