On Wed, May 7, 2025 at 5:38 PM Michael Paquier wrote:
> Yes, I was wondering if this is not the most natural approach in terms
> of structure once if we plug an extra byte into the varlena header if
> all the bits of va_extinfo for the compression information are used.
> Having all the bits may no
Hi Michael, Thanks for the feedback.
On Wed, May 7, 2025 at 12:49 AM Michael Paquier wrote:
>
> I have been reading 0001 and I'm finding that the integration does not
> seem to fit much with the existing varatt_external, making the whole
> result slightly confusing. A simple thing: the last bit
Hi Robert,
On Mon, May 5, 2025 at 8:07 AM Robert Haas wrote:
> I don't understand why we need this. I don't see why we need any sort
> of generalized concept of metadata at all here. The zstd-dict
> compression method needs to store a four-byte OID, so let it do that.
> But we don't need to brand
-0001-varattrib_4b-design-proposal-to-make-it-extended.patch:
varattrib_4b extensibility – adds varatt_cmp_extended, metadata size
dispatch and useful macros; behaviour unchanged.
v20-0002-zstd-nodict-compression.patch: Plain ZSTD (non dict) support.
--
Nikhil Veldanda
From dc6172c69453546340
on this design. Thanks!
On Mon, Apr 28, 2025 at 7:50 AM Robert Haas wrote:
>
> On Fri, Apr 25, 2025 at 11:15 AM Nikhil Kumar Veldanda
> wrote:
> > a. 24 bits for length → per-datum compression algorithm metadata is
> > capped at 16 MB, which is far more than any realistic c
Hi Michael,
Thanks for the suggestions. I agree that we should first solve the
“last–free-bit” problem in varattrib_4b compression bits before
layering on any features. Below is the approach I’ve prototyped to
keep the header compact yet fully extensible, followed by a sketch of
the plain-ZSTD(no
Hi Michael,
Thanks for the feedback and the suggested patch sequence. I completely
agree—we must minimize storage overhead when dictionaries aren’t used,
while ensuring varattrib_4b remains extensible enough to handle future
compression metadata beyond dictionary ID (for other algorithms). I’ll
ex
te:
>
> On Tue, Apr 15, 2025 at 2:13 PM Nikhil Kumar Veldanda
> wrote:
> > Addressing Compressed Datum Leaks problem (via CTAS, INSERT INTO ... SELECT
> > ...)
> >
> > As compressed datums can be copied to other unrelated tables via CTAS,
> > INSERT I
Hi,
I reviewed the discussions, and while most agreements focused on
changes to the toast pointer, the design I propose requires no
modifications to it. I’ve carefully considered the design choices made
previously, and I recognize Zstd’s clear advantages in compression
efficiency and performance o
Hi Tom,
On Thu, Mar 6, 2025 at 11:33 AM Tom Lane wrote:
>
> Robert Haas writes:
> > On Thu, Mar 6, 2025 at 12:43 AM Nikhil Kumar Veldanda
> > wrote:
> >> Notably, this is the first compression algorithm for Postgres that can
> >> make use of a di
Hi
On Thu, Mar 6, 2025 at 5:35 AM Aleksander Alekseev
wrote:
>
> Hi Nikhil,
>
> Many thanks for working on this. I proposed a similar patch some time
> ago [1] but the overall feedback was somewhat mixed so I choose to
> focus on something else. Thanks for peeking this up.
>
> > test=# select bui
Hi Robert,
> I think that solving the problems around using a dictionary is going
> to be really hard. Can we see some evidence that the results will be
> worth it?
With the latest patch I've shared,
Using a Kaggle dataset of Nintendo-related tweets[1], we leveraged
PostgreSQL's acquire_sample_r
Hi,
> Overall idea is great.
>
> I just want to mention LZ4 also have API to use dictionary. Its dictionary
> will be as simple as "virtually prepended" text (in contrast to complex
> ZStd dictionary format).
>
> I mean, it would be great if "dictionary" will be common property for
> different alg
Hi Yura,
> So, to support "super-fast" mode you have to accept negative compression
> levels. I didn't check, probably you're already support them?
>
The key point I want to emphasize is that both zstd compression levels
and dictionary size should be configurable based on user preferences
at attr
14 matches
Mail list logo
