Hi,
I reviewed the discussions, and while most agreements focused on
changes to the toast pointer, the design I propose requires no
modifications to it. I’ve carefully considered the design choices made
previously, and I recognize Zstd’s clear advantages in compression
efficiency and performance o
Hi,
> Overall idea is great.
>
> I just want to mention LZ4 also have API to use dictionary. Its dictionary
> will be as simple as "virtually prepended" text (in contrast to complex
> ZStd dictionary format).
>
> I mean, it would be great if "dictionary" will be common property for
> different alg
Hi Robert,
> I think that solving the problems around using a dictionary is going
> to be really hard. Can we see some evidence that the results will be
> worth it?
With the latest patch I've shared,
Using a Kaggle dataset of Nintendo-related tweets[1], we leveraged
PostgreSQL's acquire_sample_r
Hi
On Thu, Mar 6, 2025 at 5:35 AM Aleksander Alekseev
wrote:
>
> Hi Nikhil,
>
> Many thanks for working on this. I proposed a similar patch some time
> ago [1] but the overall feedback was somewhat mixed so I choose to
> focus on something else. Thanks for peeking this up.
>
> > test=# select bui
Hi Tom,
On Thu, Mar 6, 2025 at 11:33 AM Tom Lane wrote:
>
> Robert Haas writes:
> > On Thu, Mar 6, 2025 at 12:43 AM Nikhil Kumar Veldanda
> > wrote:
> >> Notably, this is the first compression algorithm for Postgres that can
> >> make use of a di
Hi Yura,
> So, to support "super-fast" mode you have to accept negative compression
> levels. I didn't check, probably you're already support them?
>
The key point I want to emphasize is that both zstd compression levels
and dictionary size should be configurable based on user preferences
at attr
Hi Michael,
Thanks for the feedback and the suggested patch sequence. I completely
agree—we must minimize storage overhead when dictionaries aren’t used,
while ensuring varattrib_4b remains extensible enough to handle future
compression metadata beyond dictionary ID (for other algorithms). I’ll
ex
te:
>
> On Tue, Apr 15, 2025 at 2:13 PM Nikhil Kumar Veldanda
> wrote:
> > Addressing Compressed Datum Leaks problem (via CTAS, INSERT INTO ... SELECT
> > ...)
> >
> > As compressed datums can be copied to other unrelated tables via CTAS,
> > INSERT I
on this design. Thanks!
On Mon, Apr 28, 2025 at 7:50 AM Robert Haas wrote:
>
> On Fri, Apr 25, 2025 at 11:15 AM Nikhil Kumar Veldanda
> wrote:
> > a. 24 bits for length → per-datum compression algorithm metadata is
> > capped at 16 MB, which is far more than any realistic c
Hi Michael,
Thanks for the suggestions. I agree that we should first solve the
“last–free-bit” problem in varattrib_4b compression bits before
layering on any features. Below is the approach I’ve prototyped to
keep the header compact yet fully extensible, followed by a sketch of
the plain-ZSTD(no
-0001-varattrib_4b-design-proposal-to-make-it-extended.patch:
varattrib_4b extensibility – adds varatt_cmp_extended, metadata size
dispatch and useful macros; behaviour unchanged.
v20-0002-zstd-nodict-compression.patch: Plain ZSTD (non dict) support.
--
Nikhil Veldanda
From dc6172c69453546340
Hi Robert,
On Mon, May 5, 2025 at 8:07 AM Robert Haas wrote:
> I don't understand why we need this. I don't see why we need any sort
> of generalized concept of metadata at all here. The zstd-dict
> compression method needs to store a four-byte OID, so let it do that.
> But we don't need to brand
Hi Michael, Thanks for the feedback.
On Wed, May 7, 2025 at 12:49 AM Michael Paquier wrote:
>
> I have been reading 0001 and I'm finding that the integration does not
> seem to fit much with the existing varatt_external, making the whole
> result slightly confusing. A simple thing: the last bit
13 matches
Mail list logo
