Hi Robert, On Mon, May 5, 2025 at 8:07 AM Robert Haas <robertmh...@gmail.com> wrote: > I don't understand why we need this. I don't see why we need any sort > of generalized concept of metadata at all here. The zstd-dict > compression method needs to store a four-byte OID, so let it do that. > But we don't need to brand that as metadata; and we don't need a > method for other parts of the system to ask how much metadata exists. > At least, I don't think we do. >
Thank you for the feedback. My intention in introducing the
toast_cmpid_meta_size helper to centralize header-size computation
across all compression algorithms and to provide generic macros that
can be applied to any extended algorithm methods.
I agree that algorithm-specific metadata details or its sizes need not
be exposed beyond their own routines. Each compression method
inherently knows its layout requirements and should handle them
internally in their routines. I’ve removed the toast_cmpid_meta_size
helper and eliminated the metadata branding.
In the varatt_cmp_extended, the cmp_data field carries the algorithm
payload: for zstd-nodict, it’s a ZSTD frame; for zstd-dict, it’s a
four-byte dictionary OID followed by the ZSTD frame. This approach
ensures the algorithm's framing is fully self-contained in its
routines.
/*
* varatt_cmp_extended: an optional per-datum header for extended
compression method.
* Only used when va_tcinfo’s top two bits are “11”.
*/
typedef struct varatt_cmp_extended
{
uint8 cmp_alg;
char cmp_data[FLEXIBLE_ARRAY_MEMBER];
} varatt_cmp_extended;
I’ve updated patch v21, please review it and let me know if you have
any questions or feedback? Thank you!
v21-0001-varattrib_4b-design-proposal-to-make-it-extended.patch:
varattrib_4b extensibility – adds varatt_cmp_extended, useful macros;
behaviour unchanged.
v21-0002-zstd-nodict-compression.patch: Plain ZSTD (non dict) support
and few basic tests.
--
Nikhil Veldanda
v21-0002-zstd-nodict-compression.patch
Description: Binary data
v21-0001-varattrib_4b-design-proposal-to-make-it-extended.patch
Description: Binary data
