Le 11/03/2021 à 19:54, Micah Kornfield a écrit :
Indeed, I don't think it was discussed publicly. The LZ4 frame format
has several things going for it:
- it allows streaming compression and decompression (meaning you can
avoid loading a huge compressed buffer at once)
Is this something we make use of or intend to make use of?
Good question. Currently we don't. Perhaps David Li wants to answer
this, since he's been working a lot on Flight.
- it embeds the decompressed size, allowing exact allocation of the
decompressed buffer
IIUC, We already do this in the IPC specification (the first 8 bytes of the
compressed buffer are used for this).
Ah, you're right. It doesn't matter then.
- it has an optional checksum
This seems like a good thing, so probably worth keeping (although it would
be the only place where we do checksums today).
(or of course we could add an optional higher-level checksum in the IPC
format)
Regards
Antoine.
