On Sat, Dec 5, 2015 at 12:10 AM, Simon Riggs <si...@2ndquadrant.com> wrote: > On 4 December 2015 at 13:47, Ildus Kurbangaliev > <i.kurbangal...@postgrespro.ru> wrote: > >> >> Attached patch adds support of partial decompression for datums. >> It will be useful in many cases when extracting part of data is >> enough for big varlena structures. >> >> It is especially useful for expanded datums, because it provides >> storage for partial results. > > > This isn't enough for anyone else to follow your thoughts and agree enough > to commit. > > Please explain the whole idea, starting from what problem you are trying to > solve and how well this does it, why you did it this way and the other ways > you tried/decided not to pursue. Thanks.
Yeah, I would imagine that what Ildus is trying to achieve is something close to LZ4_decompress_safe_partial, by being able to stop compression after getting a certain amount of data decompressed, and continue working once again after. And actually I think I get the idea. With his test case, what we get first is a size, and then we reuse this size to extract only what we need to fetch only a number of items from the tsvector. But that's actually linked to the length of the compressed chunk, and at the end we would still need to decompress the whole string perhaps, but it is not possible to be sure using the information provided. Ildus, using your patch for tsvector, are you aiming at being able to complete an operation by only using a portion of the compressed data? Or are you planning to use that to improve the speed of detection of corrupted data in the chunk? If that's the latter, we would need to still decompress the whole string anyway, so having a routine able to decompress only until a given position is not necessary, and based on the example given upthread it is not possible to know what you are trying to achieve. Hence could you share your thoughts regarding your stuff with tsvector? Changing pglz_decompress shape is still a bad idea anyway, I guess we had better have something new like pglz_decompress_partial instead. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
