On Sun, Mar 21, 2021 at 07:11:50PM -0400, Tom Lane wrote: > I wrote: > > Justin Pryzby <pry...@telsasoft.com> writes: > >> On Sun, Mar 21, 2021 at 04:32:31PM -0400, Tom Lane wrote: > >>> This seems somewhat repeatable (three identical failures in three > >>> attempts). Not sure why I did not see it yesterday; but anyway, > >>> there is something wrong with partial detoasting for LZ4. > > >> With what version of LZ4 ? > > > RHEL8's, which is > > lz4-1.8.3-2.el8.x86_64 > > I hate to be the bearer of bad news, but this suggests that > LZ4_decompress_safe_partial is seriously broken in 1.9.2 > as well: > > https://github.com/lz4/lz4/issues/783
Ouch > Maybe we cannot rely on that function for a few more years yet. > > Also, I don't really understand why this code: > > /* slice decompression not supported prior to 1.8.3 */ > if (LZ4_versionNumber() < 10803) > return lz4_decompress_datum(value); > > It seems likely to me that we'd get a flat out build failure > from library versions lacking LZ4_decompress_safe_partial, > and thus that this run-time test is dead code and we should > better be using a configure probe if we intend to allow old > liblz4 versions. Though that might be moot. The function existed before 1.8.3, but didn't handle slicing. https://github.com/lz4/lz4/releases/tag/v1.8.3 |Finally, an existing function, LZ4_decompress_safe_partial(), has been enhanced to make it possible to decompress only the beginning of an LZ4 block, up to a specified number of bytes. Partial decoding can be useful to save CPU time and memory, when the objective is to extract a limited portion from a larger block. Possibly we could allow v >= 1.9.3 || (ver >= 1.8.3 && ver < 1.9.2). Or maybe not: the second half apparently worked "by accident", and we shouldn't need to have intimate knowledge of someone else's patchlevel releases, -- Justin
