On Thu, Jun 20, 2019 at 1:51 AM Binguo Bao <djydew...@gmail.com> wrote: > Hi hackers! > This proposal aims to provide the ability to de-TOAST a fully TOAST'd and > compressed field using an iterator and then update the appropriate parts of > the code to use the iterator where possible instead of de-TOAST'ing and > de-compressing the entire value. Examples where this can be helpful include > using position() from the beginning of the value, or doing a pattern or > substring match. > > de-TOAST iterator overview: > 1. The caller requests the slice of the attribute value from the de-TOAST > iterator. > 2. The de-TOAST iterator checks if there is a slice available in the output > buffer, if there is, return the result directly, > otherwise goto the step3. > 3. The de-TOAST iterator checks if there is the slice available in the input > buffer, if there is, goto step44. Otherwise, > call fetch_datum_iterator to fetch datums from disk to input buffer. > 4. If the data in the input buffer is compressed, extract some data from the > input buffer to the output buffer until the caller's > needs are met. > > I've implemented the prototype and apply it to the position() function to > test performance.
Hi Binguo, Interesting work, and nice performance improvements so far. Just by the way, the patch currently generates warnings: https://travis-ci.org/postgresql-cfbot/postgresql/builds/554345719 -- Thomas Munro https://enterprisedb.com
