On Mon, 2008-06-16 at 15:35 -0400, Tom Lane wrote: > Recent discussions with the PostGIS hackers led me to think about ways > to reduce overhead when the same TOAST value is repeatedly detoasted. > In the example shown here > http://archives.postgresql.org/pgsql-hackers/2008-06/msg00384.php > 90% of the runtime is being consumed by repeated detoastings of a single > datum. That is certainly an outlier case, but we've heard complaints of > TOAST being slow before. The solution I'm about to propose should fix > this, and as a bonus it will reduce the problem of memory leakage when > a detoasted value doesn't get pfreed. > > What I am imagining is that the tuple toaster will maintain a cache of > recently-detoasted values...
> Comments, better ideas? Anyone think this is too much trouble to take > for the problem? You've not covered the idea that we just alter the execution so we just detoast once. If we tried harder to reduce the number of detoastings then we would benefit all of the cases you mention, including internal decompression. We would use memory, yes, but then so would a cache of recently detoasted values. If we see that the index scan key is toastable/ed then the lowest levels of the plan can create an expanded copy of the tuple and pass that upwards. We may need to do this in a longer lived context and explicitly free previous tuples to avoid memory bloat, but we'd have the same memory usage and same memory freeing issues as with caching. It just seems more direct and more obvious, especially since it is just an internal version of the workaround, which was to create a function to perform early detoasting. Maybe this could be done inside the IndexScan node when a tuple arrives with toasted values(s) for the scan key attribute(s). I presume there's various reasons why you've ruled that out, but with such a complex proposal it seems worth revisiting the alternatives, even if just to document them for the archives. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
