Hi, On 2019-05-17 15:13:50 -0400, Robert Haas wrote: > On Fri, May 17, 2019 at 2:34 PM Andres Freund <and...@anarazel.de> wrote: > > Yea, Robert is also working on this. In fact, we were literally chatting > > about it a few minutes ago. He'll probably chime in too. > > Yeah, I'm aiming to post a patch set very soon that does a bunch of > refactoring of the TOAST stuff to make life easier for new AMs - maybe > today, or else Monday. I think your use case of wanting to suppress > TOAST table creation altogether is a valid one, but I want to do go a > little further and make it easier for non-heap AMs to implement > heap-like toasting based on their own page format. > > Generally, I would say that the state of play with respect to table > AMs and toasting is annoyingly bad right now: > > - If your AM uses some system other than TOAST to store large values, > you are out of luck. You will get TOAST tables whether you want them > or not.
Which is aesthetically and indode usage wise annoying, but not *terrible*. You get a a bunch of useless pg_class/pg_index entries and a few close-to-empty relfilenodes. > - If your AM uses some page or tuple format that results in a > different maximum tuple size, you are out of luck. You will get TOAST > tables based on whether a heap table with the same set of columns > would need one. > - If your AM would like to use the heap for TOAST data, you are out of > luck. The AM used for TOAST data must be the same as the AM used for > the main table. > > - If your AM would like to use itself to store TOAST data, you are > also out of luck, because all of the existing TOAST code works with > heap tuples. > > - Even if you copy all of tuptoaster.c/h - which is a lot of code - > and change everything that is different for your AM than for the > regular heap, you are still out of luck, because code that knows > nothing about tableam is going to call heap_tuple_untoast_attr() to > detoast stuff, and that code is only going to be happy if you've used > the same chunk size that we use for the regular heap, and that chunk > size has a good chance of being mildly to severely suboptimal if your > heap has made any sort of page format changes. Well, I don't *quite* buy the suboptimal. As far as I can tell, the current chunking size doesn't have much going for it for heap either - and while a few people (me including) have complained about it, it's not that many people either. My impression is that the current chunking is essentially a randomly chosen choice without much to go for it, and so it's not going to be much different for other AMs. > So I think this basically just doesn't work right now. I mean, the zheap on tableam code copies more toast code than I'm happy about, and it's chunking is somewhat suboptimal, but that's not *terrible*. There's no if(zheap) branches outside of zheap related to toasting. Greetings, Andres Freund
