On Fri, May 17, 2019 at 2:34 PM Andres Freund <and...@anarazel.de> wrote: > Yea, Robert is also working on this. In fact, we were literally chatting > about it a few minutes ago. He'll probably chime in too.
Yeah, I'm aiming to post a patch set very soon that does a bunch of refactoring of the TOAST stuff to make life easier for new AMs - maybe today, or else Monday. I think your use case of wanting to suppress TOAST table creation altogether is a valid one, but I want to do go a little further and make it easier for non-heap AMs to implement heap-like toasting based on their own page format. Generally, I would say that the state of play with respect to table AMs and toasting is annoyingly bad right now: - If your AM uses some system other than TOAST to store large values, you are out of luck. You will get TOAST tables whether you want them or not. - If your AM uses some page or tuple format that results in a different maximum tuple size, you are out of luck. You will get TOAST tables based on whether a heap table with the same set of columns would need one. - If your AM would like to use the heap for TOAST data, you are out of luck. The AM used for TOAST data must be the same as the AM used for the main table. - If your AM would like to use itself to store TOAST data, you are also out of luck, because all of the existing TOAST code works with heap tuples. - Even if you copy all of tuptoaster.c/h - which is a lot of code - and change everything that is different for your AM than for the regular heap, you are still out of luck, because code that knows nothing about tableam is going to call heap_tuple_untoast_attr() to detoast stuff, and that code is only going to be happy if you've used the same chunk size that we use for the regular heap, and that chunk size has a good chance of being mildly to severely suboptimal if your heap has made any sort of page format changes. So I think this basically just doesn't work right now. I am sympathetic to Andres's position that we shouldn't go whacking the code around too much at this late date, and he's probably right that we're going to find lots of other problems with tableam as well and you have to draw the line someplace, but on the other hand given your experience and mine, it's probably pretty likely that anybody who tries to use tableam for anything is going to run into this problem, so maybe it's not crazy to think about a few last-minute changes. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
