Hi hackers! Jacob, I agree that the bytea toaster makes a bad example due to core modification, and actually is not a good example of an extension.
The vtable concept is intended for less invasive additional functionality - like providing detoast iterators in addition to standard detoast mechanics - such modification requires only adding iteration methods to toaster and registering them in vtable, without any core modifications. I'll add this as a separate commit for generic (default) Toaster. It would be more clear for complex data types like JSONB, where developers could need some additional functionality to work with internal representation of data type, and its full potential is revealed in our JSONB toaster extension. The JSONB toaster is still in development but we plan to make it available soon. For example, we can pass Toaster options with attoptions (I'm currently working on it) and these options could, say, allow switching different optimizations in one toaster like adding specific compression options or data processing directives, etc. We doubt that there would be a lot of different custom toasters, because the Toaster is quite a complex piece of machinery, but means for extending them would be heavily demanded. I have to add some more in-depth explanation of the vtable concept to README and the documentation package, the dummy toaster contrib does not cover this topic at all. On installing dummy_toaster contrib: I've just checked it by making a patch from commit and applying onto my clone of master and 2 patches provided in previous email without any errors and sll checks passed - applying with git am, configure with debug, cassert, depend and enable-tap-tests flags and run checks. Please advice what would cause such a behavior? Thank you! On Tue, Sep 13, 2022 at 12:39 AM Jacob Champion <jchamp...@timescale.com> wrote: > On Wed, Aug 24, 2022 at 2:59 AM Nikita Malakhov <huku...@gmail.com> wrote: > > I've rebased actual branch onto the latest master and re-created > patches. Checked with git am, > > all applied correctly. Please check the attached patches. > > Rebased branch resides here: > > https://github.com/postgrespro/postgres/tree/toasterapi_clean > > I tried installing and using the dummy_toaster that's provided with > the gitlink. Upgrade of that cluster fails with the following message: > > pg_restore: creating TOASTER "dummy_toaster" > pg_restore: while PROCESSING TOC: > pg_restore: from TOC entry 2044; 9861 16390 TOASTER dummy_toaster (no > owner) > pg_restore: error: could not execute query: ERROR: unrecognized > or unsupported class OID: 9861 > Command was: CREATE TOASTER "dummy_toaster" HANDLER > "public"."dummy_toaster_handler"; > > I was looking through the thread for a more in-depth description of > the "vtable" concept, but I didn't see one. It looks like it's just an > arbitrary extension point, and any new additions would require surgery > on whatever function needs the particular magic provided by the > toaster. E.g. your bytea-append toaster extension in the gitlink, > which still has to modify byteacat() in varlena.c to implement a very > specific optimization, and then declares its support for that > hardcoded optimization in the extension. > > I'm skeptical that this would remain coherent as it grows. The patch > claims the vtable API is "powerful", which... I suppose it is, if you > get to make arbitrary modifications to the core whenever you implement > it. Did you already have thoughts about which operations would belong > under that umbrella? What would the procedure be for adding > functionality to that API? What happens if a toaster wants to > implement two magic performance optimizations instead of one? > > --Jacob > -- Regards, Nikita Malakhov Postgres Professional https://postgrespro.ru/
