Hi! I'm working on this issue according to the plan Tom proposed above -
>I agree that we can't simply widen varatt_external to use 8 bytes for >the toast ID in all cases. Also, I now get the point about avoiding >use of globally assigned OIDs here: if the counter starts from zero >for each table, then a variable-width varatt_external could actually >be smaller than currently for many cases. However, that bit is somewhat >orthogonal, and it's certainly not required for fixing the basic problem. Have I understood correctly that you suppose using an individual counter for each TOAST table? I'm working on this approach, so we store counters in cache, but I see an issue with the first insert - when there is no counter in cache so we have to loop through the table with increasing steps to find available one (i.e. after restart). Or we still use single global counter, but 64-bit with a wraparound? >So it seems like the plan of attack ought to be: >1. Invent a new form or forms of varatt_external to allow different >widths of the toast ID. Use the narrowest width possible for any >given ID value. I'm using the VARTAG field - there are plenty of available values, so there is no problem in distinguishing regular toast pointer with 'short' value id (4 bytes) from long (8 bytes). varatt_external currently is 32-bit aligned, so there is no reason in using narrower type for value ids up to 16 bits.Or is it? >2. Allow TOAST tables/indexes to store either 4-byte or 8-byte IDs. >(Conversion could be done as a side effect of table-rewrite >operations, perhaps.) Still on toast/detoast part, would get to that later. >3. Localize ID selection so that tables can have small toast IDs >even when other tables have many IDs. (Optional, could do later.) > Thank you! -- Regards, Nikita Malakhov Postgres Professional https://postgrespro.ru/
