Buffer text is not a good data transfer protocol for moving return values from babel block to block. It is also not a good storage medium for table data. I propose we implement more binary data passing strategies.
Elisp memory is it's own source of problems, so what would likely work better is a dynamic module to store values, like a binary cache for data. Don't quote me. I didn't do a three year design analysis on this. Within the org document, such a cache would be implemented by returning truncated results to Elisp for display in an overlay. Table display can be handled much the same way. The user sees some data. Everything runs basically the same and with a similar user experience, but wildly faster. In fact, going the dynamic module route paves the way for displaying tables outside of Emacs display, getting around the limitations that make multi-line cells etc problematic. Such a table viewer could operate its own frames, support tabs etc, and bring Org much, much closer to a traditional spreadsheet application in terms of capability. But first things first, binary data passing between babel blocks and avoiding serialization for languages where the runtime can be convinced to do it should have been done two decades ago on POSIX. Next best time is now, using 2025 tools.
