On Sun, 26 Jan 2025 10:48:48 -0800 Sharvil Nanavati <shar...@lmnt.com> wrote: > In a different context, fetching batches one-by-one would be a good way to > control when the disk read takes place. > > In my context, I'm looking for a way to construct a Table without > performing the bulk of the IO operations until the memory is accessed. I > need random access to the table and my accesses are often sparse relative > to the size of the entire table. Obviously there has to be *some* IO to > read the schema and offsets, but that's tiny relative to the data itself.
Then you should just use a memory-mapped file. Regards Antoine.
