On Tue, Dec 21, 2021 at 10:06 PM David Gauthier <davegauthie...@gmail.com> wrote:
> I'll have to read more about sqlite_fdw. Thanks for that Steve ! > > Each SQLite isn't that big (billions of records), more like 30K records or > so. But there are lots and lots of these SQLite DBs which add up over time > to perhaps billions of records. > > This is for a big corp with an IT dept. Maybe I can get them to upgrade > the DB itself. > Thank You too David ! > >> >> So, more similar to the image storage question than I first thought, but still large enough where the specific usage patterns and needs end up being the deciding factor (keeping in mind you can pick multiple solutions - so that really old data, ideally on a partition, can be removed from the DB while still remaining accessible if just more slowly or laboriously). One possibility to consider - ditch the SQLite dependency and just store CSV (but maybe with a funky delimiter sequence). You can then us "string_to_table(...)" on that delimiter to materialize a table out of the data right in a query. David J.
