On Thu, Jan 30, 2020 at 8:59 AM DL Neil via Python-list <python-list@python.org> wrote: > * NB I don't use SQLite (in favor of going 'full-fat') and thus cannot > vouch for its behavior under load/queuing mechanism/concurrent > accesses... but I'm biased and probably think/write SQL more readily > than Python - oops!
I don't use SQLite either, and I always have a PostgreSQL database around that I can use. That said, though, I believe SQLite is fine in terms of reliability; the reason it's a bad choice for concurrency is that it uses large-scale locks to ensure safety, which means that multiple writers will block against each other. But that's fine for this use-case. So my recommendations would be: 1) Something stateless, or where the state is intrinsic to the downloaded files 2) Or failing that, use a database rather than a flat file for your state. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
