Working with the dbm module (using it as a cache), I've gotten the
following error at least twice now:
HASH: Out of overflow pages. Increase page size
Traceback (most recent call last):
[snip]
File ".py", line 83, in get_data
db[key] = data
_dbm.error: cannot add item to database
I've read over the py3 docs on dbm
https://docs.python.org/3/library/dbm.html
but don't see anything about either "page" or "size" contained
therein.
There's nothing particularly complex as far as I can tell. Nothing
more than a straightforward
import dbm
with dbm.open("cache", "c") as db:
for thing in source:
key = extract_key_as_bytes(thing)
if key in db:
data = db[key]
else:
data = long_process(thing)
db[key] = data
The keys can get a bit large (think roughly book-title length), but
not huge. I have 11k records so it seems like it shouldn't be
overwhelming, but this is the second batch where I've had to nuke the
cache and start afresh. Fortunately I've tooled the code so it can
work incrementally and no more than a hundred or so requests have to
be re-performed.
How does one increas the page-size in a dbm mapping? Or are there
limits that I should be aware of?
Thanks,
-tkc
PS: FWIW, this is Python 3.6 on FreeBSD in case that exposes any
germane implementation details.
--
https://mail.python.org/mailman/listinfo/python-list
