Hello Elephants,
As I mentioned to Robert the other day, writing a manual is a drag
but it does force you to visit all the dark corners of your system
and make sure you really understand what is going on. While writing
the Berkeley DB internals section today I discovered an interesting
fact.
The default Berkeley DB memory cache size is 256k bytes. That is 64
4k pages and each page has about 100 key-value pairs for small key
and value size (20 bytes). That is a cache big enough for 6,400 key-
value pairs. If you regularly walk over a full index, you will
certainly thrash your cache. So if you have many thousands of
objects, with several indices, your working set could easily exceed
this value.
For example, if you had an indexed class with two slots and two
indices on those slots, each object instance would eat up 3 small key-
value pairs plus the number of persistent slots per object. So you
could cache the values and indexes for a little over 1k objects.
I've added a user parameter that allows you to tune this value. I
also set the default value to 10MB for each instance of a data store
(most applications use 1 or a very small number). This is more than
large enough for most casual applications (10's of thousands of
objects, including indices) but small enough to not be a big deal
with today's generous memory systems. However, if you want to change
it, you can use the user parameter :berkeley-db-cachesize in the my-
config.sexp configuration file. Existing database will not be
effected by changes in this parameter, only new ones. (Advanced
users can delete the environment files of the database, but leave the
database intact to upgrade to a new value)
I also added a :map-using-degree2 command that takes a boolean
parameter and tells the Berkeley DB data store whether to allow
btrees that are being mapped to also be read by other transactions.
The implicates are that two processes can concurrently read an index
and commit without one aborting the other because of conflicting read
locks when each one writes different key-value pairs. This defaults
to true and is primarily a performance enhancing option, but if you
require that only one write can happen to a btree at a time, you can
disable this by default.
If you want to specialize a given map operation's transaction, you
can wrap it in a with-transaction that uses different parameters
(:read-uncommited, :degree-2, etc). The new manual documents these
options in Chapter 4, the BDB overview.
Cheers,
Ian
_______________________________________________
elephant-devel site list
elephant-devel@common-lisp.net
http://common-lisp.net/mailman/listinfo/elephant-devel
