I'd love to learn more about caching too. Great info and questions - looking forward to the response.
On Mon, Jul 1, 2013 at 9:25 PM, Aristedes Maniatis <a...@ish.com.au> wrote: > Hi Andrus > > I know you have been looking at this offline, but I thought I'd raise this > here in public to see if others can also help shed light on a difficult > area to completely understand in Cayenne. Perhaps this conversation can > also help us improve the docs. I'd like to better understand the > interactions between the DataRowStore and the ObjectStore. > > Here is how I understand the Cayenne object cache: > > 1. There is a checkbox in the modeler at the top level for "use shared > cache". If that checkbox is ticked, then a single DataRowStore (a LRU map) > is created, shared between all contexts. If not, a separate DataRowStore is > created every time you create a new context. > > 2. Each context has its own ObjectStore for keeping the actual objects > (the snapshot), and another copy of the object with any changes made in > that context. Any queries to the database fill the DataRowStore first, and > then the objects are constructed in to the ObjectStore. > > 3. When a commit is made to Context A, the ObjectStore linked to context A > has a bunch of changes in it. Those changes are pushed into the > DataRowStore, and then into the database. Events are are sent to update the > snapshot copy of objects held in all other ObjectStores. > > 4. These events can even be sent to another Java VM, but we'll ignore that > for now. > > 5. The modeler has a field called "Size of object cache". This refers to > the size of the DataRowStore map as a count of objects. Because the rows > could be 1kB or 1Mb each, it is impossible to predict how large this cache > will grow to, but it will get bigger and smaller over time. > > 6. If "use shared cache" is NOT ticked, then this size is used for *each* > of the DataRowStore maps. This means that the memory usage will grow very > rapidly as you add new contexts and fill them with data. 100 contexts, each > loaded with a query that fetches the same 10Mb of data, will contain 1000Mb > of DataRowStore memory usage. > > 6. If a DataRow is not held in the DataRowStore (eg, because it was pushed > out of the map by newer data) then an event is NOT sent to the other > contexts to notify them that the object was updated when context A is > committed. Instead stale data is left in those other contexts. > > 7. Assuming the query cache is disabled, a query against the database will > fetch brand new objects into the DataRowStore and potentially update all > the snapshots of all ObjectStores which contain a copy of those records. > > 8. A relationship join (painting.getArtist()) might find a record in the > DataRowStore and pull it into the ObjectStore without ever going back to > the database. > > 9. If the query cache is hit, then the results may already be found in the > DataRowStore. If so, the the ObjectStore snapshot is updated from these > DataRows if any of those objects are not already found in the ObjectStore. > > > So, my questions: > > A. What is the use case of separate DataRowStores per context? It appears > to be a worse possible scenario in every case I can think of, with no > upside. The label "use shared cache" gives the impression that you are > trading memory usage for staleness of data, but that does not appear to be > the case. > > B. Is it an intrinsic feature of this system that a cache miss in the > DataRowStore causes no event to be sent to other ObjectStores, or is that a > bug? This means that setting the 'size of object cache' in the modeler to 0 > results in a lot of stale data. > > C. Will a cache miss in the DataRowStore cause Cayenne to refetch that > object from the database before it then tries to commit the changes to that > object. > > > > Have I captured the above notes correctly? > > > Cheers > Ari > > > > > > -- > --------------------------> > Aristedes Maniatis > ish > http://www.ish.com.au > Level 1, 30 Wilson Street Newtown 2042 Australia > phone +61 2 9550 5001 fax +61 2 9550 4001 > GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A >
