Re: relationship query and cache refreshing in 3tier cayenne

[Marcin Skladaniec]

Hello




first initial check on client 1:


now the same on client 2:
performQuery(Painting).size() -> 10
getArtistWithName("Monet").getPaintings() -> 2

//create a new painting:
getArtistWithName("Monet").addPainting(newPainting());
commitChanges();

//client 2 has all the data updated, as expected:
performQuery(Painting).size() -> 11
getArtistWithName("Monet").getPaintings() -> 3

//back to client 1:
performQuery(Painting).size() -> 11                          [1]
getArtistWithName("Monet").getPaintings() -> 2             [2]


[2] - even so, the relationship is not ...



Best regards
Marcin



On 14/05/2008, at 8:46 AM, Andrus Adamchik wrote:

We have started to write the caching documentation, and we will post it to the list for verification this week. Before that I would like to clarify my previous email. From what I can see the LOCAL_CACHE refresh does propagate between clients. This is what I do in pseudo code: performQuery(Painting).size() -> 10 //fetching the entire list of Paintings using SelectQuery getArtistWithName("Monet").getPaintings() -> 2 //fetching the list of paintings using RelationshipQuery [1] - re-doing selectquery in different client seems to work! Andrus this opposes to what you have said about the RefreshQuery propagation! All select queries are executed with paging and LOCAL_CACHE. Every entity uses a separate cache key.

From what I see the problem is that RefreshQuery is not propagated to remote clients. LOCAL_CACHE is local the tier where the query originated. With some fixes in M4, this statement is true for both nested contexts and ROP... Since by default there are no events pushed from the server to the client and no peer-to-peer client communication, client 1 uses its local cache and knows nothing about the RefreshQuery executed by client 2.

There are a few solutions to that:





Andrus


On May 7, 2008, at 7:25 PM, Marcin Skladaniec wrote:

(1) set timemout on the query cache by using something like an OSCache cache provider. This will give you a delayed refresh. This may or may not work for your app, but this can be used with the current Cayenne. For many apps that would be a reasonable compromise, with cache ensuring that queries don't happen too often.. (2) Setup server-to-client event push. I did that in the past with a always-on XMPP connection in parallel with the main HTTPS data channel (I know it worked for object updates... it needs to be extended to cache groups invalidation). For a long time I wanted to investigate Jetty Continuations (and more generally, queuing events on the server for each live client, and letting client frequently poll for events). This would allow to reuse the main ROP connection, but this needs to be developed yet.

Hi



                if (!isSharedContext()) {
                        try {
                                List<Class> commitedClasses = new 
Vector<Class>();
                                for (Object o : uncommittedObjects())

                                                
commitedClasses.add(o.getClass());
                                
                                super.commitChanges();
                                
                                for (Class<? extends PersistentObject> c : 
commitedClasses) {

                                }

                        } catch (CayenneRuntimeException e) {
                                runtimeExceptionThrown(e, false);
                        }
                } else {

                }



                if (query instanceof SelectQuery) {
                                SelectQuery sq = ((SelectQuery) query);
                                if (isSharedContext) {
                                        // if the query is on the shared 
context then use cache
                                        // 
sq.setCachePolicy(QueryMetadata.LOCAL_CACHE);


                                                List<String> currentCacheGroups = new 
Vector<String>();
                                                if (sq.getCacheGroups() != null)
                                                        currentCacheGroups = 
Arrays.asList(sq.getCacheGroups());

                                                String key = 
PersistentObject.defaultCacheKeyForEntity(c);
                                                if 
(!currentCacheGroups.contains(key)) {
                                                        
currentCacheGroups.add(key);

                                                }
                                        }

                                }
                        }





1) client app 1: artists list shows:
        Monet - 5
        Malevich - 4
        vanGogh - 3
        painting list contains 12 lines
client app 2 adds a new painting for vanGogh, it lists now
        Monet - 5
        Malevich - 4
        vanGogh - 4
        painting list contains 13 lines
client app 1 lists the artists again:
        Monet - 5
        Malevich - 4
        vanGogh - 3 <- incorrect
        painting list contains 13 records, which is correct




Marcin

In our application we are using LOCAL_CACHE and cache keys to refresh it, but only for a special context called 'shared' . We have overridden the CayenneContext commitChanges with following to ensure cache invalidation on every commit : if (!commitedClasses.contains(o.getClass()) && o instanceof PersistentObject) performGenericQuery(new RefreshQuery(new String[] { PersistentObject.defaultCacheKeyForEntity(c) })); logger.error("Attempt to save shared context", new IllegalStateException("Shared context is read-only")); whenever a select query is executed we are setting the cache policy and keys // if the root class of the query is kind of PersistentObject then use the cache keys if (sq.getRoot() instanceof Class && PersistentObject.class.isAssignableFrom((Class<? extends PersistentObject>) (sq.getRoot()))) { Class<? extends PersistentObject> c = (Class<? extends PersistentObject>) sq.getRoot(); sq.setCacheGroups(currentCacheGroups.toArray(new String[] {})); This works nicely, but there is one problem: the relationship query does not return refreshed values. An example. there are two views: One is a list of artists with painting counts, the counts are calculated using relationship (anArtist.getPaintings().size()). The second is a simple list of paintings. The odd thing is also that for a given client application both lists (artist and painting) are using the same context, so if the record is there, why it does not show up when accessed via relationship ? Is there something I'm doing wrong ? Is there a way to force the RelationshipQuery to refresh ? We are using Cayenne build from sources about a month ago (svn 642725).

smime.p7s
Description: S/MIME cryptographic signature