Hi John, thanks for your quick and detailed offerings below. First, the funny, in the last few weeks I have actually been playing with your github repo's and particularly your contacts repo's predecessor, the 'actionable' spring gemfire repo. We've been evaluating IMDG's: Ignite (v1.5), Hazelcast (v3.6.3) and Gemfire/Geode for a few months now. We are looking for a very simple, straight forward IMDG solution to just simply read/write data to the a remote distributed durable IN-MEM DATA GRID, my team's service instances are the only client. Most of our regions will be PARTITIONED some lookup tables will REPLICATED, all will be PERSISTENT and store PdxInstances. Of the three, PIVOTAL Gemfire/Geode was our initial selection after our first evaluations, partly because we are a Spring shop already. We've used Hazelcast before and we are familiar with some it's limitations (open-source ver) and proprietary quirks and we initially found Ignite a little stubborn to config at first and a bit heavy-handed as a full-blown 'fabric'.
We've been working with Geode and Spring-Data-Gemfire/Geode, client and server configurations, for the last 3 months. But here is the sticking point at present with Geode: the maturity, stability and it's presently confusing identity and roadmap timetable, this would include your 'babies' as well, spring-data-gemfire/geode. The documentation is confusing, disjointed and misleading as you even asserted at the beginning of your reply. And it was noted as well in last month's user email thread you contributed on as well (Re: Persistence and OQL over cold data). I can also sight two simple examples from your own references in the reply you've provided me, listed at the bottom: 1) [4] http://gemfire.docs.pivotal.io/docs-gemfire/latest/developing/region_options/data_hosts_and_accessors.html states: "To configure a region so the member is a data accessor, you use configurations that specify no local data storage for the region." But nowhere on this page does it specifically identify what those "configurations" you should use are or a link to them. 2) [2] https://github.com/apache/incubator-geode/blob/rel/v1.0.0-incubating.M3/geode-core/src/main/java/com/gemstone/gemfire/internal/cache/GemFireCacheImpl.java#L4953-L4960 The code referenced in the link (see below) uses AttributesFactory which is now DEPRECATED in favor of createClientRegionFactory(ClientRegionShortcut.PROXY); And this is from an INTERNAL implementation class in the latest geode release version .M3!!! case PROXY: { AttributesFactory af = new AttributesFactory(); af.setDataPolicy(DataPolicy.EMPTY); UserSpecifiedRegionAttributes ra = (UserSpecifiedRegionAttributes) af.create(); ra.requiresPoolName = true; c.setRegionAttributes(pra.toString(), ra); break; } John you did address the problem I originally was asking clarification on: the confusing nature of what Geode/Gemfire considers a PROXY and how it behaves, different from using any local cache copy/clone. We do not want cloned data in our client application jvm memory, we want a performant remote distributed IMDG cluster to always be the canonical source, no eventqueues and eventlisteners required. I can now see that even a client PROXY region is still a LOCAL EMPTY region instance fronting the server backed region data and not acting as a pure PROXY passthrough API. The simple use case confusion below, where client proxy region calls to region.size() and region.values() can not return the server region responses just seems odd, and different than other API's we've used. Strange that when we make these calls on a client PROXY API that it doesn't have a pure PROXY/passthrough implementation to return the server region size or values, knowing it's configured to have no local data itself. So John, since you responded, let me specifically extend this inquiry topic into the realm of your spring-data-gemfire/geode. First, sorry I haven't seen your 'geode clubhouse' presentation yet, got it bookmarked for reading, and FYI I've gone a round or two with it already, seemed friendly, easier to use and config. But I'm re-thinking now that I may not have had it configured for PROXY when I tested it a couple months ago and I think now my tests/timings may have been misleading as the gets were probably being served from local region in-memory and not all the way back to the server nodes. Makes me wonder too whether the behavior applies even to PARTITIONED server/data regions. John, SPRING-DATA-G Q's: What do you feel is the maturity of spring-data-g at this time? Roadmap milestones ahead? Is the use of gemfireTemplate an improvement on any of this behavior or will it behave the same? What do you feel are the advantages of wrapping the client with a spring-data wrapper? One issue we experience often using spring wrapper projects, esp. on highly 'pivotal' projects and techs (geode/kafka) is that it grows our tech stack deeper to add spring wrapper projects and makes it harder to update and then there are the version/feature synchronization issues between core tech and wrapper project which it adds, with bugs at both levels. But these are generic 'Spring' project problems. Can using SDG gemfireTemplate offer significant enough advantages over core geode api? John, BTW, one SDG issue we had was wanting to use your latest SDG springbootapplication annotatons: @ClientCacheApplication @CacheServerApplication @EnableCacheServers @EnableLocator We would ideally want to be able to dynamically config these annotation's attributes, but, as is, it is not possible. Ideas? John, thanks again for your time and attention. We need to get comfortable and confident using Geode/Spring-Data-Geode and validate our expectations soon or move on to an alternative. -Roger From: John Blum <jb...@pivotal.io<mailto:jb...@pivotal.io>> Reply-To: "user@geode.incubator.apache.org<mailto:user@geode.incubator.apache.org>" <user@geode.incubator.apache.org<mailto:user@geode.incubator.apache.org>> Date: Friday, September 9, 2016 at 7:18 PM To: "user@geode.incubator.apache.org<mailto:user@geode.incubator.apache.org>" <user@geode.incubator.apache.org<mailto:user@geode.incubator.apache.org>> Subject: Re: Why am I getting a LocalRegion when I should be getting ProxyRegion? Hi Roger- See comments in-line... On Fri, Sep 9, 2016 at 5:25 PM, Roger Vandusen <roger.vandu...@ticketmaster.com<mailto:roger.vandu...@ticketmaster.com>> wrote: Using latest version .M3. clientCache = new ClientCacheFactory() .addPoolLocator( getLocatorUrl(), getLocatorPort() ) .set( "log-level", getLogLevel() ) .create(); ClientRegionFactory<String, PdxInstance> clientRegionFactory = getClientCache() .<String, PdxInstance>createClientRegionFactory(ClientRegionShortcut.PROXY); Region<String, PdxInstance> region = clientRegionFactory.create(regionName.getName()); The problem: Region is a instance of internal LocalRegion not ProxyRegion? Why do you think this is a problem? Technically, it has more to do with a Region's DataPolicy<http://geode.incubator.apache.org/releases/latest/javadoc/com/gemstone/gemfire/cache/DataPolicy.html> [1] than the actual class type of the (client) Region's implementation (which can be misleading as you have just discovered). See here<https://github.com/apache/incubator-geode/blob/rel/v1.0.0-incubating.M3/geode-core/src/main/java/com/gemstone/gemfire/internal/cache/GemFireCacheImpl.java#L4953-L4960> [2], for instance. In fact, you would not even be able to define a client "PROXY<http://geode.incubator.apache.org/releases/latest/javadoc/com/gemstone/gemfire/cache/client/ClientRegionShortcut.html#PROXY>" [3] Region and perform Region operations (e.g. gets/puts) if the corresponding Region (by name) did not exist on the GemFire Server to which the cache client (application) is connected. I.e. GemFire would throw an error... com.gemstone.gemfire.cache.client.ServerOperationException: remote server on 172.28.128.1(GeodeClientApplication:16387:loner):63975:b155a811:GeodeClientApplication: While performing a remote get at com.gemstone.gemfire.cache.client.internal.AbstractOp.processObjResponse(AbstractOp.java:293) at com.gemstone.gemfire.cache.client.internal.GetOp$GetOpImpl.processResponse(GetOp.java:152) at com.gemstone.gemfire.cache.client.internal.AbstractOp.attemptReadResponse(AbstractOp.java:175) at com.gemstone.gemfire.cache.client.internal.AbstractOp.attempt(AbstractOp.java:378) at com.gemstone.gemfire.cache.client.internal.ConnectionImpl.execute(ConnectionImpl.java:274) at com.gemstone.gemfire.cache.client.internal.pooling.PooledConnection.execute(PooledConnection.java:328) at com.gemstone.gemfire.cache.client.internal.OpExecutorImpl.executeWithPossibleReAuthentication(OpExecutorImpl.java:937) at com.gemstone.gemfire.cache.client.internal.OpExecutorImpl.execute(OpExecutorImpl.java:155) at com.gemstone.gemfire.cache.client.internal.OpExecutorImpl.execute(OpExecutorImpl.java:110) at com.gemstone.gemfire.cache.client.internal.PoolImpl.execute(PoolImpl.java:700) at com.gemstone.gemfire.cache.client.internal.GetOp.execute(GetOp.java:97) at com.gemstone.gemfire.cache.client.internal.ServerRegionProxy.get(ServerRegionProxy.java:112) at com.gemstone.gemfire.internal.cache.LocalRegion.findObjectInSystem(LocalRegion.java:2919) at com.gemstone.gemfire.internal.cache.LocalRegion.nonTxnFindObject(LocalRegion.java:1539) at com.gemstone.gemfire.internal.cache.LocalRegionDataView.findObject(LocalRegionDataView.java:155) at com.gemstone.gemfire.internal.cache.LocalRegion.get(LocalRegion.java:1411) at com.gemstone.gemfire.internal.cache.LocalRegion.get(LocalRegion.java:1347) at com.gemstone.gemfire.internal.cache.LocalRegion.get(LocalRegion.java:1329) at com.gemstone.gemfire.internal.cache.AbstractRegion.get(AbstractRegion.java:282) at example.app.geode.client.GeodeClientApplication.sendEchoRequest(GeodeClientApplication.java:138) at example.app.geode.client.GeodeClientApplication.run(GeodeClientApplication.java:87) at example.app.geode.client.GeodeClientApplication.run(GeodeClientApplication.java:76) at example.app.geode.client.GeodeClientApplication.run(GeodeClientApplication.java:57) at example.app.geode.client.GeodeClientApplication.main(GeodeClientApplication.java:48) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140) Caused by: com.gemstone.gemfire.cache.RegionDestroyedException: Server connection from [identity(172.28.128.1(GeodeClientApplication:16387:loner):63975:b155a811:GeodeClientApplication,connection=1; port=63975]: Region named /Echo/Echo was not found during get request at com.gemstone.gemfire.internal.cache.tier.sockets.BaseCommand.writeRegionDestroyedEx(BaseCommand.java:642) at com.gemstone.gemfire.internal.cache.tier.sockets.command.Get70.cmdExecute(Get70.java:153) at com.gemstone.gemfire.internal.cache.tier.sockets.BaseCommand.execute(BaseCommand.java:146) at com.gemstone.gemfire.internal.cache.tier.sockets.ServerConnection.doNormalMsg(ServerConnection.java:783) at com.gemstone.gemfire.internal.cache.tier.sockets.ServerConnection.doOneMessage(ServerConnection.java:913) at com.gemstone.gemfire.internal.cache.tier.sockets.ServerConnection.run(ServerConnection.java:1180) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at com.gemstone.gemfire.internal.cache.tier.sockets.AcceptorImpl$1$1.run(AcceptorImpl.java:555) at java.lang.Thread.run(Thread.java:745) This error occurs when I try to do echoRegion.get("key") and the Region does not exist on the server. No error is reported if I do NOT use the client PROXY Region, even if it does not exist on the sever when created on the client. Only when I attempt to "use" the Region in a particular way (e.g. data access) does something happen... the client tries to communicate with the server based on the pool settings. Also, not all Region operations (e.g. isEmpty()/size()) cause a server operation to occur. In the case of isEmpty()/size(), these are "locally" based operations and only have values when the local Region stores data, whether on client as a CACHING_PROXY or on a peer as a Local-only/Non-Distributed Region, a REPLICATE or PARTITION Region and so on. For instance, it is even possible to define a server-side Region that is only a "Data Accessor<http://gemfire.docs.pivotal.io/docs-gemfire/latest/developing/region_options/data_hosts_and_accessors.html>" [4] but no local state. In the "Data Accessor" Region case, I believe isEmpty() would return true and size() would return 0 even though the Data Accessor would refer to a peer Region where other data nodes in the cluster would actually maintain state. A PARTITION Region is another good example of a peer (server-side) Region where the size() would not necessarily reflect the number of entries in the "logical" Region since the PARTITION Region's data is distributed (i.e. "partitioned"/"sharded") across the cluster. When the server region has data, client side region.size() returns 0 and region.values() returns empty. This is actually an indication that indeed your (client) Region is a PROXY. As the Javadoc<http://geode.incubator.apache.org/releases/latest/javadoc/com/gemstone/gemfire/cache/client/ClientRegionShortcut.html#PROXY> [3] points out, "A PROXY region has no local state and forwards all operations to a server." Also what is the value of "regionName.getName()" in you setup? Where is "regionName" coming from? What is wrong here that I can't access my server region from the defined client proxy region? How do you mean? What Region "operations" on the client have you tried? By way example, I have a GeodeServerApplication<https://github.com/jxblum/contacts-application/blob/master/configuration-example/src/main/java/example/app/geode/server/GeodeServerApplication.java> [5] and a GeodeClientApplication<https://github.com/jxblum/contacts-application/blob/master/configuration-example/src/main/java/example/app/geode/client/GeodeClientApplication.java> [6] you can run. Play around with un/commenting the creation<https://github.com/jxblum/contacts-application/blob/master/configuration-example/src/main/java/example/app/geode/server/GeodeServerApplication.java#L133> [7] of the "/Echo" PARTITION Region on the server and executing or un/commenting the following lines<https://github.com/jxblum/contacts-application/blob/master/configuration-example/src/main/java/example/app/geode/client/GeodeClientApplication.java#L90-L92> [8] (client PROXY Region data access ops (i.e. get)) in the cache client application. You will witness when the Exception I noted above occurs and does not. For instance, when line 133 in the server application is commented out (thus preventing the creation of the /Echo PARTITION Region) and I have lines 91-93 commented on the client (even though the client still creates the corresponding /Echo PROXY Region), so long as I do not perform the Region ops in lines 91-93, no Exception occurs. If I uncomment lines 91-93 in the client before allowing the creation of the /Echo Region on line 133 in the server, I get the error. But when the Region exists on the server, no problem. In all cases, the client /Echo PROXY Region isEmpty() will be true and size() will be 0, even after the corresponding Region (data access) ops have been performed, as my assertions<https://github.com/jxblum/contacts-application/blob/master/configuration-example/src/main/java/example/app/geode/client/GeodeClientApplication.java#L94-L95> [9] indicate. However, that does not mean the corresponding server Region does not have any state... gfsh>connect Connecting to Locator at [host=localhost, port=10334] .. Connecting to Manager at [host=172.28.128.1, port=1099] .. Successfully connected to: [host=172.28.128.1, port=1099] gfsh>list members Name | Id ---------------------- | ------------------------------------------------------- GeodeServerApplication | 172.28.128.1(GeodeServerApplication:16732)<ec><v0>:1024 gfsh>list regions List of regions --------------- Echo gfsh>describe region --name=/Echo ......................................................................... Name : Echo Data Policy : partition Hosting Members : GeodeServerApplication Non-Default Attributes Shared By Hosting Members Type | Name | Value ------ | ------------ | ---------------------------------------------- Region | data-policy | PARTITION | size | 3 | cache-loader | example.app.geode.cache.loader.EchoCacheLoader -Roger Hope this helps! Cheers, -John [1] http://geode.incubator.apache.org/releases/latest/javadoc/com/gemstone/gemfire/cache/DataPolicy.html [2] https://github.com/apache/incubator-geode/blob/rel/v1.0.0-incubating.M3/geode-core/src/main/java/com/gemstone/gemfire/internal/cache/GemFireCacheImpl.java#L4953-L4960 [3] http://geode.incubator.apache.org/releases/latest/javadoc/com/gemstone/gemfire/cache/client/ClientRegionShortcut.html#PROXY [4] http://gemfire.docs.pivotal.io/docs-gemfire/latest/developing/region_options/data_hosts_and_accessors.html [5] https://github.com/jxblum/contacts-application/blob/master/configuration-example/src/main/java/example/app/geode/server/GeodeServerApplication.java [6] https://github.com/jxblum/contacts-application/blob/master/configuration-example/src/main/java/example/app/geode/client/GeodeClientApplication.java [7] https://github.com/jxblum/contacts-application/blob/master/configuration-example/src/main/java/example/app/geode/server/GeodeServerApplication.java#L133 [8] https://github.com/jxblum/contacts-application/blob/master/configuration-example/src/main/java/example/app/geode/client/GeodeClientApplication.java#L91-L93 [9] https://github.com/jxblum/contacts-application/blob/master/configuration-example/src/main/java/example/app/geode/client/GeodeClientApplication.java#L94-L95
