Ilya, when .NET Near Cache is enabled, Java invokes callbacks to update or evict near cache data in .NET, hence the overhead.
I've created a PR, everybody is welcome to do a preliminary review (just ignore the minor things for now). Core functionality is ready, put/get is supported. I'm working on other cache operations. [1] https://github.com/apache/ignite/pull/7468 On Sat, Feb 22, 2020 at 7:07 PM Pavel Tupitsyn <ptupit...@apache.org> wrote: > Sergey, > > Sure, I'll add you as a reviewer. > And a thorough review will be the best contribution. The feature is rather > complex and in the works for quite some time. > > > On Fri, Feb 21, 2020 at 4:03 PM Ilya Kasnacheev <ilya.kasnach...@gmail.com> > wrote: > >> Hello! >> >> Can you please describe what is the overhead on server nodes from starting >> .Net near cache, as compared to starting "regular" near cache? >> >> Also, why do we start near cache on all server nodes - is it required by >> our architecture? >> >> Otherwise, if you add a new feature, which you have to explicitly invoke >> from application .Net code, I assume it's OK to consume some resources >> upon >> such invocation. We can't expect new features to consume zero resources. >> >> Regards, >> -- >> Ilya Kasnacheev >> >> >> вт, 18 февр. 2020 г. в 20:46, Pavel Tupitsyn <ptupit...@apache.org>: >> >> > Ilya, looks like there is a misunderstanding. >> > >> > We don't start near cache on a subset of server nodes. Let me explain >> > again. >> > Consider the following code, executed on any node (client or server - >> does >> > not matter): >> > >> > ignite.createCache(new CacheConfiguration("c").setNearConfiguration(new >> > NearCacheConfiguration())); >> > >> > As a result, cache "c" is started on all server nodes, and corresponding >> > near cache is started on all server nodes, >> > with the same config, eviction policy, and so on. >> > >> > Now, what do we do with .NET near cache in this situation? >> > 1. Start .NET near cache as well on server nodes, always. >> > 2. Introduce a config flag, and start .NET near cache only when the >> flag is >> > set >> > >> > The problem with option (1) is that it changes the behavior of existing >> > applications, >> > increasing overhead for cache operations and increasing memory usage. >> > >> > As a user, I don't want some new features to be force-enabled on me >> > silently in a new product version. >> > That is why an opt-in config flag is required. >> > >> > Does this make sense? >> > >> > On Tue, Feb 18, 2020 at 6:42 PM Ilya Kasnacheev < >> ilya.kasnach...@gmail.com >> > > >> > wrote: >> > >> > > Hello! >> > > >> > > I'm not sure that we can/want to create near cache on a subset of >> server >> > > nodes. >> > > >> > > If this is indeed possible, I would decouple that from .Net and >> introduce >> > > as a separate feature. >> > > >> > > Ideally we should be able to start or stop near cache on any node, >> server >> > > or client, and this should be the same cache for all platforms >> (including >> > > Java). WDYT? >> > > >> > > Regards, >> > > -- >> > > Ilya Kasnacheev >> > > >> > > >> > > вт, 18 февр. 2020 г. в 17:31, Pavel Tupitsyn <ptupit...@apache.org>: >> > > >> > > > Ilya, as noted above: >> > > > >> > > > > There are 2 kinds of Near Caches: >> > > > > - On client nodes: created per-node by calling >> ignite.CreateNearCache >> > > > > - On server nodes: created on all server nodes if >> > > > CacheConfiguration.NearCacheConfiguration is not null >> > > > > >> > > > > When user says ignite.CreateCache(new CacheConfiguration >> > > > {NearCacheConfiguration = ...}), >> > > > > the whole config is sent to all server nodes, and .NET-specific >> flag >> > > has >> > > > to be included somehow. >> > > > >> > > > >> > > > >> > > > On Tue, Feb 18, 2020 at 5:10 PM Ilya Kasnacheev < >> > > ilya.kasnach...@gmail.com >> > > > > >> > > > wrote: >> > > > >> > > > > Hello! >> > > > > >> > > > > Why would it waste additional memory? All nodes are also Java >> nodes >> > so >> > > > they >> > > > > will start near caches all right. >> > > > > >> > > > > Don't we have near cache start-up on per-node basis for clients, >> > > anyway? >> > > > > >> > > > > I'm not convinced why we need this flag. I have to admit my >> > > understanding >> > > > > of near caches is limited. >> > > > > >> > > > > Regards, >> > > > > -- >> > > > > Ilya Kasnacheev >> > > > > >> > > > > >> > > > > вт, 18 февр. 2020 г. в 16:56, Pavel Tupitsyn < >> ptupit...@apache.org>: >> > > > > >> > > > > > Ilya, Aleksandr, >> > > > > > >> > > > > > The flag is called platformNearCacheEnabled, my idea is to have >> > just >> > > > one >> > > > > > flag for all platforms. >> > > > > > >> > > > > > If some platform is present on the given node (.NET, C++) and it >> > > > supports >> > > > > > native near caching, >> > > > > > then the flag is honored, otherwise it is ignored. We should not >> > > throw >> > > > > > exceptions, >> > > > > > because mixed clusters are possible (.NET nodes along with Java >> > > nodes). >> > > > > > >> > > > > > > Why would enabling it affect mixed`clusters? >> > > > > > Initially the idea was to enable .NET near cache whenever >> > > > > > NearCacheConfiguration is present. >> > > > > > If we assume .NET-only clusters, it makes sense: existing code >> will >> > > > work >> > > > > > faster automatically. >> > > > > > >> > > > > > Mixed cluster is just one of the possible use cases when this >> may >> > be >> > > a >> > > > > bad >> > > > > > idea, >> > > > > > e.g. users have enabled near caching to speed up their >> Java-based >> > > > > > computations, >> > > > > > and .NET nodes are used for something else, wasting memory on >> near >> > > > > caching >> > > > > > unnecessarily. >> > > > > > >> > > > > > On ue, Feb 18, 2020 at 4:30 PM Aleksandr Shapkin < >> > lexw...@gmail.com >> > > > >> > > > > > wrote: >> > > > > > >> > > > > > > Pavel, >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > I think it’s ok to add a new flag. >> > > > > > > >> > > > > > > Though we may change a name to something like >> > > > > > #usePlatformCacheIfAvailable >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > But it may vary depending on the implementation, i.e. >> > > > > > > >> > > > > > > should we throw an error if there is no native cache >> > > > > > > >> > > > > > > available for a platform or just ignore the configuration. >> > > > > > > >> > > > > > > вт, 18 февр. 2020 г. в 15:05, Pavel Tupitsyn < >> > ptupit...@apache.org >> > > >: >> > > > > > > >> > > > > > > > Igor, >> > > > > > > > >> > > > > > > > The problem is - we need to pass this flag around the >> > cluster`for >> > > > > > Server >> > > > > > > > Near Caches, >> > > > > > > > so that .NET near caches are started accordingly. >> > > > > > > > >> > > > > > > > There are 2 kinds of Near Caches: >> > > > > > > > - On client nodes: created on every client node separately >> by >> > > > calling >> > > > > > > > ignite.CreateNearCache >> > > > > > > > - On server nodes: created on all server nodes if >> > > > > > > > CacheConfiguration.NearCacheConfiguration is set >> > > > > > > > >> > > > > > > > When user says ignite.CreateCache(new CacheConfiguration >> > > > > > > > {NearCacheConfiguration = ...}), >> > > > > > > > the whole config is sent 4o all server nodes, and >> .NET-specific >> > > > flag >> > > > > > has >> > > > > > > to >> > > > > > > > be included somehow. >> > > > > > > > >> > > > > > > > On Tue, Feb 18, 2020 at 2:59 PM Igor Sapego < >> > isap...@apache.org> >> > > > > > wrote: >> > > > > > > > >> > > > > > > > > Do you suggest to introduce it in general configuration? >> Why >> > > not >> > > > > > > > introduce >> > > > > > > > > it only on platform side? Is there any .NET-specific >> > > > configuration? >> > > > > > > > > >> > > > > > > > > Best Regards, >> > > > > > > > > Igo2 >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > On Tue, Feb 18, 2020 at 1:10 AM Pavel Tupitsyn < >> > > > > ptupit...@apache.org >> > > > > > > >> > > > > > > > > wrote: >> > > > > > > > > >> > > > > > > > > > Igniters, >> > > > > > > > > > >> > > > > > > > > > I'm working on .NET Near Cache feature [1] >> > > > > > > > > > (storing deserialized cache entries in CLR memory to >> > improve >> > > > > > > > > performance). >> > > > > > > > > > >> > > > > > > > > > Implementation is based on Java near cache, with some >> > > callbacks >> > > > > to >> > > > > > > .NET >> > > > > > > > > > side >> > > > > > > > > > for updating and invalidating cached entries. >> > > ~ > > > > > > >> > > > > > > > > > However, I'd like to make this feature optional: >> enabling >> > > Java >> > > > > near >> > > > > > > > cache >> > > > > > > > > > should not >> > > > > > > > > > always enable .NET near cache - some users may have >> mixed >> > > > > clusters, >> > > > > > > > etc. >> > > > > > > > > > >> > > > > > > > > > Therefore I'm adding >> > > > > > NearCacheConfiguration#platformNearCacheEnabled >> > > > > > > > > > boolean flag. >> > > > > > > > > > Are there any objections or better ideas to configure >> this >> > > > > > behavior? >> > > > > > > > > > >> > > > > > > > > > Thanks, >> > > > > > > > > > Pavel >> > > > > > > > > > >> > > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12691 >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > -- >> > > > > > > Alex. >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> >
