I've updated IEP page: [1] What do you think now? To me it looks cleaner.
[1] - https://cwiki.apache.org/confluence/display/IGNITE/IEP-23%3A+Best+Effort+Affinity+for+thin+clients Best Regards, Igor On Mon, Feb 4, 2019 at 4:44 PM Igor Sapego <isap...@apache.org> wrote: > Ok, I understand now. I'll try updating IEP according to this proposal and > notify you guys. > > Best Regards, > Igor > > > On Mon, Feb 4, 2019 at 4:27 PM Vladimir Ozerov <voze...@gridgain.com> > wrote: > >> Igor, >> >> My idea is simply to add the list of caches with the same distribution to >> the end of partition response. Client can use this information to populate >> partition info for more caches in a single request. >> >> On Mon, Feb 4, 2019 at 3:06 PM Igor Sapego <isap...@apache.org> wrote: >> >> > Vladimir, >> > >> > So correct me if I'm wrong, what you propose is to avoid mentioning >> > of cache groups, and use instead of "cache group" term something like >> > "distribution"? Or do you propose some changes in protocol? If so, can >> > you briefly explain, what kind of changes they are? >> > >> > Best Regards, >> > Igor >> > >> > >> > On Mon, Feb 4, 2019 at 1:13 PM Vladimir Ozerov <voze...@gridgain.com> >> > wrote: >> > >> > > Igor, >> > > >> > > Yes, cache groups are public API. However, we try to avoid new APIs >> > > depending on them. >> > > The main point from my side is that “similar cache group” can be >> easily >> > > generalized to “similar distribution”. This way we avoid cache groups >> on >> > > protocol level at virtually no cost. >> > > >> > > Vladimir. >> > > >> > > пн, 4 февр. 2019 г. в 12:48, Igor Sapego <isap...@apache.org>: >> > > >> > > > Guys, >> > > > >> > > > Can you explain why do we want to avoid Cache groups in protocol? >> > > > >> > > > If it's about simplicity of the protocol, then removing cache groups >> > will >> > > > not help much with it - we will still need to include >> "knownCacheIds" >> > > > field in request and "cachesWithTheSamePartitioning" field in >> response. >> > > > And also, since when do Ignite prefers simplicity over performance? >> > > > >> > > > If it's about not wanting to show internals of Ignite then it sounds >> > like >> > > > a very weak argument to me, since Cache Groups is a public thing >> [1]. >> > > > >> > > > [1] - https://apacheignite.readme.io/docs/cache-groups >> > > > >> > > > Best Regards, >> > > > Igor >> > > > >> > > > >> > > > On Mon, Feb 4, 2019 at 11:47 AM Vladimir Ozerov < >> voze...@gridgain.com> >> > > > wrote: >> > > > >> > > > > Pavel, Igor, >> > > > > >> > > > > This is not very accurate to say that this will not save memory. >> In >> > > > > practice we observed a number of OOME issues on the server-side >> due >> > to >> > > > many >> > > > > caches and it was one of motivations for cache groups (another one >> > disk >> > > > > access optimizations). On the other hand, I agree that we'd >> better to >> > > > avoid >> > > > > cache groups in the protocol because this is internal >> implementation >> > > > detail >> > > > > which is likely (I hope so) to be changed in future. >> > > > > >> > > > > So I have another proposal - let's track caches with the same >> > affinity >> > > > > distribution instead. That is, normally most of PARTITIONED caches >> > will >> > > > > have very few variants of configuration: it will be Rendezvous >> > affinity >> > > > > function, most likely with default partition number and with 1-2 >> > > backups >> > > > at >> > > > > most. So when affinity distribution for specific cache is >> requested, >> > we >> > > > can >> > > > > append to the response *list of caches with the same >> distribution*. >> > > I.e.: >> > > > > >> > > > > class AffinityResponse { >> > > > > Object distribution; // Actual distribution >> > > > > List<Integer> cacheIds; // Caches with similar distribution >> > > > > } >> > > > > >> > > > > Makes sense? >> > > > > >> > > > > On Sun, Feb 3, 2019 at 8:31 PM Pavel Tupitsyn < >> ptupit...@apache.org> >> > > > > wrote: >> > > > > >> > > > > > Igor, I have a feeling that we should omit Cache Group stuff >> from >> > the >> > > > > > protocol. >> > > > > > It is a rare use case and even then dealing with them on client >> > > barely >> > > > > > saves some memory. >> > > > > > >> > > > > > We can keep it simple and have partition map per cacheId. >> Thoughts? >> > > > > > >> > > > > > On Fri, Feb 1, 2019 at 6:49 PM Igor Sapego <isap...@apache.org> >> > > wrote: >> > > > > > >> > > > > > > Guys, I've updated the proposal once again [1], so please, >> > > > > > > take a look and let me know what you think. >> > > > > > > >> > > > > > > [1] - >> > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/IGNITE/IEP-23%3A+Best+Effort+Affinity+for+thin+clients >> > > > > > > >> > > > > > > Best Regards, >> > > > > > > Igor >> > > > > > > >> > > > > > > >> > > > > > > On Thu, Jan 17, 2019 at 1:05 PM Igor Sapego < >> isap...@apache.org> >> > > > > wrote: >> > > > > > > >> > > > > > > > Yeah, I'll add it. >> > > > > > > > >> > > > > > > > Best Regards, >> > > > > > > > Igor >> > > > > > > > >> > > > > > > > >> > > > > > > > On Wed, Jan 16, 2019 at 11:08 PM Pavel Tupitsyn < >> > > > > ptupit...@apache.org> >> > > > > > > > wrote: >> > > > > > > > >> > > > > > > >> > to every server >> > > > > > > >> I did not think of this issue. Now I agree with your >> approach. >> > > > > > > >> Can you please add an explanation of this to the IEP? >> > > > > > > >> >> > > > > > > >> Thanks! >> > > > > > > >> >> > > > > > > >> On Wed, Jan 16, 2019 at 2:53 PM Igor Sapego < >> > isap...@apache.org >> > > > >> > > > > > wrote: >> > > > > > > >> >> > > > > > > >> > Pavel, >> > > > > > > >> > >> > > > > > > >> > Yeah, it makes sense, but to me it seems that this >> approach >> > > can >> > > > > lead >> > > > > > > >> > to more complicated client logic, as it will require to >> make >> > > > > > > additional >> > > > > > > >> > call >> > > > > > > >> > to every server, that reports affinity topology change. >> > > > > > > >> > >> > > > > > > >> > Guys, WDYT? >> > > > > > > >> > >> > > > > > > >> > Best Regards, >> > > > > > > >> > Igor >> > > > > > > >> > >> > > > > > > >> > >> > > > > > > >> > On Tue, Jan 15, 2019 at 10:59 PM Pavel Tupitsyn < >> > > > > > ptupit...@apache.org >> > > > > > > > >> > > > > > > >> > wrote: >> > > > > > > >> > >> > > > > > > >> > > Igor, >> > > > > > > >> > > >> > > > > > > >> > > > It is proposed to add flag to every response, that >> > shows >> > > > > > whether >> > > > > > > >> the >> > > > > > > >> > > Affinity Topology Version of the cluster has changed >> since >> > > the >> > > > > > last >> > > > > > > >> > request >> > > > > > > >> > > from the client. >> > > > > > > >> > > I propose to keep this flag. So no need for periodic >> > checks. >> > > > > Makes >> > > > > > > >> sense? >> > > > > > > >> > > >> > > > > > > >> > > On Tue, Jan 15, 2019 at 4:45 PM Igor Sapego < >> > > > isap...@apache.org >> > > > > > >> > > > > > > >> wrote: >> > > > > > > >> > > >> > > > > > > >> > > > Pavel, >> > > > > > > >> > > > >> > > > > > > >> > > > This will require from client to send this new >> request >> > > > > > > periodically, >> > > > > > > >> > I'm >> > > > > > > >> > > > not >> > > > > > > >> > > > sure this will make clients simpler. Anyway, let's >> > discuss >> > > > it. >> > > > > > > >> > > > >> > > > > > > >> > > > Vladimir, >> > > > > > > >> > > > >> > > > > > > >> > > > With current proposal, we will have affinity info in >> > > message >> > > > > > > header. >> > > > > > > >> > > > >> > > > > > > >> > > > Best Regards, >> > > > > > > >> > > > Igor >> > > > > > > >> > > > >> > > > > > > >> > > > >> > > > > > > >> > > > On Tue, Jan 15, 2019 at 11:01 AM Vladimir Ozerov < >> > > > > > > >> voze...@gridgain.com >> > > > > > > >> > > >> > > > > > > >> > > > wrote: >> > > > > > > >> > > > >> > > > > > > >> > > > > Igor, >> > > > > > > >> > > > > >> > > > > > > >> > > > > I think that "Cache Partitions Request" should >> contain >> > > > > > affinity >> > > > > > > >> > > topology >> > > > > > > >> > > > > version. Otherwise we do not know what >> distribution is >> > > > > > returned >> > > > > > > - >> > > > > > > >> the >> > > > > > > >> > > one >> > > > > > > >> > > > > we expected, or some newer one. The latter may >> happen >> > in >> > > > > case >> > > > > > > >> > topology >> > > > > > > >> > > > > changed or late affinity assignment happened >> between >> > > > server >> > > > > > > >> response >> > > > > > > >> > > and >> > > > > > > >> > > > > subsequent client partitions request. >> > > > > > > >> > > > > >> > > > > > > >> > > > > Vladimir. >> > > > > > > >> > > > > >> > > > > > > >> > > > > On Mon, Jan 14, 2019 at 6:08 PM Igor Sapego < >> > > > > > isap...@apache.org >> > > > > > > > >> > > > > > > >> > > wrote: >> > > > > > > >> > > > > >> > > > > > > >> > > > > > Hello guys, >> > > > > > > >> > > > > > >> > > > > > > >> > > > > > I've updated IEP page [1] describing proposed >> > solution >> > > > in >> > > > > > more >> > > > > > > >> > > details >> > > > > > > >> > > > > and >> > > > > > > >> > > > > > proposing some changes for a protocol. >> > > > > > > >> > > > > > >> > > > > > > >> > > > > > Please, take a look and let me know what you >> think. >> > > > > > > >> > > > > > >> > > > > > > >> > > > > > [1] - >> > > > > > > >> > > > > > >> > > > > > > >> > > > > > >> > > > > > > >> > > > > >> > > > > > > >> > > > >> > > > > > > >> > > >> > > > > > > >> > >> > > > > > > >> >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/IGNITE/IEP-23%3A+Best+Effort+Affinity+for+thin+clients >> > > > > > > >> > > > > > >> > > > > > > >> > > > > > Best Regards, >> > > > > > > >> > > > > > Igor >> > > > > > > >> > > > > > >> > > > > > > >> > > > > > >> > > > > > > >> > > > > > On Tue, Jun 19, 2018 at 11:54 AM Vladimir Ozerov >> < >> > > > > > > >> > > voze...@gridgain.com >> > > > > > > >> > > > > >> > > > > > > >> > > > > > wrote: >> > > > > > > >> > > > > > >> > > > > > > >> > > > > > > Denis, >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > Yes, in principle we can extend it. We are >> going >> > to >> > > > > > > implement >> > > > > > > >> it >> > > > > > > >> > in >> > > > > > > >> > > > > > > subsequent phases of this IEP. >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > On Tue, Jun 19, 2018 at 4:30 AM, Dmitriy >> > Setrakyan < >> > > > > > > >> > > > > > dsetrak...@apache.org> >> > > > > > > >> > > > > > > wrote: >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > > On Mon, Jun 18, 2018 at 11:07 AM, Denis >> Magda < >> > > > > > > >> > dma...@apache.org >> > > > > > > >> > > > >> > > > > > > >> > > > > > wrote: >> > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > > Folks, >> > > > > > > >> > > > > > > > > >> > > > > > > >> > > > > > > > > Feel that this functionality can be >> extended >> > to >> > > > the >> > > > > > > >> automatic >> > > > > > > >> > > > > > > reconnect, >> > > > > > > >> > > > > > > > > can't it? Presently we require to provide a >> > > static >> > > > > > list >> > > > > > > of >> > > > > > > >> > IPs >> > > > > > > >> > > to >> > > > > > > >> > > > > be >> > > > > > > >> > > > > > > used >> > > > > > > >> > > > > > > > > at a reconnect time. By having a partition >> map >> > > of >> > > > > all >> > > > > > > the >> > > > > > > >> > > nodes, >> > > > > > > >> > > > > the >> > > > > > > >> > > > > > > thin >> > > > > > > >> > > > > > > > > client should be able to automate this >> piece. >> > > > > > > >> > > > > > > > > >> > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > Not sure if static IP list can be avoided. >> What >> > > Igor >> > > > > is >> > > > > > > >> > > suggesting >> > > > > > > >> > > > is >> > > > > > > >> > > > > > > that >> > > > > > > >> > > > > > > > we try to pick the best node out of the >> static >> > IP >> > > > > list. >> > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > D. >> > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > >> > > > > > > >> > > > > >> > > > > > > >> > > > >> > > > > > > >> > > >> > > > > > > >> > >> > > > > > > >> >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> >
