Re: Distribution of keys to partitions

2017-04-06 Thread Yakov Zhdanov
Guys, I replied in ticket. Overall I liked the changes but I think this needs additional elaboration. Please see https://issues.apache.org/jira/browse/IGNITE-4828 --Yakov 2017-03-31 20:09 GMT+03:00 Denis Magda : > Michael, thanks. > > I did minor improvements and merged them to IGNITE-4828 branc

Re: Distribution of keys to partitions

2017-03-31 Thread Denis Magda
Michael, thanks. I did minor improvements and merged them to IGNITE-4828 branch and triggered TeamCity tests: http://ci.ignite.apache.org/viewQueued.html?itemId=526101&tab=queuedBuildOverviewTab *Michael*, please check the tests results lately. I won’t be available in the nearest 4 days. *Ilya

Re: Distribution of keys to partitions

2017-03-31 Thread Dmitriy Setrakyan
Vova, Not sure I understand what you mean. I believe that we just agreed that if the partition count is set to the power of 2, then we can improve the performance with a better hashing algorithm. D. On Fri, Mar 31, 2017 at 1:46 AM, Vladimir Ozerov wrote: > In order to overcome this problem, Ha

Re: Distribution of keys to partitions

2017-03-31 Thread Vladimir Ozerov
In order to overcome this problem, Hazelcast guys set non-power-of-two partition count by default - 257. On Fri, Mar 31, 2017 at 9:59 AM, michael.griggs wrote: > The change is now ready for review: > > https://github.com/apache/ignite/pull/1707 > > > > -- > View this message in context: http://a

Re: Distribution of keys to partitions

2017-03-31 Thread michael.griggs
The change is now ready for review: https://github.com/apache/ignite/pull/1707 -- View this message in context: http://apache-ignite-developers.2346864.n4.nabble.com/Distribution-of-keys-to-partitions-tp15455p16005.html Sent from the Apache Ignite Developers mailing list archive at Nabble.com.

Re: Distribution of keys to partitions

2017-03-22 Thread michael.griggs
I created a PR to implement this. I ran the TC tests, but there are a lot of errors. However, the errors seem unrelated to the change. I see that other PRs are suffering with similar test failures. Have some tests been broken by new 2.0 functionality and not fixed yet? http://ci.ignite.apach

Re: Distribution of keys to partitions

2017-03-15 Thread Denis Magda
Excellent discovery, thanks Michael! I would suggest doing the following. If we see that a number of partitions is a power of two then the new algorithm will be applied, otherwise the warning will be printed out and the *old* one approach will be used. Does this resolver all the concerns? Mich

Re: Distribution of keys to partitions

2017-03-15 Thread Dmitriy Setrakyan
On Wed, Mar 15, 2017 at 9:51 AM, Michael Griggs wrote: > Have we ever heard of somebody needing to set the partition count to a > non-power-of-two number? Perhaps we could restrict the method so that it > will only accept a power of two as the partition count? > As Valentin suggested, we should

RE: Distribution of keys to partitions

2017-03-15 Thread Michael Griggs
] Sent: 15 March 2017 16:22 To: dev@ignite.apache.org Subject: Re: Distribution of keys to partitions Andrey, Absolutely, your point is correct. I'm talking about default behavior which must be as effective as possible. In case we do this optimization, I would also show a warning if numb

Re: Distribution of keys to partitions

2017-03-15 Thread Valentin Kulichenko
Andrey, Absolutely, your point is correct. I'm talking about default behavior which must be as effective as possible. In case we do this optimization, I would also show a warning if number of partitions is not a power of two. -Val On Wed, Mar 15, 2017 at 5:09 PM, Andrey Gura wrote: > Anyway, w

Re: Distribution of keys to partitions

2017-03-15 Thread michael.griggs
Valentin Kulichenko wrote > In 99% of cases number of partition is a power of two, because it's the > default value. Almost no one changes it. If this change actually provides > better distribution, it absolutely makes sense to do it. > > Michael, can you create a Jira ticket and put you findings

Re: Distribution of keys to partitions

2017-03-15 Thread Andrey Gura
Anyway, we can't always use this optimization because it will not work for non power of two values. On Wed, Mar 15, 2017 at 6:48 PM, Valentin Kulichenko wrote: > In 99% of cases number of partition is a power of two, because it's the > default value. Almost no one changes it. If this change actua

Re: Distribution of keys to partitions

2017-03-15 Thread Valentin Kulichenko
In 99% of cases number of partition is a power of two, because it's the default value. Almost no one changes it. If this change actually provides better distribution, it absolutely makes sense to do it. Michael, can you create a Jira ticket and put you findings there? -Val On Wed, Mar 15, 2017 a

Re: Distribution of keys to partitions

2017-03-15 Thread Andrey Gura
Michael, it makes sense only for cases when partitions count is power of two. Affinity function doesn't have this limitation. Bu, of course, we can check, that partitions count is power of two and use optimized hash code calculation. On Wed, Mar 15, 2017 at 4:09 PM, Michael Griggs wrote: > Hi