Hello Amit,
You can use ClusterNodeAttributeAffinityBackupFilter and introduce
some virtual zones. For example, if you have 5 nodes in zone 1 and 5
nodes in zone 2, you can assign 'zone1' attribute value to 3 nodes
from zone 1, assign 'zone2' attribute value to 3 nodes from zone 2,
and assign 'zone3' attribute value to 4 remaining nodes (2 from zone 1
and 2 from zone 2). In this case, there will be 3 copies of each
partition, each partition will be in each virtual zone, but nodes in
virtual 'zone3' will contain a little bit less partitions than nodes
from 'zone1' and 'zone2'.
Or you can create your own backup filter, allowing no more than two
nodes for the same attribute value, for example like this:
public class MyAffinityBackupFilter implements
IgniteBiPredicate<ClusterNode, List<ClusterNode>> {
private final String attrName;
public MyAffinityBackupFilter(String attrName) {
this.attrName = attrName;
}
@Override public boolean apply(ClusterNode candidate,
List<ClusterNode> previouslySelected) {
Set<Object> usedAttrs = new HashSet<>();
for (ClusterNode node : previouslySelected) {
if (Objects.equals(candidate.attribute(attrName),
node.attribute(attrName)) &&
!usedAttrs.add(candidate.attribute(attrName)))
return false;
}
return true;
}
}
In this case you can achieve a more even distribution.
чт, 19 сент. 2024 г. в 16:58, Amit Jolly <amit.jo...@gmail.com>:
>
> Hi Pavel,
>
> Well based upon documentation of
> ClusterNodeAttributeAffinityBackupFilter.java class. It says "This
> implementation will discard backups rather than place multiple on the same
> set of nodes. This avoids trying to cram more data onto remaining nodes when
> some have failed." and i have verified the same by running a small test with
> three node cluster (one assigned with node attribute as
> AVAILABILITY_ZONE=ZONE1 and other two assigned node attribute
> AVAILABILITY_ZONE=ZONE2) , Created a cache with 2 backups and using
> ClusterNodeAttributeAffinityBackupFilter in RendezvousAffinityFunction as
> below. After that added an entry into the cache and verified the nodes count
> for both primary and backup using cache affinity function. It returned 2
> instead of 3.
>
> ClusterNodeAttributeAffinityBackupFilter backupFilter = new
> ClusterNodeAttributeAffinityBackupFilter("AVAILABILITY_ZONE");
> RendezvousAffinityFunction rendezvousAffinityFunction = new
> RendezvousAffinityFunction();
> rendezvousAffinityFunction.setAffinityBackupFilter(backupFilter);
>
> CacheConfiguration<String, String> cacheConfiguration = new
> CacheConfiguration<>();
> cacheConfiguration.setBackups(2);
> cacheConfiguration.setAffinity(rendezvousAffinityFunction);
>
> IgniteCache<String, String> cache =
> ignite.getOrCreateCache(cacheConfiguration);
> cache.put("1","1");
> Collection<ClusterNode> nodes =
> ((Ignite)cache.unwrap(Ignite.class)).affinity(cache.getName()).mapKeyToPrimaryAndBackups("1");
> assertEquals(3, nodes.size()); //This fails even though i have three nodes (1
> with node attribute AVAILABILITY_ZONE="ZONE1" and other two with node
> attribute AVAILABILITY_ZONE="ZONE2")
>
> PS: I started three nodes with Custom cache configuration
> IgniteConfiguration.setUserAttributes
>
> Server Node1
> =====================================
> Map<String, String> userAttributes = new HashMap<>();
> userAttributes.put("AVAILABILITY_ZONE", "ZONE1");
> IgniteConfiguration cfg = new IgniteConfiguration();
> cfg.setUserAttributes(userAttributes);
> Ignition.start(cfg);
>
> Server Node2
> =====================================
> Map<String, String> userAttributes = new HashMap<>();
> userAttributes.put("AVAILABILITY_ZONE", "ZONE2");
> IgniteConfiguration cfg = new IgniteConfiguration();
> cfg.setUserAttributes(userAttributes);
> Ignition.start(cfg);
>
> Server Node3
> =====================================
> Map<String, String> userAttributes = new HashMap<>();
> userAttributes.put("AVAILABILITY_ZONE", "ZONE2");
> IgniteConfiguration cfg = new IgniteConfiguration();
> cfg.setUserAttributes(userAttributes);
> Ignition.start(cfg);
>
>
> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cache/affinity/rendezvous/ClusterNodeAttributeAffinityBackupFilter.html
>
> Thanks,
>
> Amit Jolly
>
> On Thu, Sep 19, 2024 at 12:51 AM Pavel Tupitsyn <ptupit...@apache.org> wrote:
>>
>> Hi Amit,
>>
>> > if the backup count is let's say 2, Ignite won't create a second backup
>> > as there are not enough zones
>> Not correct - Ignite will create backups anyway.
>> - A backup is a copy of a partition on another node
>> - With 2 backups every partition will have 3 copies (1 primary, 2 backup),
>> all on different nodes (since you have 10 nodes)
>> - Use ClusterNodeAttributeAffinityBackupFilter to ensure that at least one
>> of the copies is in a different AZ
>>
>> And that is enough for 3 copies.
>>
>> On Thu, Sep 19, 2024 at 12:10 AM Amit Jolly <amit.jo...@gmail.com> wrote:
>>>
>>> Hi Team
>>>
>>> We are planning to run 10 node Ignite clusters in AWS with 5 nodes each
>>> into two availability zones. Using Kubernetes topologyspreadconstraints we
>>> have made sure that no two Ignite pods are started on the same virtual
>>> machine/node/host.
>>>
>>> I understand with ClusterNodeAttributeAffinityBackupFilter i can force
>>> ignite to store the backup in a different zone if backup count is 1.
>>>
>>> But if the backup count is let's say 2, Ignite won't create a second backup
>>> as there are not enough zones.
>>>
>>> My question is if i have backup count 2, How can I use
>>> ClusterNodeAttributeAffinityBackupFilter or (custom AffinityBackupFilter)
>>> to have at least one backup in each zone and another backup anywhere else
>>> where available.
>>>
>>> I think in order to achieve what I am thinking somehow I need
>>> currentTopologySnapshot available in
>>> ClusterNodeAttributeAffinityBackupFilter or custom AffinityBackupFilter
>>>
>>> Thanks,
>>>
>>> Amit Jolly
