Re: regarding the use of colocation groups

2018-03-28 Thread Konstantinos Barmpis
Hello Chesnay, Thank you for your prompt reply and helpful feedback on the matter, For the fist case-study a partitioner seems like a good idea, we have managed to get it working on the default local minicluster but unfortunately are running into issues deploying it on AWS. When we pinpoint the i

Re: regarding the use of colocation groups

2018-03-27 Thread Chesnay Schepler
Hello, your first use-case should be achievable by using a custom partitioner , probably with a KeySelector that returns the word. As for the second use-case, typically this would be achieved b

regarding the use of colocation groups

2018-03-27 Thread Konstantinos Barmpis
Hello, I was wondering how to properly use colocation groups (if applicable) to achieve the required functionality in the following two simple contrived use-cases (focusing on the essence of the problem), both of which aim to be executed on a multi-node cluster (2 or more slaves and a master), wit