Re: keyBy using custom partitioner

2016-03-09 Thread madhu phatak
Hi, Thank you. On Mar 9, 2016 5:27 PM, "Stephan Ewen" wrote: > Hi! > > You can currently not override the hash function used by "keyBy()". The > reason is that this function is used in multiple places, for the stream > partitioning, and also for the partitioning of state. Both have to be > aligne

Re: keyBy using custom partitioner

2016-03-09 Thread Stephan Ewen
Hi! You can currently not override the hash function used by "keyBy()". The reason is that this function is used in multiple places, for the stream partitioning, and also for the partitioning of state. Both have to be aligned. What you can do is use "partitionCustom(...)" to use an arbitrary part

keyBy using custom partitioner

2016-03-09 Thread madhu phatak
Hi, How to use a custom partitioner in keyBy operation? As of now it's using hash partitioner to load balance across parallel tasks. I tried custom partitioning the schema before calling keyBy operation. It doesn't seem to preserve that partition. -- Regards, Madhukara Phatak http://datamantra.io