Re: Self-join with filter

2015-08-18 Thread Ashwin Jayaprakash
Stephan, this is exactly what I was looking for :) Thanks, will try it out. I know the combineGroup() needed a reduceGroup() too, but I was just trying out the APIs. I did not realize that the other streaming APIs were already available. I will have a look. Thanks again. On Mon, Aug 17, 2015

Re: Self-join with filter

2015-08-18 Thread Stephan Ewen
Hi! I am not 100% sure that I understand your question completely, but I'll give it my best shot. If you want to push IDs into the connector, I assume you mean that you use some form of connector that can filter by ID directly in the low level data access paths, in order to read as little data as

Self-join with filter

2015-08-17 Thread Ashwin Jayaprakash
Hi, I'm trying to evaluate Flink to see if it can do efficient semi-joins or self-joins with filter. Problem description: I have 1 stream that can contain "near duplicates" records. The records share a "family name" and so, many records can have the same family name. But each record has a unique i