Re: KStreams to KTable join

2017-07-05 Thread Guozhang Wang
When KStream / KTable is created from a source topic, both of them has a record as a key-value pair, and the key is read from Kafka as the message key. What you showed in JSON seems only be the value of the message, and hence I'm asking what's the key of the message, which will be the key of the s

Re: KStreams to KTable join

2017-06-29 Thread Shekar Tippur
Guozhang, "1) if the coming record's key is null, then when it flows into the join processor inside the topology this record will be dropped as it cannot be joined with any records from the other stream." Can you please elaborate on the notion of key? By keys, do you mean kafka partition keys? Fo

Re: KStreams to KTable join

2017-06-29 Thread Shekar Tippur
Guozhang, "1) if the coming record's key is null, then when it flows into the join processor inside the topology this record will be dropped as it cannot be joined with any records from the other stream." Can you please elaborate on the notion of key? By keys, do you mean kafka partition keys? Fo

Re: KStreams to KTable join

2017-06-26 Thread Shekar Tippur
If I understand it right, I need to pass the key and ensure that I handle it correctly in the pojo constructor. Let me give it a shot. - Shekar On Mon, Jun 26, 2017 at 4:42 PM, Guozhang Wang wrote: > I think your issue is in two folds: > > 1) if the coming record's key is null, then when it flo

Re: KStreams to KTable join

2017-06-26 Thread Shekar Tippur
If I understand it right, I need to pass the key and ensure that I handle it correctly in the pojo constructor. Let me give it a shot. - Shekar On Mon, Jun 26, 2017 at 4:42 PM, Guozhang Wang wrote: > I think your issue is in two folds: > > 1) if the coming record's key is null, then when it flo

Re: KStreams to KTable join

2017-06-26 Thread Guozhang Wang
I think your issue is in two folds: 1) if the coming record's key is null, then when it flows into the join processor inside the topology this record will be dropped as it cannot be joined with any records from the other stream. 2) the NPE you are getting when giving it the non-null keyed record

Re: KStreams to KTable join

2017-06-25 Thread Shekar Tippur
Guozhang I am using 0.10.2.1 version - Shekar On Sun, Jun 25, 2017 at 10:36 AM, Guozhang Wang wrote: > Hi Shekar, > > Could you demonstrate your input data. More specifically, what are the key > types of your input streams, and are they not-null values? It seems the > root cause is similar to

Re: KStreams to KTable join

2017-06-25 Thread Shekar Tippur
Guozhang, Thanks for responding. The raw and cache keys are null. Both KStream and KTable entries are json's. Here is the input to cache (KTable) {"user_name": "Joe", "location": "US", "gender": "male"} {"user_name": "Julie", "location": "US", "gender": "female"}} {"user_name": "Kawasaki", "lo

Re: KStreams to KTable join

2017-06-25 Thread Guozhang Wang
Hi Shekar, Could you demonstrate your input data. More specifically, what are the key types of your input streams, and are they not-null values? It seems the root cause is similar to the other thread you asked on the mailing list. Also, could you provide your used Kafka Streams version? Guozhan