Michael,

Thanks! Looking forward to the update.
An interface like KTable is very conducive for joins. Hopefully, it will
get more flexible.

Srikanth

On Thu, Jul 14, 2016 at 4:35 AM, Michael Noll <mich...@confluent.io> wrote:

> Srikant,
>
> > Its not a case for join() as the keys don't match. Its more a lookup
> table.
>
> Yes, the semantics of streaming joins in Kafka Streams are bit different
> from joins in traditional RDBMS.
> See
> http://docs.confluent.io/3.0.0/streams/developer-guide.html#joining-streams
> .
>
> Also, a heads up:  It turned out that user questions around joins in Kafka
> Streams are pretty common.  We are currently working on improving the
> documentation for joins to make this more clear.
>
> Best wishes,
> Michael
>
>
>
> On Thu, Jul 14, 2016 at 10:12 AM, Matthias J. Sax <matth...@confluent.io>
> wrote:
>
> > I would recommend re-partitioning as described in Option 2.
> >
> > -Matthias
> >
> > On 07/13/2016 10:53 PM, Srikanth wrote:
> > > Hello,
> > >
> > > I'm trying the following join using KTable. There are two change log
> > tables.
> > > Table1
> > >   111 -> aaa
> > >   222 -> bbb
> > >   333 -> aaa
> > >
> > > Table2
> > >   aaa -> 999
> > >   bbb -> 888
> > >   ccc -> 777
> > >
> > > My result table should be
> > >   111 -> 999
> > >   222 -> 888
> > >   333 -> 999
> > >
> > > Its not a case for join() as the keys don't match. Its more a lookup
> > table.
> > >
> > > Option1 is to use a Table1.toStream().process(ProcessSupplier(),
> > > "storeName")
> > > punctuate() will use regular kafka consumer that reads updates from
> > Table2
> > > and updates a private map.
> > > Process() will do a key-value lookup.
> > > This has an advantage when Table1 is much larger than Table2.
> > > Each instance of the processor will have to hold entire Table2.
> > >
> > > Option2 is to re-partition Table1 using through(StreamPartitioner) and
> > > partition using value.
> > > This will ensure co-location. Then join with Table2. This part might be
> > > tricky??
> > >
> > > Your comments and suggestions are welcome!
> > >
> > > Srikanth
> > >
> >
> >
>
>
> --
>
> *Michael G. Noll | Product Manager | Confluent | +1 650.453.5860Download
> Apache Kafka and Confluent Platform: www.confluent.io/download
> <http://www.confluent.io/download>*
>

Reply via email to