I have pipeline defined . pipeline does source(kafka)
,transformation,dedup and persisting to DB .
[image: image.png]
Before reaching to DB task lots of transformation is applied on the
pipeline Therefore want to validate the record with raw json message which
is available in kafka with the tra
If c1 and c2 are listing to the same topic they will consume the same
data .
so i cant understand this
> these two streams one(c2) is fast and other(c1)
On Tue, Jan 29, 2019 at 2:44 PM Selvaraj chennappan <
selvarajchennap...@gmail.com> wrote:
> Team,
>
> I have two kafka consumer for s
Team,
I have two kafka consumer for same topic and want to join second stream to
first after couple of subtasks computation in the first stream then
validate the record . KT - C1 ,C2
KT - C1 - Transformation(FlatMap) - Dedup - Validate --ifvalidsave it to DB
-C2 - Process
