Dear community, I have a quite tough problem I struggle to diagnose and fix. I'm not sure if it's a bug in Kafka-streams or some subtlety I didn't get in using the DSL api.
The problem is the following. We have a quite elaborate stream app, working well, in production. We'd like to add a left join with a KTable (data is coming from a DB via kafka connect jdbc source). So we end-up with a topology like this: Event Source ---- (some transformations and joins) ----- leftJoin(A: KTable) ----- leftJoin(B: KTable) ---- sinks The new leftjoin is the one joining A. The transformations are several custom Transformers. In tests with TopologyTestDriver, all is good, we can validate the general logic. In integration tests with a real Kafka (in a docker, though), we can't manage to have both left joins work at the same time ! The leftJoin with `A` always return null. I ran dozen of tests, tweaking and fiddling everything, and I found out that it's related to partitioning. If I force the number of partitions down to 1 (by setting all input topics to 1 partition), the join works. In one of the tests, I suspected one of the transformations, so I removed it. The topology shown by describe() changed quite significantly (going from 2 subtoplogies to 1), and this made the leftJoin with A work...and the leftJoin with B fail. It drives me crazy. Activating the optimization didn't help. The input topics for KTables A and B are read with a TimestamExtractor to 0, since this is static data, to make sure we don't run into timestamp ordering issues. We double-checked and tripled-checked the keys in various stages, and we're sure they're good (by the way, it works with 1 partition). Partitioner is always the default everywhere (in inputs, kafka-connect...), actually we never touch this. Actually it seems related to repartitioning placed in the topology by StreamsBuilder (probably in relation to transformers ?) So, I imagine you can't help much without seeing the code, but if you think of anything that could help diagnosing this further, please tell. Mathieu
