Hello
I have been developing a Kafka Streams app that takes as input two topics
as KStreams, processes them in some way and joins them and sends the
combined message to an output topic.
Here's some code,
final StreamJoined<String, TransactionEvent, BalanceEvent> joinParams =
StreamJoined.with(
STRING_SERDE,
StreamSerdeConstants.TRANSACTION_EVENT_SERDE,
StreamSerdeConstants.BALANCE_EVENT_SERDE);
JoinWindows joinWindows = JoinWindows
.of(Duration.ofSeconds(streamsProperties.getJoinWindowDuration()))
.grace(Duration.ofSeconds(streamsProperties.getJoinGraceDuration()));
ValueJoiner<TransactionEvent, BalanceEvent, BalanceHistoryEvent> valueJoiner =
(transactionEvent, balanceEvent) -> buildMessage(balanceEvent,
transactionEvent);
transactions
// TODO: change to leftJoin
.join(beWithTransaction, valueJoiner, joinWindows, joinParams)
It's pretty simple, but for my use case I need to process in some way the
messages that are not joined, so I thought I could use a LEFT JOIN. But
according to my tests and this documentation
https://www.confluent.io/blog/crossing-streams-joins-apache-kafka/
I have seen in the end I could end up with both the combined message as the
regular inner join performs and the message with one side as NULL, for
example (A,B) and (A, null)
I thought the JOIN Window could force the output of the left join to just
output if it found a match to just (A,B) not both. Maybe I have a bug in my
Window configuration
Is there a way to force the behavior I need, meaning... using left join and
a JoinWindows output only one message (A,B) or (A, null)
regards
- Miguel
