Answering my own question, obviously this is a stateless application, so
there’s no reset needed. Mu bad.
But the NPE does seem to be linked to the fact that the topics were not
sorted, therefore different instances could have different topologies in
respect to the order.
Sorting the topics seems to have fixed the NPEs.
Thanks
Murilo
On Wed, Feb 5, 2020 at 1:34 PM Murilo Tavares <murilo...@gmail.com> wrote:
> Hi
> I have a KafkaStreams application that's pretty simple, and acts as a
> repartitioner... It reads from input topics and send to output topics,
> based on a input-to-output topics map. It has a custom Repartitioner that
> will be responsible for assigning new partitions for the data in the output
> topics.
>
> The topology is simple:
> protected Topology buildTopology(Map<String, String> topics) {
> StreamsBuilder builder = new StreamsBuilder();
> for (Entry<String, String> entry : topics.entrySet()) {
> builder
> .stream(entry.getKey(), Consumed.with(Serdes.ByteArray(),
> Serdes.ByteArray()))
> .to(entry.getValue(), Produced.with(Serdes.ByteArray(),
> Serdes.ByteArray()));
> }
> return builder.build();
> }
>
> The resulting topolgy will look like this:
> Sub-topology: 0
> Source: KSTREAM-SOURCE-0000000000 (topics: [kafka.inputA])
> --> KSTREAM-SINK-0000000001
> Sink: KSTREAM-SINK-0000000001 (topic: kafka.inputA-repartitioned)
> <-- KSTREAM-SOURCE-0000000000
>
> Sub-topology: 1
> Source: KSTREAM-SOURCE-0000000002 (topics: [kafka.inputB])
> --> KSTREAM-SINK-0000000003
> Sink: KSTREAM-SINK-0000000003 (topic: kafka.inputB-repartitioned)
> <-- KSTREAM-SOURCE-0000000002
>
> Sub-topology: 2
> Source: KSTREAM-SOURCE-0000000004 (topics: [kafka.inputC])
> --> KSTREAM-SINK-0000000005
> Sink: KSTREAM-SINK-0000000005 (topic: kafka.inputC-repartitioned)
> <-- KSTREAM-SOURCE-0000000004
> ... (about 25 diffent sub-topologies in total)
>
> I am facing some inconsistent NPE in some environments, that I'll send in
> the next e-mail not to polute this one anymore.
>
> But I am wondering if I can really do that, and what would happen if I:
> 1) add a new sub-topolgy;
> 2) adding the new sub-topology breaks the order of the iteration;
>
> Will any of the above operations require a reset to the KafkaStreams
> application (or new app.id)?
>
> Thanks
> Murilo
>
