Thanks Boyang
I created this ticket: https://issues.apache.org/jira/browse/KAFKA-9518
I attached a sample maven application that reproduces the issue.
Thanks
Murilo
On Thu, Feb 6, 2020 at 11:38 AM Boyang Chen <reluctanthero...@gmail.com>
wrote:
> Hey Murilo,
>
> feel free to file a JIRA and paste your full topology. It seems like a bug
> to me.
>
> Boyang
>
> On Thu, Feb 6, 2020 at 8:17 AM Murilo Tavares <murilo...@gmail.com> wrote:
>
> > Answering my own question, obviously this is a stateless application, so
> > there’s no reset needed. Mu bad.
> > But the NPE does seem to be linked to the fact that the topics were not
> > sorted, therefore different instances could have different topologies in
> > respect to the order.
> > Sorting the topics seems to have fixed the NPEs.
> > Thanks
> > Murilo
> >
> >
> > On Wed, Feb 5, 2020 at 1:34 PM Murilo Tavares <murilo...@gmail.com>
> wrote:
> >
> > > Hi
> > > I have a KafkaStreams application that's pretty simple, and acts as a
> > > repartitioner... It reads from input topics and send to output topics,
> > > based on a input-to-output topics map. It has a custom Repartitioner
> that
> > > will be responsible for assigning new partitions for the data in the
> > output
> > > topics.
> > >
> > > The topology is simple:
> > > protected Topology buildTopology(Map<String, String> topics) {
> > > StreamsBuilder builder = new StreamsBuilder();
> > > for (Entry<String, String> entry : topics.entrySet()) {
> > > builder
> > > .stream(entry.getKey(), Consumed.with(Serdes.ByteArray(),
> > > Serdes.ByteArray()))
> > > .to(entry.getValue(), Produced.with(Serdes.ByteArray(),
> > > Serdes.ByteArray()));
> > > }
> > > return builder.build();
> > > }
> > >
> > > The resulting topolgy will look like this:
> > > Sub-topology: 0
> > > Source: KSTREAM-SOURCE-0000000000 (topics: [kafka.inputA])
> > > --> KSTREAM-SINK-0000000001
> > > Sink: KSTREAM-SINK-0000000001 (topic: kafka.inputA-repartitioned)
> > > <-- KSTREAM-SOURCE-0000000000
> > >
> > > Sub-topology: 1
> > > Source: KSTREAM-SOURCE-0000000002 (topics: [kafka.inputB])
> > > --> KSTREAM-SINK-0000000003
> > > Sink: KSTREAM-SINK-0000000003 (topic: kafka.inputB-repartitioned)
> > > <-- KSTREAM-SOURCE-0000000002
> > >
> > > Sub-topology: 2
> > > Source: KSTREAM-SOURCE-0000000004 (topics: [kafka.inputC])
> > > --> KSTREAM-SINK-0000000005
> > > Sink: KSTREAM-SINK-0000000005 (topic: kafka.inputC-repartitioned)
> > > <-- KSTREAM-SOURCE-0000000004
> > > ... (about 25 diffent sub-topologies in total)
> > >
> > > I am facing some inconsistent NPE in some environments, that I'll send
> in
> > > the next e-mail not to polute this one anymore.
> > >
> > > But I am wondering if I can really do that, and what would happen if I:
> > > 1) add a new sub-topolgy;
> > > 2) adding the new sub-topology breaks the order of the iteration;
> > >
> > > Will any of the above operations require a reset to the KafkaStreams
> > > application (or new app.id)?
> > >
> > > Thanks
> > > Murilo
> > >
> >
>
