The default partitioner takes a hash of the key of a topic to determine the
partition number.
It would be useful for a key to be able to specify the object on which the
default partitioner should base its hash on. This would allow us to use
different composite keys and still be certain that the
Hi,
I have a rather complicated kafka streams application involving multiple
joins, aggregates, maps etc. At a certain point, parts of the data needs to
be removed throughout the entire streams topology, both in the topics, the
changelogs and the rocksdb state stores.
Managing this requires a lot
st we could provide an API to inject
> > tombstone records for aggregation logic
> > so that they don't get ignored eventually. This sounds like a good future
> > work.
> >
> > Boyang
> >
> > On Thu, Jan 2, 2020 at 1:47 PM Jan Bols wrote:
> >
>
Hi all,
I'm trying to aggregate a stream of messages and return a stream of
aggregated results using kafka streams.
At some point, depending on the incoming message, the old aggregate needs
to be closed and a new aggregate needs to be created, just like a session
that is closed due to some close ev
Hi all,
I'm trying to aggregate a stream of messages and return a stream of
aggregated results using kafka streams.
At some point, depending on the incoming message, the old aggregate needs
to be closed and a new aggregate needs to be created, just like a session
that is closed due to some close ev
in contrast
> to `groupByKey()`.
>
>
> -Matthias
>
> On 3/25/20 3:49 AM, Jan Bols wrote:
> > Hi all,
> > I'm trying to aggregate a stream of messages and return a stream of
> > aggregated results using kafka streams.
> > At some point, depending on the in
#x27;s also by design that `transform()` does not do auto-repartitioning
> because it's Processor API integration, and when using the Processor API
> it's the developers responsibility to reason about correct data
> partitioning.
>
>
> -Matthias
>
> On 4/1/20
For a kafka-streams application, we keep data per team. Data from 2 teams
never meet but within a team, data is highly integrated. A team has team
members but also has several types of equipment.
A team has a lifespan of about 1-3 days after which the team is removed and
all data relating to that t
