Hello Mangat,
I am using Kstreams, does this make a difference?
Anyways, I've read about stateful stream processing here:
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Streams+Internal+Data+Management
I think I can manage to solve my issue using changelog topics for my state
store. I'll
Urko,
You can enable changelog topics for your state store. This will enable the
application to persist the data to a Kafka topic. Next time when
application start, it will first build it's state by using this topic. Are
you using Kstreams or the low-level processor API?
Regards,
Mangat
On Mon,
It's self-service.
Please follow the instructions from the web page:
https://kafka.apache.org/contact
-Matthias
On 6/7/20 4:49 PM, wang120445...@sina.com wrote:
> hi kafka:
>I want to apply Contributor's List.
> end
>
>
>
> wang120445...@sina.com
>
One other option is to use something like Druid, especially if you care
about doing arbitrary dimensional drilldowns.
http://druid.io
It reads from Kafka and can do simple rollups for you automatically
(meaning you don't need storm if all you are doing with Storm is a simple
"group by" style roll
You could also use something more oriented at timeseries data like
https://github.com/rackerlabs/blueflood/. Then you'd have to write some
output adapters to feed the additional processing of your data elsewhere.
I think the team is working on making an output adapter for Kafka for the
rolled-up m
This is a very common problem in my experience. Late-arriving and
semi-ordered data make a lot of stream processing problems more difficult.
Are you able to perform analysis with part of the data? For instance
buffering some number of events and then analyze?
How exactly do you know definitively
Well, you can only store data in Kafka, you can't put application logic in
there.
Storm is good for processing data, but it is not a data store, so that is out.
Redis might work, but it is only an in-memory store (seems like it does have
persistence, but I don't know much about that).
You cou
Hi Ahmed,
I can share with you my experience, I have built a system similar to yours.
1. If all your messages are the same, I think you should use the default
partitioner, so the messages will spread evenly across all the
brokers/partition combinations, unless you have a better function to
s
Partition is useful for increasing the degree of parallelism of consumers
and to a certain degree, producers too. You can have multiple consumer
groups consuming the same topic/partition.
Thanks,
Jun
On Sun, Jan 27, 2013 at 1:06 PM, S Ahmed wrote:
> Say I create web application/service where c
