Hi, thank you for your reply.
I have different devices in different regions. When a device is started it opens a websocket connection, where the region and details about the device are delivered once the connection is opened. After this, all messages from this device will be sent via this opened connection (every minute), so a message only contains the value (sensor reading). Therefore my question was if I need a topic for every opened websocket connection. Or how do I know which region it is? Can I add this information in Kafka? What exactly do you mean with Kafka Stream? Do I not need Storm or Spark Streaming for the clustering and aggregation? Can I solve my use case with Kafka only? My thoughts was to use Kafka and Storm/Spark Streaming and to persist the results to HBase or Redis. Thank you in advance. Regards, Daniela Gesendet: Mittwoch, 23. März 2016 um 13:02 Uhr Von: "Manikumar Reddy" <ku...@nmsworks.co.in> An: "users@kafka.apache.org" <users@kafka.apache.org> Betreff: Re: Re: Topics in Kafka Hi, you can have single topic with multiple partitions. It looks like you have messages with key=region and value=sensor reading, and you want to run some aggregate, windowing operations by time. Kafka Streams is perfect fit for this use case. On Wed, Mar 23, 2016 at 2:30 PM, Maria Musterfrau <daniela_4...@gmx.at> wrote: > Hi, > > Thank you for your reply. > > I would like to cluster the values (every message contains one value) > after their region and to add the values up in realtime afterwards (the > actual/last minute). > > I would like to use Kafka to subscribe the messages to a stream processing > framework like Storm or Spark Streaming and to use a time window because I > would like to add up the values of the last minute. > > Would it be better to use different topics for different regions or to use > one topic and to do the clustering in Storm or Spark Streaming afterwards? > > Thank you in advance. > > Regards, > Daniela > > > > Gesendet: Mittwoch, 23. März 2016 um 09:42 Uhr > Von: "Manikumar Reddy" <ku...@nmsworks.co.in> > An: "users@kafka.apache.org" <users@kafka.apache.org> > Betreff: Re: Topics in Kafka > Hi, > > 1. Based on your design, it can be one or more topics. You can design one > topic per region or > one topic for all region devices. > > 2. Yes, you need to listen to web socket messages and write to kafka > server using kafka producer. > In your use case, you can also send messages using Kafka Rest Proxy > from confluent. > http://docs.confluent.io/2.0.1/kafka-rest/docs/index.html > > > On Wed, Mar 23, 2016 at 1:59 PM, Maria Musterfrau <daniela_4...@gmx.at> > wrote: > > > Hi > > > > I am new and I have a question regarding Kafka. I have devices in > > different regions. Every device opens a websocket connection when it gets > > active and it sends its messages over the opened websocket connection to > a > > server. My question is: is every region a topic or is every websocket > > connection (means every device) a topic? Or can I choose which one is > used > > as topic? > > > > Is it true that I need a piece of code as Kafka cannot consume messages > > directly from the websocket connection? > > > > Thank you and regards, > > Daniela > > >
