Hi Jens, Thanks for your suggestion. So here is a bit more text:
I have a List of signals for a specific device: Map[DeviceId, List[Signal]], where Signal is case class Signal(name: String, value: String) Now I will write a producer to connect to the device and read these signals at constant intervals. I want to know how I should create my topic assuming that I have say 10 devices to start with, later scaling it up to 100000 such devices. How should I model my topic? Should I create one topic per device? Thanks and Regards, Joe On Tue, Jan 19, 2016 at 4:58 PM, Jens Rantil <jens.ran...@tink.se> wrote: > Hi Joe, > > I think you are leaving out all your requirements to be able to get decent > answer from anyone here. > > On Tue, Jan 19, 2016 at 8:47 AM, Joe San <codeintheo...@gmail.com> wrote: > > > I soon realized that this is a doomed approach. > > > > Could you elaborate? Why would it be doomed? > > > > Is there a better way to model my topic so that I could > > guarantee replication and high throughput and scale if I need to add > 10,000 > > of those devices? Any suggestions? > > > > What does "high throughput and scale" mean? What is "high throughput" for > you? What is "scale" for you? > > Let's start with a simple proposal: How about having a single topic with a > single partition and you replicate it to two other brokers? You can > increase the number of partitions in the future and you have your > "guaranteed" (whatever that means) replication. > > Cheers, > Jens > > -- > Jens Rantil > Backend engineer > Tink AB > > Email: jens.ran...@tink.se > Phone: +46 708 84 18 32 > Web: www.tink.se > > Facebook <https://www.facebook.com/#!/tink.se> Linkedin > < > http://www.linkedin.com/company/2735919?trk=vsrp_companies_res_photo&trkInfo=VSRPsearchId%3A1057023381369207406670%2CVSRPtargetId%3A2735919%2CVSRPcmpt%3Aprimary > > > Twitter <https://twitter.com/tink> >
