Yeah, I thought of that (running 2 kafkas on one box), but it doesn't really add the benefit of redundancy through replication (e.g. if we have 2 replicas mapping to the same physical machine).
Jason On Fri, May 17, 2013 at 2:50 PM, Chris Riccomini <criccom...@linkedin.com>wrote: > Hey guys, > > I have no idea if this would be reasonable, but what about just running > two Kafka processes on the bigger box? > > Cheers, > Chris > > On 5/17/13 2:48 PM, "Jason Rosenberg" <j...@squareup.com> wrote: > > >Just resource allocation issues. E.g. imagine having an existing kafka > >cluster with one machine spec, and getting access to a few more hosts to > >augment the cluster, which are newer and therefore have twice the disk > >storage. I'd like to seamlessly add them into the cluster, without having > >to replace everything en masse. Thus, it would be nice for the newer ones > >to take proportionally more load based on the relative storage available, > >etc. > > > >Jason > > > > > >On Fri, May 17, 2013 at 2:34 PM, Neha Narkhede > ><neha.narkh...@gmail.com>wrote: > > > >> That does seem a little hacky. But I'm trying to understand the > >>requirement > >> behind having to deploy heterogeneous hardware. What are you trying to > >> achieve or optimize? > >> > >> Thanks, > >> Neha > >> > >> > >> On Fri, May 17, 2013 at 2:29 PM, Jason Rosenberg <j...@squareup.com> > >>wrote: > >> > >> > Hi, > >> > > >> > I'm wondering if there's a good way to have a heterogenous kafka > >>cluster > >> > (specifically, if we have nodes with different sized disks). So, we > >> might > >> > want a larger node to receive more messages than a smaller node, etc. > >> > > >> > I expect there's something we can do with using a partitioner that has > >> > specific knowledge about the hosts in the cluster, but this feels > >>messy, > >> to > >> > have this config on every producer client.... > >> > > >> > Thoughts? > >> > > >> > Jason > >> > > >> > >
