Hey guys, I have no idea if this would be reasonable, but what about just running two Kafka processes on the bigger box?
Cheers, Chris On 5/17/13 2:48 PM, "Jason Rosenberg" <j...@squareup.com> wrote: >Just resource allocation issues. E.g. imagine having an existing kafka >cluster with one machine spec, and getting access to a few more hosts to >augment the cluster, which are newer and therefore have twice the disk >storage. I'd like to seamlessly add them into the cluster, without having >to replace everything en masse. Thus, it would be nice for the newer ones >to take proportionally more load based on the relative storage available, >etc. > >Jason > > >On Fri, May 17, 2013 at 2:34 PM, Neha Narkhede ><neha.narkh...@gmail.com>wrote: > >> That does seem a little hacky. But I'm trying to understand the >>requirement >> behind having to deploy heterogeneous hardware. What are you trying to >> achieve or optimize? >> >> Thanks, >> Neha >> >> >> On Fri, May 17, 2013 at 2:29 PM, Jason Rosenberg <j...@squareup.com> >>wrote: >> >> > Hi, >> > >> > I'm wondering if there's a good way to have a heterogenous kafka >>cluster >> > (specifically, if we have nodes with different sized disks). So, we >> might >> > want a larger node to receive more messages than a smaller node, etc. >> > >> > I expect there's something we can do with using a partitioner that has >> > specific knowledge about the hosts in the cluster, but this feels >>messy, >> to >> > have this config on every producer client.... >> > >> > Thoughts? >> > >> > Jason >> > >>
