Currently, we don't support embedded consumers in the broker. Doing that requires running user-level code in the broker, which we haven't thought about.
Are you concerned about network latency? That's probably in the order of 1ms. In terms of throughput, if you make fetch size big enough, you can achieve good throughput even with a remote consumer, since the network overhead is amortized. Thanks, Jun On Fri, Mar 1, 2013 at 3:05 PM, Sining Ma <sinin...@gmail.com> wrote: > Hi > > I have a question about kafka servers. > Can I generate several consumer thread when I start kafka servers? > In other words, right now in our program kafka servers and consumers are > two different processes. We use SimpleConsumer instance and do > inter-process communication in order to fetch messages. We want to optimize > our system architecture through integrating consumers into kafka servers. > In this way, consumers will become kafka server threads and we can read > messages from shared memory. This will be more efficient than current > design. > > Are there any api that allow me to do this or I can pass some flags to > kafka server to invoke this functionality? > > -- > Regards, > Sining Ma >
