Hey Praveen,
I also suspect that you can get away with far fewer threads. Here's the
general starting point I recommend:
* start with just a little over 1 thread per hardware thread (accounting
for cores and hyperthreading). For example, on my machine, I have 4 cores
with 2 threads of execution e
You are doing something wrong if you need 10k threads to produce 800k
messages per second. It feels you are a factor of 1000 off. What size are
your messages?
On Thu, Sep 13, 2018, 21:04 Praveen wrote:
> Hi there,
>
> I have a kafka application that uses kafka consumer low-level api to help
> us
Hi there,
I have a kafka application that uses kafka consumer low-level api to help
us process data from a single partition concurrently. Our use case is to
send out 800k messages per sec. We are able to do that with 4 boxes using
10k threads and each request taking 50ms in a thread. (1000/50*1000
