Multiple answers: - a KafkaStreams instance start one *processing* thread by default (you can configure more processing threads, too)
- internally, KafkaStreams uses two KafkaConsumers and one KafkaProducer (if you turn on EOS, it uses even more KafkaProducers): a KafkaConsumer starts a background heartbeat thread and a KafkaProducer starts a background sender thread => you get 4 threads in total (processing, 2x heartbeat, sender) -- if you configure two processing threads, you end up with 8 threads in total, etc) - there is more than one TCP connection as the consumer and the producer (and the restore consumer, if you enable StandbyTasks) connect to the cluster - it's not possible to share any TPC connections atm (this would require a mayor rewrite of consumers and producers) - how many thread you can efficient run depends on your hardward and workload... monitor you CPU utilization and see how buys your machine is... Hope this helps -Matthias On 1/19/18 9:23 AM, Saloni Vithalani wrote: > In our architecture, we are assuming to run three jvm processes on one > machine (approx.) and each jvm machine can host upto 15 kafka-stream apps. > > And if I am not wrong each kafka-stream app spawns one java thread. So, > this seems like an awkward architecture to have with around 45 kafka-stream > apps running on a single machine. > > So, I have question in three parts > > 1) Is my understanding correct that each kafka-stream app spawns one java > thread ? Also, each kafka-stream starts a new tcp connection with > kafka-broker ? > > 2) Is there a way to share one tcp connection for multiple kafka-streams ? > > 3) Is is difficult(not recommended) to run 45 streams on single machine ? > The answer to this is definitely NO unless there is a real use case in > production. > > Regards, > Saloni Vithalani > Developer > Email salo...@thoughtworks.com > Telephone +91 8552889571 <8552889571> > [image: ThoughtWorks] > <http://www.thoughtworks.com/?utm_campaign=saloni-vithalani-signature&utm_medium=email&utm_source=thoughtworks-email-signature-generator> >
signature.asc
Description: OpenPGP digital signature
