Generally we discourage colocating services with Kafka. Kafka relies heavily on the page cache. It's generally light on CPU (except maybe if it has to recompress messages), but may not play well with other services.
For very light installations, colocating some services (e.g. both ZK and Kafka), might work fine. For most production settings, you'll want to keep them separate unless you're *very* sure that they won't interact poorly. -Ewen On Wed, Jun 29, 2016 at 1:19 AM, Kristoffer Sjögren <sto...@gmail.com> wrote: > Hi > > We want to use Kafka Connect to copy data to HDFS (using > kafka-connect-hdfs) in parquet format and was wondering if its a good > idea to collocate distributed Kafka Connect 1-1 on Kafka Brokers? > > Considering the parquet indexing process would steal (a lot of / too > much?) computing resources from the brokers. The alternative is to > deploy on separate hardware (1-1?). > > Experiences in this area? > > Cheers, > -Kristoffer > -- Thanks, Ewen
