Hi, I'm using Kafka 0.8.2.1 SimpleConsumer . Could somebody clarify the meaning of a few config parameters for the SimpleConsumer and FetchREquestBuilder? My questions are below: -- Q1: in the signature of the SimpleConsumer constructor I see the Int 'soTimeout' parameter - what is the meaning of this timeout? Is this a timeout to connect to the Kafka broker? Timeout on getting a response from any [or specific??] request to Kafka (like FetchRequest)? Something else? kafka.javaapi.consumer.SimpleConsumer (val host: String, val port: Int, val soTimeout: Int, val bufferSize: Int, val clientId: String)
-- Q2: also, SimpleConsumer constructor takes Int 'bufferSize' parameter. What is the meaning of it? Is this how many bytes SimpleConsumer will read when a fetchRequest is issued? -- When building FetchREquest via FetchRequestBuilder (see below), I also need to specify 'fetchSize': FetchRequest req= newFetchRequestBuilder () .clientId(kafkaGroupId) .addFetch(topic, partition, offset, fetchSizeInBytes) .build(); Looking at the source code of the FetchRequestBuilder , I think (I'm not a Scala pro) those calls translate into the below method calls - and there the final parameter passed into the FetchRequest is called 'minBytes', hinting that this is not the exact fetch size, possibly…. class FetchRequestBuilder(): def addFetch(topic: String, partition: Int, offset: Long, fetchSize: Int) def build() = { val fetchRequest= FetchRequest(versionId, correlationId.getAndIncrement, clientId, replicaId, maxWait, minBytes, requestMap.toMap) FetchRequest(versionId: Short = FetchRequest.CurrentVersion, correlationId: Int = FetchRequest.DefaultCorrelationId, clientId: String = ConsumerConfig.DefaultClientId, replicaId: Int = Request.OrdinaryConsumerId, maxWait: Int = FetchRequest.DefaultMaxWait, minBytes: Int = FetchRequest.DefaultMinBytes, ...) So, my final question is: -- Q3: how do 'bufferSize' and 'fetchSize/minBytes' relate? What exactly do they define? Do I have to make sure one is smaller or grater than the other? Thanks, Marina