Re: kafka connection with zookeeper

[Ben Stopford]

Hi Sadanand

Kafka secures it’s connection with Zookeeper via SASL and it’s a little 
different to the way brokers secure connections between themselves and with 
clients. 

There’s more info here: 
http://docs.confluent.io/2.0.0/kafka/zookeeper-authentication.html 


This applies to the latest release of Kafka

Best regards

Ben
> On 11 Dec 2015, at 19:54, sadanand.kulka...@wipro.com wrote:
> 
> Hi there!
> 
>  Couldn’t find anything so far! please help..
> 
>  - Can Kafka broker communicates with Zookeeper using secure (SSL/TLLS) 
> protocol?  If yes, which zookeeper and Kafka versions supports it?
> 
> Thanks a ton!
> 
> Appreciate for your time and help!
> 
> Regards
> Sadanand
> The information contained in this electronic message and any attachments to 
> this message are intended for the exclusive use of the addressee(s) and may 
> contain proprietary, confidential or privileged information. If you are not 
> the intended recipient, you should not disseminate, distribute or copy this 
> e-mail. Please notify the sender immediately and destroy all copies of this 
> message and any attachments. WARNING: Computer viruses can be transmitted via 
> email. The recipient should check this email and any attachments for the 
> presence of viruses. The company accepts no liability for any damage caused 
> by any virus transmitted by this email. www.wipro.com

Kafka Producer 0.9 performance issue with small messages

[Gary Gershon]

I’m moving this issue from Stack Overflow to the Apache Kafka Users List

http://stackoverflow.com/questions/34213272/kafka-producer-0-9-performance-issue-with-small-messages
 


There was some discussion on Stack Overflow, but I’m expecting the Apache List 
will be better able to help in identifying the root cause since it appears not 
necessarily to be a BlueMix Message Hub issue.  Thanks!

-

We are observing very poor performance with a Java Kafka Producer 0.9 client 
when sending small messages.  The messages are not being accumulated into a 
larger request batch and thus each small record is being sent separately.

What is wrong with our client configuration?  Or is this some other issue?

-

Using Kafka Client 0.9.0.0.  We did not see any related postings in the Kafka  
unreleased 9.0.1 or 9.1 fixed or unresolved lists, so we are focused on our 
client configuration and server instance.

We understand the linger.ms should cause the client to accumulate records into 
a batch.

We set linger.ms to 10 (and also tried 100 and 1000) but these did not result 
in the batch accumulating records.  With a record size of about 100 bytes and a 
request buffer size of 16K, We would have expected about 160 messages to be 
sent in a single request.

The trace at the client seems to indicate that the partition may be full, 
despite having allocated a fresh Bluemix Messaging Hub (Kafka Server 0.9) 
service instance.  The test client is sending multiple messages in a loop with 
no other I/O.  

-

The log shows a repeating sequence with a suspect line: "**Waking up the sender 
since topic mytopic partition 0 is either full or getting a new batch**".

So the newly allocated partition should be essentially empty in our test case, 
thus why would the producer client be getting a new batch?


2015-12-10 15:14:41,335 3677 [main] TRACE 
com.isllc.client.producer.ExploreProducer  - Sending record: Topic='mytopic', 
Key='records', Value='Kafka 0.9 Java Client Record Test Message 00011 
2015-12-10T15:14:41.335-05:00'  
2015-12-10 15:14:41,336 3678 [main] TRACE 
org.apache.kafka.clients.producer.KafkaProducer  - Sending record 
ProducerRecord(topic=mytopic, partition=null, key=[B@670b40af, 
value=[B@4923ab24 with callback null to topic mytopic partition 0  
2015-12-10 15:14:41,336 3678 [main] TRACE 
org.apache.kafka.clients.producer.internals.RecordAccumulator  - Allocating a 
new 16384 byte message buffer for topic mytopic partition 0  
2015-12-10 15:14:41,336 3678 [main] TRACE 
org.apache.kafka.clients.producer.KafkaProducer  - Waking up the sender since 
topic mytopic partition 0 is either full or getting a new batch  
2015-12-10 15:14:41,348 3690 [kafka-producer-network-thread | ExploreProducer] 
TRACE org.apache.kafka.clients.producer.internals.Sender  - Nodes with data 
ready to send: [Node(0, 
kafka01-prod01.messagehub.services.us-south.bluemix.net, 9094)]  
2015-12-10 15:14:41,348 3690 [kafka-producer-network-thread | ExploreProducer] 
TRACE org.apache.kafka.clients.producer.internals.Sender  - Created 1 produce 
requests: [ClientRequest(expectResponse=true, 
callback=org.apache.kafka.clients.producer.internals.Sender$1@6d62e963, 
request=RequestSend(header={api_key=0,api_version=1,correlation_id=11,client_id=ExploreProducer},
 
body={acks=-1,timeout=3,topic_data=[{topic=mytopic,data=[{partition=0,record_set=java.nio.HeapByteBuffer[pos=0
 lim=110 cap=16384]}]}]}), createdTimeMs=1449778481348, sendTimeMs=0)]  
2015-12-10 15:14:41,412 3754 [kafka-producer-network-thread | ExploreProducer] 
TRACE org.apache.kafka.clients.producer.internals.Sender  - Received produce 
response from node 0 with correlation id 11  
2015-12-10 15:14:41,412 3754 [kafka-producer-network-thread | ExploreProducer] 
TRACE org.apache.kafka.clients.producer.internals.RecordBatch  - Produced 
messages to topic-partition mytopic-0 with base offset offset 130 and error: 
null.  
2015-12-10 15:14:41,412 3754 [main] TRACE 
com.isllc.client.producer.ExploreProducer  - Send returned metadata: 
Topic='mytopic', Partition=0, Offset=130  
2015-12-10 15:14:41,412 3754 [main] TRACE 
com.isllc.client.producer.ExploreProducer  - Sending record: Topic='mytopic', 
Key='records', Value='Kafka 0.9 Java Client Record Test Message 00012 
2015-12-10T15:14:41.412-05:00'

Log entries repeat like the above for each record sent


--

We provided the following properties file:


2015-12-10 15:14:37,843 185  [main] INFO  com.isllc.client.AbstractClient  - 
Properties retrieved from file for Kafka client: kafka-producer.properties
2015-12-10 15:14:37,909 251  [main] INFO  com.isllc.client.AbstractClient  -
 acks=-1
2015-12-10 15:14:37,909 251  [main] INFO  com.isllc.client.AbstractClient  -
 ssl.protocol=TLSv1.2
2015-12-10 15:14:37,909 251  [main] INFO  com.isllc.client.AbstractClient  -
 key.serializer=org.apache.kafka.common.serialization.Byte

Re: Kafka Producer 0.9 performance issue with small messages

[Guozhang Wang]

Gary,

So you observed records-per-request-avg = 1.0 still with linger.ms = 100 or
1000?

It seems you are not use Kafka's ProducerPerformance but are using
your own ExploreProducer
implementation, could you elaborate a bit about how messages are piped to
this client?

Guozhang


On Sat, Dec 12, 2015 at 11:34 AM, Gary Gershon 
wrote:

> I’m moving this issue from Stack Overflow to the Apache Kafka Users List
>
>
> http://stackoverflow.com/questions/34213272/kafka-producer-0-9-performance-issue-with-small-messages
> <
> http://stackoverflow.com/questions/34213272/kafka-producer-0-9-performance-issue-with-small-messages
> >
>
> There was some discussion on Stack Overflow, but I’m expecting the Apache
> List will be better able to help in identifying the root cause since it
> appears not necessarily to be a BlueMix Message Hub issue.  Thanks!
>
> -
>
> We are observing very poor performance with a Java Kafka Producer 0.9
> client when sending small messages.  The messages are not being accumulated
> into a larger request batch and thus each small record is being sent
> separately.
>
> What is wrong with our client configuration?  Or is this some other issue?
>
> -
>
> Using Kafka Client 0.9.0.0.  We did not see any related postings in the
> Kafka  unreleased 9.0.1 or 9.1 fixed or unresolved lists, so we are focused
> on our client configuration and server instance.
>
> We understand the linger.ms should cause the client to accumulate records
> into a batch.
>
> We set linger.ms to 10 (and also tried 100 and 1000) but these did not
> result in the batch accumulating records.  With a record size of about 100
> bytes and a request buffer size of 16K, We would have expected about 160
> messages to be sent in a single request.
>
> The trace at the client seems to indicate that the partition may be full,
> despite having allocated a fresh Bluemix Messaging Hub (Kafka Server 0.9)
> service instance.  The test client is sending multiple messages in a loop
> with no other I/O.
>
> -
>
> The log shows a repeating sequence with a suspect line: "**Waking up the
> sender since topic mytopic partition 0 is either full or getting a new
> batch**".
>
> So the newly allocated partition should be essentially empty in our test
> case, thus why would the producer client be getting a new batch?
>
> 
> 2015-12-10 15:14:41,335 3677 [main] TRACE
> com.isllc.client.producer.ExploreProducer  - Sending record:
> Topic='mytopic', Key='records', Value='Kafka 0.9 Java Client Record Test
> Message 00011 2015-12-10T15:14:41.335-05:00'
> 2015-12-10 15:14:41,336 3678 [main] TRACE
> org.apache.kafka.clients.producer.KafkaProducer  - Sending record
> ProducerRecord(topic=mytopic, partition=null, key=[B@670b40af,
> value=[B@4923ab24 with callback null to topic mytopic partition 0
> 2015-12-10 15:14:41,336 3678 [main] TRACE
> org.apache.kafka.clients.producer.internals.RecordAccumulator  - Allocating
> a new 16384 byte message buffer for topic mytopic partition 0
> 2015-12-10 15:14:41,336 3678 [main] TRACE
> org.apache.kafka.clients.producer.KafkaProducer  - Waking up the sender
> since topic mytopic partition 0 is either full or getting a new batch
> 2015-12-10 15:14:41,348 3690 [kafka-producer-network-thread |
> ExploreProducer] TRACE org.apache.kafka.clients.producer.internals.Sender
> - Nodes with data ready to send: [Node(0,
> kafka01-prod01.messagehub.services.us-south.bluemix.net, 9094)]
> 2015-12-10 15:14:41,348 3690 [kafka-producer-network-thread |
> ExploreProducer] TRACE org.apache.kafka.clients.producer.internals.Sender
> - Created 1 produce requests: [ClientRequest(expectResponse=true,
> callback=org.apache.kafka.clients.producer.internals.Sender$1@6d62e963,
> request=RequestSend(header={api_key=0,api_version=1,correlation_id=11,client_id=ExploreProducer},
> body={acks=-1,timeout=3,topic_data=[{topic=mytopic,data=[{partition=0,record_set=java.nio.HeapByteBuffer[pos=0
> lim=110 cap=16384]}]}]}), createdTimeMs=1449778481348, sendTimeMs=0)]
> 2015-12-10 15:14:41,412 3754 [kafka-producer-network-thread |
> ExploreProducer] TRACE org.apache.kafka.clients.producer.internals.Sender
> - Received produce response from node 0 with correlation id 11
> 2015-12-10 15:14:41,412 3754 [kafka-producer-network-thread |
> ExploreProducer] TRACE
> org.apache.kafka.clients.producer.internals.RecordBatch  - Produced
> messages to topic-partition mytopic-0 with base offset offset 130 and
> error: null.
> 2015-12-10 15:14:41,412 3754 [main] TRACE
> com.isllc.client.producer.ExploreProducer  - Send returned metadata:
> Topic='mytopic', Partition=0, Offset=130
> 2015-12-10 15:14:41,412 3754 [main] TRACE
> com.isllc.client.producer.ExploreProducer  - Sending record:
> Topic='mytopic', Key='records', Value='Kafka 0.9 Java Client Record Test
> Message 00012 2015-12-10T15:14:41.412-05:00'
>
> Log entries repeat like the above for each record sent
> 
>
> --
>
> We provided the following properties file:
>
> 
> 2015-12-10 15:14:37,843 185  [main