> bootstrap.servers = <BROKER-1-IP:9091> , <BROKER-2-IP:9091>

Is your bootstrap.servers configuration is correct ? You have specified
port `9091`, but running the GetOffsetShell command on `9094`





On Wed, Apr 19, 2017 at 11:58 AM, Ranjith Anbazhakan <
ranjith.anbazha...@aspiresys.com> wrote:

> Unfortunately, there is no specific information (error/exception) in the
> logs when the buffer to queue records data goes missing i.e) When stopped
> broker (say broker 2) is started and followed by stopping current running
> broker that received all producer sent records in buffer (say broker 1).
>
> Thanks,
> Ranjith
>
> -----Original Message-----
> From: David Garcia [mailto:]
> Sent: Wednesday, April 19, 2017 09:31
> To: users@kafka.apache.org
> Subject: Re: Kafka Producer - Multiple broker - Data sent to buffer but
> not in Queue
>
> What do broker logs say around the time you send your messages?
>
> On 4/18/17, 3:21 AM, "Ranjith Anbazhakan" <ranjith.anbazhakan@aspiresys.
> com> wrote:
>
>     Hi,
>
>     I have been testing behavior of multiple broker instances of kafka in
> same machine and facing inconsistent behavior of producer sent records to
> buffer not being available in queue always.
>
>     Tried kafka versions:
>     0.10.2.0
>     0.10.1.0
>
>     Scenario:
>
>     1.       Ran two broker instances in same machine. Say broker 1 as
> initial leader, broker 2 as initial follower.
>
>     2.       Stopped broker 1. Now broker 2 became leader.
>
>     3.       Now producer sends records for a given topic TEST through
> send() method, followed by flush(). Records have to go to Broker 2
> logically. No error/exception is thrown by code. (So it is assumed data has
> been sent successfully to buffer)
>
>     4.       When using command to check the records count for TEST topic
> in Broker 2, the sent records are not added to existing records count for
> that topic in queue.
>
>     a.       Used command - kafka-run-class.bat kafka.tools.GetOffsetShell
> --broker-list localhost:9094 --topic TEST --time -1 (where TEST is the used
> topic)
>
>     NOTE: **Step 4 is not happening always and is inconsistent**. In the
> scenario when it does not work, if Broker 1 is made UP and then made DOWN,
> records are always been available in queue in Broker 2 post doing Step 3.
>
>     Configurations:
>     Overall Producer configurations: (most are default values)
>     acks = all
>                     batch.size = 16384
>                     block.on.buffer.full = false
>                     bootstrap.servers = <BROKER-1-IP:9091> ,
> <BROKER-2-IP:9091>
>                     buffer.memory = 33554432
>                     client.id = producer-1
>                     compression.type = none
>                     connections.max.idle.ms = 540000
>                     interceptor.classes = null
>                     key.serializer = class org.apache.kafka.common.
> serialization.StringSerializer
>                     linger.ms = 1
>                     max.block.ms = 60000
>                     max.in.flight.requests.per.connection = 5
>                     max.request.size = 1048576
>                     metadata.fetch.timeout.ms = 60000
>                     metadata.max.age.ms = 300000
>                     metric.reporters = []
>                     metrics.num.samples = 2
>                     metrics.sample.window.ms = 30000
>                     partitioner.class = class org.apache.kafka.clients.
> producer.internals.DefaultPartitioner
>                     receive.buffer.bytes = 32768
>                     reconnect.backoff.ms = 50
>                     request.timeout.ms = 30000
>                     retries = 0
>                     retry.backoff.ms = 100
>                     sasl.kerberos.kinit.cmd = /usr/bin/kinit
>                     sasl.kerberos.min.time.before.relogin = 60000
>                     sasl.kerberos.service.name = null
>                     sasl.kerberos.ticket.renew.jitter = 0.05
>                     sasl.kerberos.ticket.renew.window.factor = 0.8
>                     sasl.mechanism = GSSAPI
>                     security.protocol = PLAINTEXT
>                     send.buffer.bytes = 131072
>                     ssl.cipher.suites = null
>                     ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
>                     ssl.endpoint.identification.algorithm = null
>                     ssl.key.password = null
>                     ssl.keymanager.algorithm = SunX509
>                     ssl.keystore.location = null
>                     ssl.keystore.password = null
>                     ssl.keystore.type = JKS
>                     ssl.protocol = TLS
>                     ssl.provider = null
>                     ssl.secure.random.implementation = null
>                     ssl.trustmanager.algorithm = PKIX
>                     ssl.truststore.location = null
>                     ssl.truststore.password = null
>                     ssl.truststore.type = JKS
>                     timeout.ms = 30000
>                     value.serializer = class org.apache.kafka.common.
> serialization.StringSerializer
>
>     Broker 1: (server.properties)
>     broker.id=1
>     port=9091
>     advertised.host.name=<System-IP>
>     auto.create.topics.enable=true
>     default.replication.factor=2
>     leader.imbalance.check.interval.seconds=20
>     topic.metadata.refresh.interval.ms=-1
>
>     Broker 2: (server1.properties)
>     broker.id=2
>     port=9094
>     advertised.host.name=<System-IP>
>     auto.create.topics.enable=true
>     default.replication.factor=2
>     leader.imbalance.check.interval.seconds=20
>     topic.metadata.refresh.interval.ms=-1
>
>     Do let know if anyone have faced similar scenarios and why such an
> issue occurs? Let know if any more details are needed.
>
>     Thanks,
>     Ranjith
>     [Aspire Systems]
>
>     This e-mail message and any attachments are for the sole use of the
> intended recipient(s) and may contain proprietary, confidential, trade
> secret or privileged information. Any unauthorized review, use, disclosure
> or distribution is prohibited and may be a violation of law. If you are not
> the intended recipient, please contact the sender by reply e-mail and
> destroy all copies of the original message.
>
>
> [Aspire Systems]
>
> This e-mail message and any attachments are for the sole use of the
> intended recipient(s) and may contain proprietary, confidential, trade
> secret or privileged information. Any unauthorized review, use, disclosure
> or distribution is prohibited and may be a violation of law. If you are not
> the intended recipient, please contact the sender by reply e-mail and
> destroy all copies of the original message.
>

Reply via email to