Hi,
I fixed the earlier issue using returning a valid partition number.
Now I am facing a different error
2019-07-03 04:16:44.334 WARN 1524 --- [ad | producer-1]
o.a.k.clients.producer.internals.Sender : [Producer clientId=producer-1]
Received invalid metadata error in produce request on partition
inbound_topic-1 due to org.apache.kafka.common.errors.NetworkException: The
server disconnected before a response was received.. Going to request
metadata update now
2019-07-03 04:16:44.556 ERROR 1524 --- [ad | producer-1]
o.s.k.support.LoggingProducerListener : Exception thrown when sending a
message with key='378513965-4Energia S.r.l.-Milan-MI-20124' and
payload='com.spgmi.ca.prescore.model.Company@6bd123de' to topic
inbound_topic:
org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for
inbound_topic-1: 122 ms has passed since last attempt plus backoff time
I am using below settings.
acks: 1
retries: 1
batchSize: 10
lingerMs: 5
bufferMemory: 33554432
requestTimeoutMs: 90
What should be corrected here to fix above error.
On Wed, Jul 3, 2019 at 1:18 PM Shyam P <shyamabigd...@gmail.com> wrote:
> Hi SenthilKumar,
> thanks a lot .
>
> Yeah I set up local set up and print the log with partition info which
> shows as below
>
> 2019-07-03 02:48:28.822 INFO 7092 --- [lt-dispatcher-2]
> c.s.c.p.p.CompanyInfoPartitioner : Topic : inbound_topic Key = 597736248-
> Entropy Cayman Solar Ltd.-null-null-null *Partition = -1* 2019-07-03
> 02:48:28.931 ERROR 7092 --- [ad | producer-1]
> o.s.k.support.LoggingProducerListener : Exception thrown when sending a
> message with key='597736248- Entropy Cayman Solar Ltd.-null-null-null' and
> payload='com.spgmi.ca.prescore.model.Company@8b12343' to topic
> inbound_topic :
>
> org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for
> inbound_topic --1: 104 ms has passed since batch creation plus linger time
>
> My topics inbound_topic has two partitions as you see below
> C:\Software\kafka\kafka_2.11-2.1.1\bin\windows>kafka-topics.bat --describe
> --zookeeper localhost:2181 --topic inbound_topic Topic:inbound_topic
> PartitionCount:2 ReplicationFactor:1 Configs: Topic: inbound_topic
> Partition: 0 Leader: 0 Replicas: 0 Isr: 0 Topic: inbound_topic Partition: 1
> Leader: 0 Replicas: 0 Isr: 0
>
> *But my producer seems to trying to send to Partition = -1.*
>
> My partition logic is as below
>
> int p = (((String)key).hashCode() * Integer.MAX_VALUE) % numPartitions;
> logger.info("Topic : "+ topic + "\t Key = " + (String)key + "
> Partition = " + p );
>
> On key i am doing hashCode(). What need to be corrected here to avoid this
> negative number partition number ? i.e. Partition = -1
>
> What should be my partition key logic like ?
>
> any help highly appreciated.
> Regards,
> Shyam
>
> On Tue, Jul 2, 2019 at 8:48 PM SenthilKumar K <senthilec...@gmail.com>
> wrote:
>
>> Does it happen to all partitions or only few partitions ? Can you make
>> sure your local setup working fine ? Were you able to produce using
>> console-producer ?
>>
>> Example :
>> EVERE: Expiring 7 record(s) for topic-9{partition:9}: 30022 ms has passed
>> since last append
>> Expiring 9 record(s) for topic-2{partition:2}: 30015 ms has passed since
>> batch creation plus linger time
>>
>> --Senthil
>>
>> On Tue, Jul 2, 2019 at 5:34 PM Shyam P <shyamabigd...@gmail.com> wrote:
>>
>>> Thanks a lot Senthil for quick reply.
>>> I am using kafka_2.11-2.1.1 .
>>> In your case Kafka Producer Client in One DataCenter and Kafka Broker
>>> in other DataCenter but in my case I installed Kafka on the same machine
>>> where Producer is running.
>>> i.e. currently I am in development mode , so everything now on my local
>>> for timebeing ...i.e. Kafka broker , zk and my producer code in eclipse.
>>>
>>> If is is a set up issue at least it should run fine in my local right.
>>> I tried several producer configurations combinations as explained in the
>>> SOF link.
>>>
>>> So not sure now what is the issue and how to fix it ?
>>>
>>> Is in your case the issue fixed ?
>>>
>>> Regards,
>>> Shyam
>>>
>>> On Tue, Jul 2, 2019 at 5:12 PM SenthilKumar K <senthilec...@gmail.com>
>>> wrote:
>>>
>>>> Hi Shyam, We also faced `TimeoutException: Expiring 1 record(s)` issue
>>>> in
>>>> our Kafka Producer Client. As described here
>>>> <
>>>> https://stackoverflow.com/questions/56807188/how-to-fix-kafka-common-errors-timeoutexception-expiring-1-records-xxx-ms-has
>>>> >
>>>> ,
>>>> first we tried increasing request timeout but that didn't help. We had
>>>> setup like Kafka Producer Client in One DataCenter and Kafka Broker in
>>>> other DataCenter & thats why the producer failed to push records to
>>>> brokers
>>>> on time due to network issue. In your case , Could be setup issue ?
>>>>
>>>> --Senthil
>>>>
>>>> On Tue, Jul 2, 2019 at 3:57 PM Shyam P <shyamabigd...@gmail.com> wrote:
>>>>
>>>> > Hi,
>>>> > I am facing the below issue.
>>>> >
>>>> > org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s)
>>>> > for <topic-name> 229 ms has passed since batch creation plus linger
>>>> > time
>>>> >
>>>> >
>>>> > I tried many producer configuration settings. more details below :
>>>> >
>>>> >
>>>> https://stackoverflow.com/questions/56807188/how-to-fix-kafka-common-errors-timeoutexception-expiring-1-records-xxx-ms-has
>>>> >
>>>> > But nothing working.
>>>> >
>>>> > Can anyone plz help me , what is wrong here and how to fix it ?
>>>> >
>>>> > thanks,
>>>> > Shyam
>>>> >
>>>>
>>>
