Correct. As long as there are only LOG statements about it, you don't need to worry. If the producer cannot recover from this error internally, it would throw an `UnknowdProducerIdException`; for this case, the write was _not_ successful. Your application could handle this case by creating a new `KafkaProducer` with the same `transactional.id` and retry the transaction.
Note that KIP-360 is partly implemented in 2.4.0 already what should reduce the likelihood to hit this issue. -Matthias On 12/1/19 9:42 AM, Jose Manuel Vega Monroy wrote: > Hi Matthias, > > So looking will be fixed in Kafka 2.5.0, as per > https://issues.apache.org/jira/browse/KAFKA-8710, so we will take into > account to upgrade properly to that version when ready and stable. > > We see this kind of errors in broker logs: > > [2019-12-01 15:27:39,790] ERROR [ReplicaManager broker=3] Error processing > append operation on partition OB_SP11-12 (kafka.server.ReplicaManager) > org.apache.kafka.common.errors.UnknownProducerIdException: Found no record of > producerId=23001 on the broker at offset 24761in partition OB_SP11-12. It is > possible that the last message with the producerId=23001 has been removed due > to hitting the retention limit. > > Just to confirm, I guess they are correlated to producer errors right? > > Thanks > > <http://www.williamhill.com/> > <http://www.whenthefunstops.co.uk/> > Jose Manuel Vega Monroy > Java Developer / Software Developer Engineer in Test > Direct: +0035 0 2008038 (Ext. 8038) > Email: jose.mon...@williamhill.com > William Hill | 6/1 Waterport Place | Gibraltar | GX11 1AA > > > > > On 12/10/2019, 22:11, "Matthias J. Sax" <matth...@confluent.io> wrote: > > >> Does it mean producer’s metadata was removed due to retention time, > > Yes. > > >> and message is failed to be sent? > > No. Note, it's a warning not an error. > > The producer resets its internal state for this case and will send the > message successfully. It's a known issue and addressed via KIP-360 that > provides a more detailed explanation: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-360%3A+Improve+handling+of+unknown+producer > > > -Matthias > > On 10/12/19 12:18 PM, Jose Manuel Vega Monroy wrote: > > Hi there, > > > > > > > > We are seeing WARN log traces from time to time in our Kafka producer: > > > > > > > > 2019-10-12 15:19:14:707 > > WARN[org.apache.kafka.clients.producer.internals.Sender] [Producer > > clientId=prdxpds21dsm001.prod.williamhill.plc] Got error produce > > response with correlation id 4899937 on topic-partition OB_SP25-3, > > retrying (0 attempts left). Error: UNKNOWN_PRODUCER_ID > > > > > > > > 2019-10-1220:02:21:686 WARN > > [org.apache.kafka.clients.producer.internals.Sender] [Producer > > clientId=prdxpds21dsm001.prod.williamhill.plc] Got errorproduce response > > with correlation id 5354758 on topic-partition OB_SP87-16, retrying (0 > > attempts left). Error:UNKNOWN_PRODUCER_ID > > > > > > > > Does it mean producer’s metadata was removed due to retention time, and > > message is failed to be sent? > > > > > > > > I’m not sure but looking like happening since we enabled in producer: > > > > > > > > *enable.idempotence=true* > > > > > > > > Any idea? > > > > > > > > Thanks > > > > > > > > https://www.williamhillplc.com/content/signature/WHlogo.gif?width=180 > > <http://www.williamhill.com/> > > > > https://www.williamhillplc.com/content/signature/senet.gif?width=180 > > <http://www.whenthefunstops.co.uk/> > > > > > > > > *Jose Manuel Vega Monroy ** > > **Java Developer / Software Developer Engineer in Test* > > > > Direct: +*0035 0 2008038 (Ext. 8038)* > > Email: jose.mon...@williamhill.com <mailto:jose.mon...@williamhill.com> > > > > William Hill | 6/1 Waterport Place | Gibraltar | GX11 1AA > > > > > > > > > > > > > > > > > > Confidentiality: The contents of this e-mail and any attachments transmitted > with it are intended to be confidential to the intended recipient; and may be > privileged or otherwise protected from disclosure. If you are not an intended > recipient of this e-mail, do not duplicate or redistribute it by any means. > Please delete it and any attachments and notify the sender that you have > received it in error. This e-mail is sent by a William Hill PLC group > company. The William Hill group companies include, among others, William Hill > PLC (registered number 4212563), William Hill Organization Limited > (registered number 278208), William Hill US HoldCo Inc, WHG (International) > Limited (registered number 99191) and Mr Green Limited (registered number > C43260). Each of William Hill PLC and William Hill Organization Limited is > registered in England and Wales and has its registered office at 1 Bedford > Avenue, London, WC1B 3AU, UK. William Hill U.S. HoldCo, Inc. is registered in > Delaware and has its registered office at 1007 N. Orange Street, 9 Floor, > Wilmington, New Castle County DE 19801 Delaware, United States of America. > WHG (International) Limited is registered in Gibraltar and has its registered > office at 6/1 Waterport Place, Gibraltar. Mr Green Limited is registered in > Malta and has its registered office at Tagliaferro Business Centre, Level 7, > 14 High Street, Sliema SLM 1549, Malta. Unless specifically indicated > otherwise, the contents of this e-mail are subject to contract; and are not > an official statement, and do not necessarily represent the views, of William > Hill PLC, its subsidiaries or affiliated companies. Please note that neither > William Hill PLC, nor its subsidiaries and affiliated companies can accept > any responsibility for any viruses contained within this e-mail and it is > your responsibility to scan any emails and their attachments. William Hill > PLC, its subsidiaries and affiliated companies may monitor e-mail traffic > data and also the content of e-mails for effective operation of the e-mail > system, or for security, purposes. >
signature.asc
Description: OpenPGP digital signature
