The new producer should work with the 0.8.0 broker. Thanks,
Jun On Tue, May 6, 2014 at 11:30 AM, Bhavesh Mistry <mistry.p.bhav...@gmail.com>wrote: > So to retrieve the message that was not transmitted due to error, I will > have to store the message in each call back the message and store this in a > file. Is there an alternative solution ? > > What is best way to achieve QOS on producer side ? > > Also, will new Producer API will work with Broker version 0.8.0 ? Please > let me know. > > Thanks, > > Bhavesh > > > On Tue, May 6, 2014 at 8:46 AM, Jun Rao <jun...@gmail.com> wrote: > > > The callback will be invoked on completion. The exception tells you > whether > > the send completed or not. This api will be available in the next > release. > > > > Thanks, > > > > Jun > > > > > > On Mon, May 5, 2014 at 10:03 PM, Bhavesh Mistry > > <mistry.p.bhav...@gmail.com>wrote: > > > > > Thanks for answers. > > > > > > Does the callback get call on failure only or for success as well ? > > Also, > > > how do I do this on Kafka 0.8.0 ? Is there any plan for adding > > buffering > > > on disk for next version ? Also, when application restart, kafka > > producer > > > will have to transmit the messages (buffered messages on disk) ? > > > > > > I can not upgrade to Kafka 0.8.1.1 right now. how can I achieve this > on > > > Kafka version 0.8.0. Also, when retransmitting the messages I wanted > to > > > transfer dedicated Kafka back-up or reserved partition for topic > > (near-real > > > time topic). > > > > > > How would you handle retransmission of messages ? > > > > > > Any idea about implementation will be very helpful. > > > > > > Thanks, > > > > > > Bhavesh > > > > > > On Mon, May 5, 2014 at 9:29 PM, Jun Rao <jun...@gmail.com> wrote: > > > > > > > At LinkedIn, we only buffer messages in memory in the producer. We > try > > as > > > > much as we can to make the Kafka cluster always available (with > > > > replication). The "on failure" feature you mentioned can be > implemented > > > by > > > > providing a callback. You can see the example in the javadoc. > > > > > > > > *send > > > > < > > > > > > > > > > http://empathybox.com/kafka-javadoc/kafka/clients/producer/KafkaProducer.html#send(kafka.clients.producer.ProducerRecord > > > > , > > > > kafka.clients.producer.Callback)>*(ProducerRecord< > > > > > > > > > > http://empathybox.com/kafka-javadoc/kafka/clients/producer/ProducerRecord.html > > > > > > > > > record, Callback< > > > > > > http://empathybox.com/kafka-javadoc/kafka/clients/producer/Callback.html > > > > > > > > callback) > > > > > > > > Thanks, > > > > > > > > Jun > > > > > > > > On Mon, May 5, 2014 at 6:30 PM, Bhavesh Mistry > > > > <mistry.p.bhav...@gmail.com>wrote: > > > > > > > > > We are using Kafka for operation metrics and we do not want to > loose > > > any > > > > > data at all if there is issue with Network or all brokers needs to > be > > > > > rebooted or operation reason down time while all producers are > still > > > > > producing data on front end side. We use async to publish messages > > and > > > > we > > > > > are using Kafka version 0.8.0. > > > > > > > > > > > > > > > > > > > > Has any one implemented buffering on local disk (on producer side) > > and > > > > > transmit messages when network connection is restored? How do I > get > > > > > handle to list of messages async thread could not transfer after x > > > reties > > > > > ? I know new producer API has callback interface, but is per > message > > > not > > > > > per producer instance ? > > > > > > > > > > > > > > > > > > > > Is this final new Producer API ? > > > > > > > > > > > > > > > > > > > > http://empathybox.com/kafka-javadoc/index.html?kafka/clients/producer/KafkaProducer.html > > > > > > > > > > > > > > > is there a plan to add method like. onFailure(List<Messages> > > messages, > > > > > Exception exception ) ? > > > > > > > > > > > > > > > > > > > > > > > > > Basically, I have to address the QOS on producer side, and be able > to > > > > > buffer on disk and retransmit all message to partitions that are > > > reserved > > > > > for messages that happened in past. > > > > > > > > > > > > > > > > > > > > How does Linked-in handle QOS on producer side ? > > > > > > > > > > > > > > > Is there any plan to add this QOS feature on Producer Side with > > > > Strategies > > > > > to > > > > > store and retransmit the message ? If we do get the list of > messages > > > is > > > > > call back, will it be compressed data ? > > > > > > > > > > > > > > > > > > > > I would appreciate Kafka Developers and others feedback on how to > > > > implement > > > > > QOS. > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > Bhavesh > > > > > > > > > > > > > > >
