Adding more context, with the 0.8.2.1 consumers to find the number of
pending messages, we perform an OffsetFetchRequest to get the last
committed offset and an OffsetRequest to get the last offset in the log. I
see that the new KafkaConsumer does a good job at replacing the
OffsetFetchRequest (wi
Hi Jeff,
The java doc is very nice, thank you and thanks to whoever wrote it.
I do have one question about the API. For what we're doing, it's important
for us to calculate the "lag" or pending message count. Today we do that
using the simple consumer to ask kafka for the committed offset (beca
Hi Jeff,
Thanks for your response.
On scala side, is there a Producer implementation that I could use? is the
java based KafkaProducer (org.apache.kafka.clients.producer.KafkaProducer;)
same as Producer in Producer.scala ?
Thanks,
Prabhjot
On Thu, Nov 5, 2015 at 11:28 PM, Jeff Holoman wrote:
>
The best thing that I know is the latest javadoc that's committed to trunk:
https://github.com/apache/kafka/blob/ef5d168cc8f10ad4f0efe9df4cbe849a4b35496e/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java
Thanks
Jeff
On Thu, Nov 5, 2015 at 12:51 PM, Cliff Rhyne wrote:
Hi Jeff,
Is there a writeup of how to use the new consumer API (either in general or
for Java)? I've seen various proposals but I don't see a recent one on the
actual implementation. My team wants to start the development work to
migrate to 0.9.
Thanks,
Cliff
On Thu, Nov 5, 2015 at 11:18 AM, J
Prabhjot,
The answer changes slightly for the Producer and Consumer and depends on
your timeline and comfort with using new APIs
Today and in the future, for the Producer, you should be using the "new"
producer, which isn't all that new anymore:
org.apache.kafka.clients.producer.KafkaProducer;
Hello Folks,
Requesting your expertise on this.
I see that under core/src/main/scala/kafka/producer/, there are many
implementations - Producer.scala and SyncProducer.scala
Also, going via the producerPerformance.scala, there are 2 implementations
- NewShinyProducer (which points to KafkaProducer
Adding users as well
On Thu, Nov 5, 2015 at 3:37 PM, Prabhjot Bharaj
wrote:
> Hi,
>
> I'm using the latest update: 0.8.2.2
> I would like to use the latest producer and consumer apis
> over the past few weeks, I have tried to do some performance benchmarking
> using the producer and consumer scr
Hi,
I'm using the latest update: 0.8.2.2
I would like to use the latest producer and consumer apis
over the past few weeks, I have tried to do some performance benchmarking
using the producer and consumer scripts provided in the bin directory. It
was a fun activity and I have learnt a lot about ka
