Some folks came up with a cool hack in 0.8 that makes acks=0 send no
response. This changes the performance for small message sends to be
equivalent to 0.7. This is proposed for inclusion in 0.8. It would
obviously be less useful for the java/scala client in 0.9 if we are able to
properly pipeline
This is a good question.
As mentioned we have some experience running this with no ack and there are
a lot of downsides. We considered making the ack optional, but this would
complicate the producer api since we could give back the offset only in the
case where there is an ack.
Thinking about it
Neha,
I see, so that is a fairly substantial change, ofcourse it has its
advantage of guaranteeing a higher degree of durability but as a
significant cost (round trip that the consumer has to wait for). I know
someone mentioned creating a asych. consumer with a future.
Do you have a 'gut' feelin
>> producer.num.acks=0
There is still a difference between the 0.7 and 0.8 Kafka behavior in the
sense that in 0.7, the producer fired away requests at the broker without
waiting for an ack. In 0.8, even with num.acks=0, the producer writes are
going to be synchronous and it won't be able to send
I see ok, so if you wanted to compare .7 with .8 on the same footing, then
you would set it to 0 right? (since 0.7 is fire and forget)
producer.num.acks=0
On Thu, Jan 17, 2013 at 11:45 PM, Jun Rao wrote:
> I means wait for the data reaches all replicas (that are in sync).
>
> Thanks,
>
> Jun
>
I means wait for the data reaches all replicas (that are in sync).
Thanks,
Jun
On Thu, Jan 17, 2013 at 6:42 PM, S Ahmed wrote:
> producer.num.acks=-1 means what sorry? is it that all replica's are written
> too?
>
>
> On Thu, Jan 17, 2013 at 12:09 PM, Neha Narkhede >wrote:
>
> > Looks like Ju
producer.num.acks=-1 means what sorry? is it that all replica's are written
too?
On Thu, Jan 17, 2013 at 12:09 PM, Neha Narkhede wrote:
> Looks like Jun's email didn't format the output properly. I've published
> some preliminary producer throughput performance numbers on our performance
> wiki
Looks like Jun's email didn't format the output properly. I've published
some preliminary producer throughput performance numbers on our performance
wiki -
https://cwiki.apache.org/confluence/display/KAFKA/Performance+testing#Performancetesting-Producerthroughput
These tests measure producer throu
We also did some perf test on 0.8 using the following command. All configs
on the broker are the defaults.
bin/kafka-run-class.sh kafka.perf.ProducerPerformance --broker-list
localhost:9092 --initial-message-id 0 --messages 200 --topics topic_001
--request-num-acks -1 --batch-size 100 --threads
Hi,
I do producer(Kafka 0.8) throughput test many times. But the average
value is 3MB/S. Below is my test environment:
CPU core :16
Vendor_id ：GenuineIntel
Cpu family ：6
Cpu MHz :2899.999
Cache size:20480 KB
Cpu level :13
We haven't done a ton of performance work on 0.8 yet.
Regardless, requiring the ack will certainly reduce per-producer
throughput, but it is too early to say by how much. Obviously this won't
impact broker throughput (so if you have many producers you may not notice).
The plan to fix this is just
In 0.8, ack is always required. The ack returns an errorcode that indicates
the reason if a produce request fails (e.g., the request is sent to a
broker that's not a leader). It also returns the offset of the produced
messages. However, the producer can choose when to receive the acks (e.g.,
when
What's the ack for? If it fails, it will try another broker? Can this be
disabled or it's a major design change?
On Wed, Jan 9, 2013 at 12:40 AM, Jun Rao wrote:
> The 50MB/s number is for 0.7. We haven't carefully measured the performance
> in 0.8 yet. We do expect the throughput that a singl
The 50MB/s number is for 0.7. We haven't carefully measured the performance
in 0.8 yet. We do expect the throughput that a single producer can drive in
0.8 to be less. This is because the 0.8 producer needs to wait for an RPC
response from the broker while in 0.7, there is no ack for the producer.
According to Kafka official document, the producer throughput is about 50MB/S.
But I do some test, the producer throughout is only about 2MB/S. The test
environment is the same with document says. One producer, One broker, One
Zookeeper are in independent machine. Message size is 100 bytes, batc
15 matches
Mail list logo
