Cheers!
Nobody has an Idea what to do with InvalidMessageExceptions?
Jörg
Forwarded Message
Subject:MirrorMaker kafka.message.InvalidMessageException
Date: Fri, 18 Sep 2015 11:37:20 +0200
From: Jörg Wagner
Reply-To: users@kafka.apache.org
To: users@kafk
I have a topic with three partitions, to which I send 5 messages (very rapidly
after one another) and they get partitioned well ([1,2,2] as it happens).
I'm launching three identical high level java customers (kafka 8.2.1),
single-threaded, to consume those messages. Once a message is received, t
Assuming this is a test case with a new topic/consumer groups for each run,
do you have auto.offset.reset=smallest? This happens to me constantly in
tests because my consumers end up missing the first message since the
default is largest (in which case auto commit is a red herring.)
On Wed, Sep 23
Hello,
I have a question about performance testing:
Performance tests for producers can run (using a Java class) with:
bin/kafka-run-class.sh org.apache.kafka.clients.tools.ProducerPerformance
but there is also:
./bin/kafka-producer-perf-test.sh
This is calling a Scala Class called kafka.tools
Looks like you are right, although I am not sure I understand why. :)
The outlined scenario is indeed part of a test suite, which goes through the
following stages (with some temporal separation between each),
1/ Create a new topic (with some guid in the name), with three partitions
2/ Launch t
We are using kafka 0.8.2.1 and are noticing a weird behavior when the high
level consumers are restarted.
We have two consumers (C1 and C2, processes on different machines) in a
consumer group. The topic has two partitions. We use kafka as offsets
storage and have auto commit disabled.
Here is th
I've found that (especially in tests) I can get a ConsumerIterator, but
that the consumer is not actually connected and settled onto its partitions
yet. Unfortunately, I haven't found a good way to establish that aside from
sending pilot messages through first. smallest is easier :)
On Wed, Sep 23
Hi,
Just wondering when is 0.9 version of the kafka library releasing? I am
particularly interested in the KafkaConsumer pause/resume version.
Is there any other way to pause consumer without triggering a rebalancing
process in 0.8.x?
Thanks,
John
Both classes work ok. I prefer the Java one simply because has better output
and it does less overriding of default values.
However, in both cases you probably need to tweak settings to suit your use
case. Most notably:
acks
batch.size
linger.ms
based on whether you are interested in latency or
Hello Folks,
I've noticed that 2 producer machines, that I had configured, have become
very slow over time
They are giving 17-19 MB/s
But, a producer that I setup today is giving 70MB/s as the write throughput
If I see the contents of bin, libs, config directories, nothing is
different in the fi
It looks like that fix will not be included in a release until 0.9.0.0.
I'm thinking maybe it makes sense not to switch to kafka storage for
offsets until then?
Jason
On Fri, Sep 18, 2015 at 1:25 PM, Todd Palino wrote:
> I think the last major issue with log compaction (that it couldn't handle
On Sep 18, 2015, at 10:25 AM, Todd Palino wrote:
> I think the last major issue with log compaction (that it couldn't handle
> compressed messages) was committed as part of
> https://issues.apache.org/jira/browse/KAFKA-1374 in August, but I'm not
> certain what version this will end up in. It ma
Hi,
I would like to dig deep into this issue. I've changed log4j.properties for
logging in ALL mode in all places. I am getting lost in the logs.
Any pointers would be welcome
Please let me know if you would need any information regarding this
Thanks,
Prabhjot
On Wed, Sep 23, 2015 at 6:46 PM,
13 matches
Mail list logo
