I ran ConsumerPerformance with different number of messages config for a
topic that has 100s of messages in the log. It did not hang in any of these
tests -
nnarkhed-mn1:kafka-git-idea nnarkhed$ ./bin/kafka-run-class.sh
kafka.perf.ConsumerPerformance --zookeeper localhost:2181 --topic test
*--messages
10* --threads 1
start.time, end.time, fetch.size, data.consumed.in.MB, MB.sec,
data.consumed.in.nMsg, nMsg.sec
1 messages read
2 messages read
3 messages read
4 messages read
5 messages read
6 messages read
7 messages read
8 messages read
9 messages read
10 messages read
*2014-02-09 21:39:47:385, 2014-02-09 21:39:52:641, 1048576, 0.0015, 0.0060,
10, 39.0625* (test ends with this)
nnarkhed-mn1:kafka-git-idea nnarkhed$ ./bin/kafka-run-class.sh
kafka.perf.ConsumerPerformance --zookeeper localhost:2181 --topic test
*--messages
2* --threads 1
1 messages read
2 messages read
*2014-02-09 21:39:59:853, 2014-02-09 21:40:05:100, 1048576, 0.0003, 0.0011,
2, 8.0972* (test ends with this)
The way I read the --messages config is "read exactly --messages # of
messages, not less not more". If the topic has less than --messages
records, it will hang until it gets more data, else it exits properly.
Seems like expected behavior.
Thanks,
Neha
On Sat, Feb 8, 2014 at 2:11 PM, Jay Kreps <jay.kr...@gmail.com> wrote:
> It loops on
> for (messageAndMetadata <- stream if messagesRead < config.numMessages) {
> // count message
> }
>
> That means it loops over *all* messages in the stream (eventually blocking
> when all messages are exhausted) but only counts the first N as determined
> by the config.
>
> As a result this command seems to always hang forever. Seems like someone
> "improved" this without ever trying to run it :-(
>
> -Jay
>
