Also, what's the flush.interval on the broker?
Thanks,
Jun
On Mon, May 6, 2013 at 2:48 PM, Sining Ma <sinin...@aol.com> wrote:
> Hi,
> I tried some simple performance tests for kafka producer.
> I read 1000 lines of text from one file into memory and write these data
> to a file, regular socket and using kafka producer.
>
>
> The results is as follows:
> When I use BufferedOutputStream writing the data to a file, it takes 17
> milliseconds to write 1000 lines.
>
> FileOutputStream fos = new FileOutputStream(filename,
> true);
> BufferedOutputStream bufferdOutput = new
> BufferedOutputStream(fos);
>
> long start = System.currentTimeMillis();
> for (String line2: strList) {
> bufferdOutput.write(line2.getBytes());
> }
> long end = System.currentTimeMillis();
>
>
>
> When I use BufferedOutputStream writing the data to a regular socket, it
> takes 21 milliseconds to write 1000 lines.
> os = new BufferedOutputStream(new
> DataOutputStream(socket.getOutputStream()));
>
> long start = System.currentTimeMillis();
> for (String line: strList) {
> os.write(line.getBytes());
> }
> long end = System.currentTimeMillis();
>
>
>
> At last, I use kafka producer to send the data. My current configuration is
> producer.type=async
>
> max.message.size=1000000
> queue.time=30000
> queue.size=1000
> batch.size=100
>
> It takes 25 milliseconds to write 1000 lines. But if I use sync send, it
> takes 84 milliseconds.
>
>
> My question is that is there any method that I can reduce the time and
> improve the efficiency for sending messages through kafka producer.
>
>
>
> -- Regards
> Sining Ma
>
>
>
