Re: Delete topic and logs

2016-02-20 Thread Feroze Daud
unsubscribe On Saturday, February 20, 2016 9:22 AM, Neelesh wrote: I searched for this and could not get a definitive answer- when do logs get deleted , after a topic is deleted? What happens if I delete a topic and recreate it immediately? So far I know that deleting a topic marks it f

Delete topic and logs

2016-02-20 Thread Neelesh
I searched for this and could not get a definitive answer- when do logs get deleted , after a topic is deleted? What happens if I delete a topic and recreate it immediately? So far I know that deleting a topic marks it for deleted in zk, and some sweeper eventually deletes the topic meta and logs.

Re: Decreasing number of partitions INCREASED write throughput--thoughts?

2016-02-20 Thread Gerard Klijs
Interesting, Do you have any way to look into the amount and size of the actual traffic being send. I assume because there are more partitions the actual sending of the messages is separated into more messages, leading to more overhead. On Sat, Feb 20, 2016 at 12:57 PM John Yost wrote: > Hi Eve

Decreasing number of partitions INCREASED write throughput--thoughts?

2016-02-20 Thread John Yost
Hi Everyone, I discovered yesterday completely by accident. When I went from writing to 10 topics each with 10 partitions to 10 topics each with 2 partitions--all other config params are the same--my write throughput almost doubled! I was not expecting this as I've always thought--and, admittedly

Re: Does kafka.common.QueueFullException indicate back pressure in Kafka?

2016-02-20 Thread John Yost
Hi Alex, Excellent information, thanks! I very much appreciate your time. BTW, Kafka is an EXCELLENT product. It seems like my situation may be a bit of an edge case, based upon your response. Specifically, when I added more producers (in the case of Storm, a Kafka producer is a KafkaBolt), that