Re: Kafka topic partition directory

2020-03-28 Thread Peter Bukowinski
Kafka doesn’t monitor the contents of the log data directories unless it created the file or directory. If it didn’t create the directory/file it will ignore it. -- Peter > On Mar 28, 2020, at 4:17 PM, anila devi > wrote: > > Hi Users, > If I create a directory or a file in the same direct

Kafka topic partition directory

2020-03-28 Thread anila devi
Hi Users,  If I create a directory or a file in the same directory where kafka creates partition topic, the kafka broker node does not restart. Is it expected ?  Thanks,Dhiman

Re: Newbie Question

2020-03-28 Thread Colin Ross
Thanks Hans - this makes sense, except for the debug messages give me exactly what I need without having to instrument any clients. It should be noted that for now, I am running a single server, so perhaps the messages change when I cluster? I maybe caused confusion by mentioning that I want to kno

Re: Newbie Question

2020-03-28 Thread Hans Jespersen
I can tell from the terminology you use that you are familiar with traditional message queue products. Kafka is very different. Thats what makes it so interesting and revolutionary in my opinion. Clients do not connect to topics because kafka is a distributed and clustered system where topics a

Newbie Question

2020-03-28 Thread Colin Ross
Hi All - just started to use Kafka. Just one thing driving me nuts. I want to get logs of each time a publisher or subscriber connects. I am trying to just get the IP that they connected from and the topic to which they connected. I have managed to do this through enabling debug in the kafka-author

Re: Kafka with RAID 5 on. busy cluster.

2020-03-28 Thread Hans Jespersen
RAID 5 typically is slower because Kafka is very write heavy load and that creates a bottleneck because writes to any disk require parity writes on the other disks. -hans > On Mar 28, 2020, at 2:55 PM, Vishal Santoshi > wrote: > > Ny one ? We doing a series of tests to be confident, but if

Re: Kafka with RAID 5 on. busy cluster.

2020-03-28 Thread Vishal Santoshi
Ny one ? We doing a series of tests to be confident, but if there is some data folks, who have had RAID 5 on kafka, have to share, please do. Regards. On Mon, Mar 23, 2020 at 11:29 PM Vishal Santoshi wrote: > << In RAID 5 one can loose more than only one disk RAID here will be data > corrupti