I didn't get it ... Can you explain me in form of example or something for which you are feasible
On Oct 17, 2016 2:08 PM, "Ben Davison" <ben.davi...@7digital.com> wrote: > We have it setup so that both log ms is set to 7 days and log delete > bytes(can't remember exactly what the setting is called. So we never run > out of space (don't set the value to something like 99% of your disk, as > the log cleaner thread might not kick in time, we leave it at 90% of disks > space) > > On Monday, 17 October 2016, Kunal Gupta <kunal.gu...@cube26.com> wrote: > > > Please help me :( > > > > *Thanks, Kunal* > > *+91-9958189589* > > *Data Analyst* > > *First Paper Publication : **http://dl.acm.org/citation.cfm?id=2790798 > > <http://dl.acm.org/citation.cfm?id=2790798> * > > *Blog:- **http://learnhardwithkunalgupta.blogspot.in > > <http://learnhardwithkunalgupta.blogspot.in> * > > > > > > > > > > > > On Sun, Oct 16, 2016 at 11:23 AM, Kunal Gupta <kunal.gu...@cube26.com > > <javascript:;>> > > wrote: > > > > > > > > In my organisation I have 3 machine cluster of Kafka and each topic > > > assigned two machine for storing there data. > > > > > > There is one topic for which I get lot of data from clients thats data > > > exceeds my disk space in one machine because that machine is a leader > of > > > that topic, when I look into kafka-logs seeing all topic data there is > > only > > > one topic whose data has consumed lot of space. > > > > > > In one month I have 36 GB data from that topic. What I have thought so > > far > > > is kafka logs will be delete after 24 hours and new data will be > retained > > > for 24 hours and my consumer will consume all data with in 24 hours, so > > > there will be no problem of disk space. > > > > > > I have enable everything in server.properties to clean kafka-logs but > > > still data is there. Below data is my settings in server.properties > file. > > > > > > *log.retention.minutes=1440* > > > *log.retention.hours=24* > > > *log.retention.ms <http://log.retention.ms>=86400000* > > > *log.cleaner.delete.retention.ms > > > <http://log.cleaner.delete.retention.ms>=24* > > > *log.segment.bytes=1048576* > > > *log.retention.check.interval.ms > > > <http://log.retention.check.interval.ms>=3000* > > > *log.cleaner.enable=true* > > > *zookeeper.connection.timeout.ms > > > <http://zookeeper.connection.timeout.ms>=30000* > > > *delete.topic.enable = true* > > > *auto.create.topics.enable = true* > > > *default.replication.factor=2* > > > *auto.leader.rebalance.enable=true* > > > *controlled.shutdown.enable=true* > > > *controller.socket.timeout.ms <http://controller.socket.timeout.ms > > >=120000* > > > > > > Please help me in this so that my machine can able to handle large > number > > > of request and data well using 3 machine cluster. > > > > > > *Thanks, Kunal* > > > *+91-9958189589* > > > *Data Analyst* > > > *First Paper Publication : **http://dl.acm.org/citation.cfm?id=2790798 > > > <http://dl.acm.org/citation.cfm?id=2790798> * > > > *Blog:- **http://learnhardwithkunalgupta.blogspot.in > > > <http://learnhardwithkunalgupta.blogspot.in> * > > > > > > > -- > > > This email, including attachments, is private and confidential. If you have > received this email in error please notify the sender and delete it from > your system. Emails are not secure and may contain viruses. No liability > can be accepted for viruses that might be transferred by this email or any > attachment. Any unauthorised copying of this message or unauthorised > distribution and publication of the information contained herein are > prohibited. > > 7digital Limited. Registered office: 69 Wilson Street, London EC2A 2BB. > Registered in England and Wales. Registered No. 04843573. >
