This is the exact parameter I added to server.config file on all the nodes.
log.retention.ms=1000 Thanks, Regards, Eranga Heshan *Undergraduate* Computer Science & Engineering University of Moratuwa Mobile: +94 71 138 2686 <%2B94%2071%20552%202087> Email: era...@wso2.com <osanda.des...@gmail.com> <https://www.facebook.com/erangaheshan> <https://twitter.com/erangaheshan> <https://www.linkedin.com/in/erangaheshan> On Wed, Nov 23, 2016 at 9:43 AM, Eranga Heshan <eranga....@gmail.com> wrote: > I have 4 node cluster and node1 has zookeeper running. I added those > parameters on each of my server.properties files in kafka nodes. > > > Eranga Heshan > *Undergraduate* > Computer Science & Engineering > University of Moratuwa > Mobile: +94 71 138 2686 <%2B94%2071%20552%202087> > Email: era...@wso2.com <osanda.des...@gmail.com> > <https://www.facebook.com/erangaheshan> > <https://twitter.com/erangaheshan> > <https://www.linkedin.com/in/erangaheshan> > > On Wed, Nov 23, 2016 at 9:36 AM, Sachin Mittal <sjmit...@gmail.com> wrote: > >> Kafka logs are not kept based on data consumption. They are deleted after >> it reaches a particular size and retention time. >> Even if they are consumed multiple times in that duration, they are still >> kept. >> >> Where have you added those params and how are you starting your zookeeper >> and server. >> >> >> On Wed, Nov 23, 2016 at 8:00 AM, Eranga Heshan <eranga....@gmail.com> >> wrote: >> >> > Sorry for repeatedly asking the same question, but although I tried all >> > those parameters; my log file still at large about 2.5GB. Is it due to >> the >> > data inside my topic is flushed to disk? I meant that there might be >> data >> > inside topic which is not consumed yet or not been deleted even after >> > consumed. >> > >> > Thanks, >> > Regards, >> > >> > >> > Eranga Heshan >> > *Undergraduate* >> > Computer Science & Engineering >> > University of Moratuwa >> > Mobile: +94 71 138 2686 <%2B94%2071%20552%202087> >> > Email: era...@wso2.com <osanda.des...@gmail.com> >> > <https://www.facebook.com/erangaheshan> <https://twitter.com/ >> > erangaheshan> >> > <https://www.linkedin.com/in/erangaheshan> >> > >> > On Tue, Nov 22, 2016 at 2:15 PM, Sachin Mittal <sjmit...@gmail.com> >> wrote: >> > >> > > Set some value to log.retention.ms which is ms time to retain log and >> > also >> > > log.segment.bytes which is size of a single log file. >> > > delete.topic.enable is needed if you want to delete the topic itself. >> > > >> > > Thanks >> > > Sachin >> > > >> > > >> > > On Tue, Nov 22, 2016 at 1:52 PM, Eranga Heshan <eranga....@gmail.com> >> > > wrote: >> > > >> > > > I tried log.retention.bytes=1049000 which is about 10MB. But it did >> not >> > > > work. Is it because I had not given any value to >> delete.topic.enable? >> > > > >> > > > >> > > > Eranga Heshan >> > > > *Undergraduate* >> > > > Computer Science & Engineering >> > > > University of Moratuwa >> > > > Mobile: +94 71 138 2686 <%2B94%2071%20552%202087> >> > > > Email: era...@wso2.com <osanda.des...@gmail.com> >> > > > <https://www.facebook.com/erangaheshan> <https://twitter.com/ >> > > > erangaheshan> >> > > > <https://www.linkedin.com/in/erangaheshan> >> > > > >> > > > On Tue, Nov 22, 2016 at 1:47 PM, Sachin Mittal <sjmit...@gmail.com> >> > > wrote: >> > > > >> > > > > Check >> > > > > http://kafka.apache.org/documentation.html#brokerconfigs >> > > > > log.retention.bytes >> > > > > log.retention.ms >> > > > > log.segment.bytes >> > > > > delete.topic.enable >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > On Tue, Nov 22, 2016 at 1:36 PM, Eranga Heshan < >> eranga....@gmail.com >> > > >> > > > > wrote: >> > > > > >> > > > > > Hi all, >> > > > > > >> > > > > > I want to keep the size of kafka-logs reduced as much as >> possible. >> > To >> > > > do >> > > > > it >> > > > > > I need to set it to delete its unwanted logs frequently. Is >> there a >> > > way >> > > > > to >> > > > > > do it? >> > > > > > >> > > > > > Thanks, >> > > > > > Regards, >> > > > > > >> > > > > > Eranga Heshan >> > > > > > *Undergraduate* >> > > > > > Computer Science & Engineering >> > > > > > University of Moratuwa >> > > > > > Mobile: +94 71 138 2686 <%2B94%2071%20552%202087> >> > > > > > Email: era...@wso2.com <osanda.des...@gmail.com> >> > > > > > <https://www.facebook.com/erangaheshan> <https://twitter.com/ >> > > > > > erangaheshan> >> > > > > > <https://www.linkedin.com/in/erangaheshan> >> > > > > > >> > > > > >> > > > >> > > >> > >> > >
