Glenn Morris wrote: > Bob Proulx wrote: > > > /var/cache/cgit directory. > [...] > > Some of the files are really quite large. > > > > -rw------- 1 www-data www-data 513M Mar 9 02:40 f0110000 > > Any idea what causes these enormous files to be produced?
Links to commit diffs. And some of those commits can be very large. That one I looked at it was a tremendously large commit diff. In an html display format. All of those files are html getting thrown out to the web browsers. That one was a 512 meg html file for some client web query. Probably not a useful thing for a web browser and so I imagine the user didn't wait for it. That is a huge display page. No information on whether it was a web browser or a robot crawler but hopefully a person because have the robots.txt file set to avoid crawling the /cgit/ directory. I forget now which project that was for. It think it was for emacs. I already deleted the file. > I see cgit has an option: > > max-blob-size > Specifies the maximum size of a blob to display HTML for in > KBytes. Default value: "0" (limit disabled). > > Don't know if it's relevant. That could be useful. There are various tunings available in the /etc/cgitrc file. Assaf has already done some tuning. #cache-size=10000 # 2017-feb-20,agn: # Reduce cache size to 5000, # see http://lists.gnu.org/archive/html/savannah-hackers-public/2017-02/msg00106.html cache-size=5000 # 2017-feb-20,agn: # Set TTL even for static pages (those with fixed SHA1 in the URL), # Some of savannah's pages generate cache files of ~500MB. cache-static-ttl=5 Adding max-blob-size probably makes sense too. Because I just don't think it makes sense to view a 513M web page. However I decided that instead of looking at cgitrc options that it was just a lot simpler and more direct to use find and the hourly cron to prune that directory. Even with that agressive pruner I put in there are still a lot of files in that directory. They are now mostly small files however. root@vcs0:~# ls /var/cache/cgit | wc -l 4081 root@vcs0:~# du -sh /var/cache/cgit 314M /var/cache/cgit Bob
