Ludovic Courtès wrote: > This particular file takes 7 MiB and has a large history (it is updated > daily), which perhaps makes things worse.
Oh! I apologize that I didn't fully investigate this with the previous exchange. Because vcs often gets overloaded I assumed that was the problem again. Because when I looked it was all very happy at that time. That was bad of me. Sorry. But reading this I now realize that it is a somewhat different problem. vcs:/srv/cvs/web/guix/guix# du -sh 1.9G . Wow! 1.9G of web page! That is a LOT. Most of it in one file. vcs:/srv/cvs/web/guix/guix/packages# ll total 1267580 -r--r--r--+ 1 civodul guix 1243780406 Apr 19 11:07 index.html,v -r--r--r--+ 1 civodul guix 54200311 Apr 14 01:29 issues.html,v Oh my goodness. That is a very large rcs file! Looking in the file I see that the main body of the web page is all on *one* line. There are 29 lines in total. Wow. If it were newline'd normally that would probably have made a huge reduction in the diff sizes. wget -O-.index.html https://www.gnu.org/software/guix/packages/ | wc index.html 29 427729 7002030 There are 293 versions checked in. Which means that pretty much every version is a complete copy of the file since line based diffs are saved and there is only one line. That isn't going to be sustainable. The actual root error is this one. vcs:/tmp/junk# cvs -d /srv/cvs/web/guix co guix/packages cvs checkout: Updating guix/packages cvs [checkout aborted]: Could not map memory to RCS archive /srv/cvs/web/guix/guix/packages/index.html,v: Cannot allocate memory vcs has 6 gig of virtual memory available. This must be exceeding a different limit. But this is why the web page isn't updating. It is hard stuck and isn't going to make any more progress until something changes. > Also, I’m under the impression that web page updates are queued. For > instance the timestamp at the bottom of the page above says “April 14”, > whereas the commit from now succeeded (and so it should read “April > 19”.) Normally they are updated reasonably quickly. But the above problem has this stuck and is why it isn't updating at the moment. It can't be checked out. Normally there is a convoluted Rube Goldberg machine that hits a URL on the FSF machine which touches a file which then runs a script and sees the flag files telling it to do a cvs update right now on the page. That usually flows through in the amount of time it takes the machine to operate and check out the file. (This was broken about a month ago and fixed and therefore I believe it is operating now.) So normally there is no queue as such but not quite immediately. There are propagation delays in the mirroring. > Regardless, if GNU packages and other projects hosted on Savannah can no > longer update their web pages, then it’s the beginning of the end. We > should make sure the FSF shares this conclusion and that the issue will > be addressed in a timely fashion, IMO. Obviously my first feedback was wrong. Sorry. And I know that led to the above which is cascade failure on the top of my misdirection. So I will ignore that. But I wanted to say that Assaf Gordon has spent a significant amount of time trying to improve the web page system on the savannah.gnu.org side of things. Primarily to enable use of git for web page updates. But there are limits to what we can do on our (savannah) side of things. Presently his efforts are waiting for help on the fsf.org side of things. The improvements are stuck until more resources are available to help there. > Thanks for your feedback! My feedback is that something different needs to be done for this index.html page. I don't want to suggest something without thinking about it. I might suggest something worse! But the first thought I had was that this needs some kind of dedicated status page. This shouldn't really be checked into cvs. This looks to be some type of real time dynamically created status page that shows the information you want. I would put this on a different location entirely. Perhaps a dedicated page? That seems like the way it should go. If you want to have it displayed right where it is displayed then an <iframe> can be used to place it there. How is the data on that page generated? Bob
