On Tue, Jun 5, 2012 at 11:25 AM, Greg Stein <gst...@gmail.com> wrote: > On Tue, Jun 5, 2012 at 10:49 AM, Mark Phippard <markp...@gmail.com> wrote: >>... >> The net effect is just the impact that a seemingly small HTTP request >> can have on a highly trafficked site where those requests start to add >> up. The main problem here is that the Eclipse update process looks >> for a small index file that is currently always a HTTP 404. This > > Heh. Reminds me of the traffic spike that occurred when a Google Code > project used one of its version control for a configuration file. It > was a proxy add-on for Firefox to navigate the Chinese firewall. > Suddenly, every user who had the add-on installed was pinging > code.google.com for this one file, retrieved via a Subversion URL. > Daniel Berlin moved fast and introduced a cache that dropped our > server load. > >> results in 1.5GB of network packets per day which basically amounts to >> 152 minutes per day of their available bandwidth. The 404 itself is >> not the problem, the bandwidth usage would be worse if the file >> existed. The issue is more the design of the protocol and the >> discovery it does each time it runs. This is probably similar to all >> of the HEAD and PROPFIND requests we do in our protocol. They are >> seeing similar bandwidth issues resulting from the usage of HEAD in >> their protocol. It looks like the webmaster would like them to >> explore doing more with a single request. Someone that understands >> how our REPORT requests work might want to throw some commentary into: > > See above: cache. > > Not "lump things into one response". > > A reverse proxy (or several) in front of the primary (Subversion) > server can 404 that request before introducing any load on the origin > server.
Keep in mind that this is not about server load, they are looking at this from bandwidth. So a cache in front of the server would not help them at all. Eclipse.org has a 10MB Internet link that is almost always saturated. Also, if it was not clear, Subversion is not involved here. This is a plain Apache server. -- Thanks Mark Phippard http://markphip.blogspot.com/
