On 21/07/14 04:11, Paul Wise wrote: > Unfortunately the final word on that is that Debian needs to replace > our current redirector with one based on the RSS feature and also to > add a cache mechanism (they suggested a 1 hour cache time) so that we > don't overload the RSS feature. > > Do you think it would be possible for you to add a caching mechanism > and convert your version into a patch that we can apply to SVN? >
It should definitely be possible to add a caching mechanism to the the
new redirector, currently I have a couple of ideas on this but both have
drawbacks.
1. Use a Berkeley DB to store the retrieved data, similar to what is
currently done.
Something like:
| project | file | update_time |
--------------------------------
| | | |
Problems I foresee:
* My intention would be to check at the time the script is requested if
the update_time > 1 hour ago...
- if yes... get the new RSS and update the DB
- if no.... use the information from the DB
What happens if this happens for multiple requests at the same time?
2. Save the XML file to a cache folder
Then at request time check the time on that file and it's age.
The only problem I can see this causing is disk space (I don't know how
much of an issue this is for Debian)
The RSS file for the VPCS project is almost 52KB. Picking a figure out
of the air (as I've no idea how packages use the redirector) of 10000,
this is going to create 520MB of cached files.
Obviously some projects may have a smaller RSS and others larger....
which may skew that estimate.
Any comments on these would be most appreciated.
Cheers,
Daniel
signature.asc
Description: OpenPGP digital signature
