Hi Alejandro and Russell, On 12/28/2014 07:29 PM, Russell Keith-Magee wrote: > On Sun, Dec 28, 2014 at 8:13 PM, Alejandro Dubrovsky <[email protected]> > wrote: >> On Sunday, December 28, 2014 7:23:34 PM UTC+11, Alejandro Dubrovsky wrote: >>> On Sunday, December 28, 2014 1:03:26 PM UTC+11, Russell Keith-Magee wrote: >>>> On Sat, Dec 27, 2014 at 9:30 PM, Alejandro Dubrovsky <[email protected]> >>>> wrote: >>>>> >>>>> When running a small, low traffic site on slow cloud servers (eg >>>>> DigitalOcean), it'd be useful to be able to cache pages, not because the >>>>> site might be taken down from too many request, but to improve >>>>> responsiveness. A non-trivial page that might take a few hundredth of a >>>>> second to server on a local desktop might take over a second on a slow >>>>> server, but that times goes back down to less than a hundredth of a second >>>>> when cached. Ideally, I'd want to pre-generate most of the popular pages >>>>> on >>>>> a site but not send a large max-age, so that if the content does change, >>>>> the new pages can be regenerated and stale content would not be served. >>>>> >>>>> At the moment, the Django cache framework ties up how long it thinks >>>>> the cached values are valid with what it sends the client with max-age (eg >>>>> if you set CACHE_MIDDLEWARE_SECONDS to 600, it'll send a max-age of 600, >>>>> and set the validity of the server-side content valid for 600 seconds). It >>>>> ignores the CACHES[cache]['TIMEOUT'] and goes by CACHE_MIDDLEWARE_SECONDS, >>>>> or its per-view parameter equivalent. I think it would be useful to be >>>>> able >>>>> to separate the two, keeping server-stored caches valid for a long time >>>>> (or >>>>> indefinitely until they are forcefully invalidated), while sending a small >>>>> max-age (or none at all). A CACHE_MIDDLEWARE_CLIENT_SECONDS setting >>>>> would seem appropriate, or maybe a setting to tell it to respect >>>>> CACHES[name]['TIMEOUT'] for server-side validity checking. >>>>> >>>>> I can prepare a patch if the idea seems worthwhile to others. >>>>> >>>> >>>> Hi Alejandro, >>>> >>>> I can see how the case you describe would lead to computed cache content >>>> having long validity, but that validity not necessarily being the same as >>>> the appropriate max-age for the client. There are some related cases that >>>> would be good to control - for example, if you receive a request near the >>>> end of the lifespan of a cached value, you'll effectively double the >>>> lifespan of the cached value in the wild, because you'll return a long >>>> max-age to the client just before the value actually expires. >>>> >>>> So: yes, I think a patch for this feature would be worthwhile. >>>> >>>> From an implementation point of view, the addition of a new setting is >>>> the only point of concern - we generally try to avoid adding a new setting >>>> unless we have to. However, in this case, there really is a new thing being >>>> configured here, so a new setting makes sense to me. The only caveat I >>>> would flag is the backwards compatibility path; existing users won't have >>>> CACHE_MIDDLEWARE_CLIENT_SECONDS defined, so the implementation will >>>> need to fall back to CACHE_MIDDLEWARE_CLIENT_SECONDS = >>>> CACHE_MIDDLEWARE_SECONDS if it the new setting isn't provided. >>>> >>>> Yours, >>>> Russ Magee %-) >>>> >>>> >>> Hi Russell, >>> >>> Good to hear that it seems like a reasonable idea. >>> >>> One more design decision that is needed is how strictly we want to keep >>> the current behaviour in case of no changes to the settings file. I am >>> assuming that the answer is 'very'. >>> >> > Correct. > > >> There is an issue in that the current code looks in the response from a >>> view for a 'max-age' attribute, and if it finds that value set, that value >>> becomes the cache's lifespan (both on the client and server side). My plan >>> is to keep that behaviour _unless_ a value for >>> CACHE_MIDDLEWARE_CLIENT_SECONDS >>> is explicitly set in the settings (implying that the user seems aware of >>> the new feature), in which case a preset 'max-age' value in the response >>> will only override the client-side time of validity (ie it will not affect >>> how long the server deems the server-side cache valid). >>> >>> Do tell if that behaviour seems too magicky, or if some other behaviour >>> would be preferable. >>> >>> > This sounds right to me, and not at all magicky - it's "use one value (the > old setting) for both settings unless both are provided". This retains the > old behavior, which makes *some* sense; but allows opt in to the > fine-grained behavior. > > >> >>> While updating the documentation, I noticed that it states that the >> cache_control and the never_cache decorators can be used to control the >> caching of a page. The documentation doesn't differentiate between >> server-side and client-side caching so people could easily read that as >> saying that no server-side caching would occur with a never_cache >> decorator. As far as I can tell, this means that the current behaviour in >> which the max-age setting overrides all other settings for both server- and >> client-side caching lifespan will have to stay, regardless of the existence >> of the CACHE_MIDDLEWARE_CLIENT_SECONDS setting. >> > > If someone has gone to the trouble of explicitly setting a max-age, I think > it makes sense that the explicitly specified value would override any > defaults for client side. However, I don't think that necessarily makes > sense for server side - the connection between max-age and server-side > cache isn't as obvious. There's behavior that we need to retain for > backwards compatibility reasons, but that follows the same "use one setting > to control two values" behavior - it's just a setting that is explicitly > client-side this time. > > Ideally, it would be possible to set both the server cache timeout *and* > the client cache timeout explicitly. This *could* be done via a keyword > argument on the cache-control decorator (e.g., `server_age`); that argument > would be a "hint" to any server-side caching infrastructure indicating how > long a server-side cache should be retained, and would override any > CACHE_MIDDLEWARE_SECONDS setting. > > So, the complete behavior would be something like: > > For server-side cache age: > * If there's a server_age argument, use it > * If there's no server_age, use max-age > * If there's no max-age, use CACHE_MIDDLEWARE_SECONDS > > For client-side cache age: > * If there's a max_age argument, use it > * If there's no max-age, use CACHE_CLIENT_MIDDLEWARE_SECONDS > * If there's no CACHE_CLIENT_MIDDLEWARE_SECONDS, use > CACHE_MIDDLEWARE_SECONDS > > Does that make sense?
I'm a bit leery of this. Currently CacheMiddleware's behavior is quite simple and predictable -- it acts just like a normal downstream HTTP cache. I recognize that it is different because it is actually server-side, and thus in theory can be manually invalidated as needed, making it safe to use longer cache expiry times than can be safely sent to the client. Though I note that there's no built-in feature to allow manual invalidation of cached full-page responses stored by UpdateCacheMiddleware (apart from in an extremely blunt way using VERSION to invalidate the entire cache at once) so treating the cache middleware as manually-invalidatable already requires some messing with internals. If you want to have finer-grained control over caching than HTTP caching provides, the current answer is to use the lower-level caching API instead of the cache middleware. I can see that there is a use case for something like the cache middleware that does automatic full-response caching, but decoupled from client-side HTTP caching headers and with provisions for manual invalidation. I'm not sure how common this use case is in practice. I think a proposal for API for this needs to include both the cache-expiry side and the invalidation side (because there is no use case for the former without the latter), and it strikes me as a good case for proving the API (and the extent of need) in a third-party package first, since I don't think there's anything here that couldn't be fairly easily implemented in a subclass of UpdateCacheMiddleware, without changes to core Django. Carl -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/54A0C3F0.7020904%40oddbird.net. For more options, visit https://groups.google.com/d/optout.
signature.asc
Description: OpenPGP digital signature
