On Wed, Aug 20, 2014 at 4:27 AM, Subodh Nijsure <subodh.nijs...@gmail.com> wrote:
> Hello, > > I have application that servers time-series data for 10+ units. This > data can get big and I don't want to retrieve 5-6MB worth of data > from cassandra and serve it out as JSON to the client. > > I want to maintain cache of this time series data. I had horrible > thought of just storing this data in local python dictionary instead > of doing something exotic like memcach server. > > What are the plus/minus of just storing this cache in process running > django? Of course this doesn't scale well for 1000+ units.. > This approach will probably work, but it's a bad idea. Firstly, if you store it in-process, the process is the web server. So - your web server will have a huge memory footprint. If your web server has multiple processes - which would be a common configuration for handling load - then the memory requirements will be duplicated for every process. Secondly, web processes aren't long lived. Depending on your web server configuration, each web server process will serve a certain number of requests, and then restart itself. This means that every N requests, the process will need to restart, and reload the in-process memory cache. This will obviously take time, and that's time the end-user is going to experience as a delay in loading a page. So - you'll be much better served using memcache (or similar) - this keeps the "data I need to keep readily available in memory" separate from "data needed to keep the web server running". The memcache image is shared across processes - even across machines if necessary - and only requires an initial warming (which can be done out of the request-response cycle, if necessary). Yours, Russ Magee %-) -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAJxq848G8MMfCtR5rWFbj%2Bz2rH7-5KZBJRq5efkPz7EhZNe6mg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
