maybe you want to spend a few minutes reading about Haystack over at facebook to give you some ideas...
https://www.facebook.com/note.php?note_id=76191543919 Not saying what they've done is the right way... just sayin' On Thu, Jun 23, 2011 at 6:29 AM, AJ <a...@dude.podzone.net> wrote: > > I was thinking of doing the same thing. But, to compensate for the > bandwidth usage during the read, I was hoping to find a way for the httpd or > app server to cache the file either in RAM or on disk so subsequent reads > could just reference the in-mem cache or local hdd. I have big data > requirements, so duplicating the storage of file blobs by adding them to the > hdd would almost double my storage requirements. So, the hdd cache would > have to be limited with the LRU removed periodically. > > I was thinking about making the key for each file be a relative file path as > if it were on disk. This same path could also be used as it's actual > location on disk in the local disk cache. Using a path as the key makes it > flexible in many ways if I ever change my mind and want to store all files > on disk, or when backing-up or archiving, etc.. > > But, I'm rusty on my apache http knowledge but I also thought there was an > apache cache mod that would use both ram and disk depending on the frequency > of use. But, I don't know if you can tell it to "cache this blob like it's > a file". > > Just some thoughts.
