Hello, my apologies if this is a bit long. We are considering using ATS as our future video cache. There are many advantages to ATS but our main concerns are regarding the cache storage architecture. The video cache use case is very different than the ATS sweet-spot.
*Writing to cache* We need to be able to write huge files and be able to receive them in chunks (ranges) as they are received over separated requests. When writing a chunk of a file we pre-allocate sequential space for the next chunks. Small chunks are buffered and written to disk only when enough is aggregated across separate requests. This also inherently solves the http ranges problem as space is always reserved for ranges which were not yet acquired. *Reading from cache* When reading an object we use read-ahead into memory to optimize on IOPS as in most cases the next chunks of the objects will be requested within a few seconds. *Evacuation* Evacuation should be done based on popularity (for example LRU), and works for the whole object together, not per chunk. The cyclone-buffer is not ideal for such a use case as the evacuation policy is different and we can't pay for the the copy of large objects every time the write cursor pass them. Since objects are very large and writing out is done in very large chunks, file system overhead is negligible and therefore an architecture relying on Linux file system may have benefits, such as the ability to read objects from FS rather than via HTTP for debug and analysis. It is also possible to implement something which is based on the existing cache architecture with the required adaptations. We are trying to evaluate if implementing such a new cache storage mode is possible ? is it reasonable ? were there previous thoughts in such directions ? I could find in the wiki reference for both large files and ranges requests, but no conclusions. In a fast glimpse, It does look like it would not be very hard to replace the existing *CacheProcessor *and *CacheVConnection *with different ones which will implement a different cache storage. Thank you in advance for any feedback. -- *Ori Finkelman*Qwilt | Work: +972-72-2221647 | Mobile: +972-52-3832189 | o...@qwilt.com
