Thanks for the detailed explanation. How does ATS handle storage of HTTP
range replies? Is there a single copy of the object in cache or is there an
cache entry for every range request which comes in. For example
GET /foo HTTP/1.1
Range: bytes=0-2047
GET /foo HTTP/1.1
Range: bytes=1024-2047
Would these 2 requests in the order given result in the 2nd entry being
served from cache?
I am assuming if the entire object is in the cache, any subsequent range
requests would be satisfied from cache if the object is still fresh?
thanks,
Sridhar
On Tue, May 10, 2011 at 6:36 PM, John Plevyak <[email protected]> wrote:
> I really need to generate some documentation on this.
>
> The structure of the cache is a cyclone buffer (circular buffer).
> Documents over 2MB are broken into fragments. The 'head'
> (which for HTTP includes a vector of headers for the alternate versions
> of documents (think encodings in your case) appears at the "end", so
> any document which has a "head" and a "first" fragment has all the
> fragments in the cache.
>
> The "directory" is highly optimized hash from CacheKey (MD5) to
> offset, size pairs for fragments. For multi-fragment documents the head
> contains a map from offsets -> CacheKeys for the fragments.
>
> Is it possible to read from outside.... on a fully synced system which
> isn't running, yes. On a running system, it would be very hard as
> (like any file system) the disk image not completely in sync with the
> in-core directory and the cache deletes and updates documents and will
> evacuate (think log cleaning) fragments which are in use but
> about to be overwritten. On restart from a unclean shutdown (frankly the
> normal kind if you are like me and use control-c to shut it down, the
> disk image needs to be brought back to consistency on startup which
> requires
> running through the log of writes.
>
> You would be better off just writing your code within ATS or using the
> iocore
> as a separate module via configure --enable-standalone-iocore
>
> As of 2.1.8 the cache should work very well for video content as it
> supports
> 2MB fragments and large disks efficiently. 2.0.1 breaks documents into 32K
> chunks and disks into 8GB chunks which is terrible for video.
>
> You could either do video over HTTP or write your own protocol handler. At
> some point
> I'll release an example of that.
>
> john
>
>
>
> On Tue, May 10, 2011 at 2:44 PM, Nelson Perez <[email protected]> wrote:
>
> > Hi there, I would like to learn more details about the cache
> > structure. Specially is it possible for an outside process to somehow
> > have read access to it? for what I know it is supposed to be a huge
> > hash table, so given the right permissions and an offset would it be
> > possible to extract usable data from the cache file directly?
> >
> > Also since I'm mostly interested in caching video content, I would
> > like to know if TS cache mechanism may pose any drawback for possibly
> > large files.
> >
> > Thanks a lot.
> > Nelson R. Pérez
> >
>
