These docs might help https://kafka.apache.org/08/design.html http://research.microsoft.com/en-us/um/people/srikanth/netdb11/netdb11papers/netdb11-final12.pdf -Harsha
On Sun, Mar 1, 2015, at 09:42 PM, Philip O'Toole wrote: > Thanks Guozhang -- no this isn't quite it. The doc I read before > contained the rationale for using physical offsets in the file, not > logical offsets. I know the current version of Kafka now uses logical > offsets again. It's not a big deal though, I generally remember the > contents of the page, and the important section about using the OS for > caching is also contained the 0.8 docs. I was more curious about > re-reading it. > > I do have one question though. There are two ways (that I know of) of > accessing a file -- the read() and write() system calls, or mmap'ing the > file. Both go through the OS file cache, as far as I know. Which > technique does Kafka actually use, when accessing log files? I always > wondered. I started looking at the Scala source, but it's not immediately > clear to me. > > Thanks, > Philip > ----------------------------------------- > http://www.philipotoole.com > > On Saturday, February 28, 2015 9:33 PM, Guozhang Wang > <wangg...@gmail.com> wrote: > > > Is this you are looking for? > > http://kafka.apache.org/07/documentation.html > > On Fri, Feb 27, 2015 at 7:02 PM, Philip O'Toole < > philip.oto...@yahoo.com.invalid> wrote: > > > There used to be available a very lucid page describing Kafka 0.7, its > > design, and the rationale behind certain decisions. I last saw it about 18 > > months ago. I can't find it now. Is it still available? I can find the 0.8 > > version, it's up there on the site. > > > > Any help? Any links? > > > > Philip > > > > ------------------------------------ > > http://www.philipotoole.com > > > > > -- > -- Guozhang > > >
