Kafka use write() calls to append data to log files, note it is sequential writes. In 0.8 we include an index file to improve searching for physical positions given the offsets, which used mmaping.
On Sun, Mar 1, 2015 at 9:42 PM, Philip O'Toole < philip.oto...@yahoo.com.invalid> wrote: > Thanks Guozhang -- no this isn't quite it. The doc I read before contained > the rationale for using physical offsets in the file, not logical offsets. > I know the current version of Kafka now uses logical offsets again. It's > not a big deal though, I generally remember the contents of the page, and > the important section about using the OS for caching is also contained the > 0.8 docs. I was more curious about re-reading it. > > I do have one question though. There are two ways (that I know of) of > accessing a file -- the read() and write() system calls, or mmap'ing the > file. Both go through the OS file cache, as far as I know. Which technique > does Kafka actually use, when accessing log files? I always wondered. I > started looking at the Scala source, but it's not immediately clear to me. > > Thanks, > Philip > ----------------------------------------- > http://www.philipotoole.com > > On Saturday, February 28, 2015 9:33 PM, Guozhang Wang < > wangg...@gmail.com> wrote: > > > Is this you are looking for? > > http://kafka.apache.org/07/documentation.html > > On Fri, Feb 27, 2015 at 7:02 PM, Philip O'Toole < > philip.oto...@yahoo.com.invalid> wrote: > > > There used to be available a very lucid page describing Kafka 0.7, its > > design, and the rationale behind certain decisions. I last saw it about > 18 > > months ago. I can't find it now. Is it still available? I can find the > 0.8 > > version, it's up there on the site. > > > > Any help? Any links? > > > > Philip > > > > ------------------------------------ > > http://www.philipotoole.com > > > > > -- > -- Guozhang > > > > -- -- Guozhang
