It's used to align input splits of the SequenceFile. A reader can start at an arbitrary offset, then find the boundary of the next block of records by looking for the sync marker defined in the header. -C
On Mon, Mar 21, 2011 at 7:40 AM, Weishung Chung <weish...@gmail.com> wrote: > Hello my fellow Hadoop users/developers, > > I'm reading the SequenceFile source code, and there is a checkAndWriteSync() > method that writes a sync marker every so many bytes. I was wondering what's > the use of the sync marker. I know one can use it to designate the end of a > header, but it's also used in the process of writing the uncompressed data. > I wish I could have figured it out :( > > Thank you so much >
