Hello my fellow Hadoop users/developers, I'm reading the SequenceFile source code, and there is a checkAndWriteSync() method that writes a sync marker every so many bytes. I was wondering what's the use of the sync marker. I know one can use it to designate the end of a header, but it's also used in the process of writing the uncompressed data. I wish I could have figured it out :(
Thank you so much