By metadata I mean metadata of file
in BufferedChannel.java
public void flush(boolean shouldForceWrite) throws IOException {
synchronized(this) {
flushInternal();
}
if (shouldForceWrite) {
forceWrite(false); <------------- false here
}
}
public long forceWrite(boolean forceMetadata) throws IOException {
// This is the point up to which we had flushed to the file system
page cache
// before issuing this force write hence is guaranteed to be made
durable by
// the force write, any flush that happens after this may or may
// not be flushed
long positionForceWrite = writeBufferStartPosition.get();
fileChannel.force(forceMetadata); <--------------- forcemetadata
false
return positionForceWrite;
}
On Fri, May 5, 2017 at 5:56 PM, Charan Reddy G <reddychara...@gmail.com>
wrote:
> Hey,
>
> In GarbageCollectorThread.doCompactEntryLogs {
>
> ...
>
> try {
> compactEntryLog(scannerFactory, meta);
> scannerFactory.flush(); <--------------- this
> will eventually call entrylogger.flushCurrentLog and it force writes the
> content of the BufferedChannel but not the metadata of the file
>
> LOG.info("Removing entry log {} after compaction",
> meta.getEntryLogId());
> removeEntryLog(meta.getEntryLogId()); <----------- this
> will delete the compacted entrylog
>
> }
> ...
>
>
> in doCompactEntryLogs, we first write the non-deleted entries of the
> entryLog which is getting compacted to the currentLog in entryLogger, then
> we flush the entrylogger before deleting the compacted entrylog. But when
> currentLog is flushed by the entrylogger, it flushes only the content of
> the file but not the metadata. After flush is completed the compacted
> entrylog is deleted. It is not ok to not to force flush the metadata of the
> currentLog for the persisted (checkpointed) data. The filesystem behaviour
> is unexpected in this case and there is possibility of data loss if the
> Bookie crashes before closing that logchannel.
>
> Thanks,
> Charan
>
