Hello,
I am seeing in some low traffic bookie a weird behaviour about garbage
collection of journal files.
Let's say that I have a bookie that has this data on the journal directory
drwxrwxr-x. 2 65536 27 set 09.01 .
drwxrwxr-x. 3 4096 9 gen 2019 ..
-rw-rw-r--. 1 1107296256 24 set 18.08 168331bd95f.txn
-rw-rw-r--. 1 1107296256 26 set 07.46 168331bd960.txn
-rw-rw-r--. 1 704643072 26 set 15.21 168331bd961.txn
-rw-rw-r--. 1 301989888 27 set 08.57 168331bd962.txn
-rw-rw-r--. 1 16777216 27 set 08.57 168331bd963.txn
-rw-rw-r--. 1 16777216 27 set 09.01 168331bd964.txn
-rw-rw-r--. 1 208 9 gen 2019 VERSION
Last mark tells:
LastLogMark : Journal Id - 1547045689698(168331bd962.txn), Pos - 301541888
I see on logs only lines like:
Flush ledger storage at checkpoint CheckpointList{checkpoints=[LogMark:
logFileId - 1547045689698 , logFileOffset - 301541888]}
In **other** heavy traffic bookies I sometime see this line:
garbage collected journal 150cdc02736.txn
As I have maxJournalBackups=1 I would like to see that the bookie drops the
files before 16833db962.txn file and release disk space but this is
actually not happening.
I see that in Journal.java we are not deleting date if the checkkpoint is
not a LogMarkCheckpoint
https://github.com/apache/bookkeeper/blob/afd8dbadd8a205fe4de4daa9d0c09680effab49d/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Journal.java#L735
It seems to me that the actual class is a CheckpointList that is wrapping a
LogMarkCheckpoint.
I think that a LogMarkCheckpoint may arrive to that line of code only in
case of an hard flush due to entry log file size limit reached
https://github.com/apache/bookkeeper/blob/master/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/SortedLedgerStorage.java#L302
If my analysis is correct.....
- do we have a bug in Journal.java#L735 (liked above) ?
- is there any way to force the Journal to reclaim disk space without
writes (that force a flush of the EntryLogger) ?
Enrico
