https://issues.apache.org/jira/browse/NIFI-9056
On Wed, Sep 22, 2021 at 10:26 AM Mark Bean <[email protected]> wrote: > We are experiencing an issue where content claims are not being cleaned up. > This causes the content repo to grow, and eventually fill the disk. We only > see this behavior when using the RocksDBFlowFileRepository, but not when > using the WriteAheadFlowFileRepository. > > Unfortunately, we have not been able to create a consistently reproducible > test case to recreate the problem. It is only observed on a cluster that is > heavily loaded. > > Evidence of the problem (besides the content repo filling up) is that upon > restarting NiFi, there are hundreds of messages: > > 2021-09-22 11:39:50,052 INFO [main] 0.a.n.c.repository.FileSystemRepository > Found unknown file /content_repository/60/1632307980953-60 (814776 bytes) > in File System Repository; removing file > > The crux of the problem is that the claimant count for a given claim is not > being updated correctly. The count never reaches zero even when all > flowfiles which had been in the given claim have been dropped from NiFi. > (This is why the claim is able to be removed on restart.) Therefore, the > claim is considered "in use" and is not cleaned up. (Content archiving is > disabled as well, so no claims are advanced to the archive.) > > The issue is not seen in 1.11.4, but is seen in 1.12.1 and 1.14.0. I > suspect it may be related to changes made in NIFI-6849. > https://issues.apache.org/jira/browse/NIFI-6849 > > Any insights or suggestions are greatly appreciated. > > Thanks! > -Mark >
