Got the root cause, it's a dup of HDFS-8072
https://issues.apache.org/jira/browse/HDFS-8072
On Wed, Jan 10, 2018 at 2:20 PM, Xie Gang <xiegang...@gmail.com> wrote:
> Hi,
>
> Recently, we hit an issue that, there is a difference between the
> freeSpace of the datanode volume info and the OS df:
>
> For example:
> the jmx of the dn shows:
>
> "VolumeInfo" :
> "{\"/....\":{\"freeSpace\":1445398864500,\"usedSpace\":228138206927,\"reservedSpace\":53687091200}}",
>
> But the df shows:
> /dev/sda 2146676656 <(214)%20667-6656> 253778008 1785508084 13% /...
>
> There is about 400GB gap which is regarded as Non DFS used. And the most
> strange thing is that, after I restart the dn process, the gap disappear.
> And after some days, the gap shows again.
>
> The yarn shared the same server of the dn and has some file cache. Could
> it be related?
>
> The direct cause is that the freeSpace from dn is quit different from the
> available space from df. After tracking down the code, freeSpace of the dn
> is from dirFile.getUsableSpace(). could it have some problem? Do we hit
> this issue before?
>
> Thanks,
> Gang
>
>
> --
> Xie Gang
>
--
Xie Gang
