Rick, I've run into the same issue.I don't think this is like LU-11306. Moving a file into a directory assigned to another MDT doesn't change its MDT assignment. Copying a the same file into that directory does. That behavior is as expected.
However, when I migrated files off of MDT1 with 'lfs migrate -m0 robinhood' the number of inodes used on MDT1 did not change. If I migrated back and forth between the MDTs, the number of inodes used on MDT1 increased each time I migrated files to it but never decreased unless i deleted the files. And then inodes used decreased only by the amount of the most recent migration.
I migrated the directory back to MDT1, mounted the MDTs as ldiskfs and examined them:
===== /mnt/mdt0-ldiskfs/ROOT/robinhood # exists but is empty /mnt/mdt1-ldiskfs/REMOTE_PARENT_DIR/0xcc0000402:0x3c:0x0/ # contained the files within robinhood/ =====Again, that is as I expected it would be. But the IUsed count doesn't seem to reflect that.
For our use case, I think we're okay. But it would be nice to know why the number of inodes used does not decrease when file are migrated off of the MDT using 'lfs migrate -m'.
Best, Jesse On 3/29/19 4:36 PM, Mohr Jr, Richard Frank (Rick Mohr) wrote:
I have been playing a little bit with DNE today, and I had a question about some odd behavior I saw regarding inode counts. My Lustre 2.10.6 file system has 2 MDTs. I created a directory (which by default resides on MDT0) and then created 10 files in that directory: [root@sip-mgmt2 test]# lfs df -i /lustre/newfs | grep MDT siplfs-MDT0000_UUID 3906232784 395 3906232389 0% /lustre/newfs[MDT:0] siplfs-MDT0001_UUID 2822275072 276 2822274796 0% /lustre/newfs[MDT:1] [root@sip-mgmt2 test]# mkdir subdir [root@sip-mgmt2 test]# for i in `seq 1 10`; do touch subdir/file.$i; done [root@sip-mgmt2 test]# lfs df -i /lustre/newfs | grep MDT siplfs-MDT0000_UUID 3906232784 406 3906232378 0% /lustre/newfs[MDT:0] siplfs-MDT0001_UUID 2822275072 276 2822274796 0% /lustre/newfs[MDT:1] As expected, the inode count for MDT0 increases by about 10. Then I used “lfs migrate” to move that directory to MDT1: [root@sip-mgmt2 test]# lfs migrate -m 1 subdir [root@sip-mgmt2 test]# lfs df -i /lustre/newfs | grep MDT siplfs-MDT0000_UUID 3906232784 408 3906232376 0% /lustre/newfs[MDT:0] siplfs-MDT0001_UUID 2822275072 289 2822274783 0% /lustre/newfs[MDT:1] I expected the inode count for MDT1 to increase by about 10, but I did not see the MDT0 inode count decrease. So then I decided to migrate the directory back to MDT0: [root@sip-mgmt2 test]# lfs migrate -m 0 subdir [root@sip-mgmt2 test]# lfs df -i /lustre/newfs | grep MDT siplfs-MDT0000_UUID 3906232784 420 3906232364 0% /lustre/newfs[MDT:0] siplfs-MDT0001_UUID 2822275072 290 2822274782 0% /lustre/newfs[MDT:1] The inode count increases again on MDT0, but it does not decrease on MDT1. So then I decided to delete the directory and all the files: [root@sip-mgmt2 test]# rm -rf subdir [root@sip-mgmt2 test]# lfs df -i /lustre/newfs | grep MDT siplfs-MDT0000_UUID 3906232784 409 3906232375 0% /lustre/newfs[MDT:0] siplfs-MDT0001_UUID 2822275072 290 2822274782 0% /lustre/newfs[MDT:1] This reclaims about 10 inodes on MDT0, but both MDT0 and MDT1 still seem to have a permanently increased inode count resulting from the previous lfs migrate commands. Even after waiting a while and remounting the MDTs, the inodes don’t seem to be reclaimed. It seems like this is a bug, but since this is the first time I have used DNE, I thought maybe I am just misunderstanding something or using the lfs migrate command incorrectly. It looks similar to LU-11306, but that issue seemed to deal with renaming files and not migrating them. Any help would be appreciated. -- Rick Mohr Senior HPC System Administrator National Institute for Computational Sciences http://www.nics.tennessee.edu _______________________________________________ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
smime.p7s
Description: S/MIME Cryptographic Signature
_______________________________________________ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
