I've got a problem where fsck behaves differently from my expectations.
The problem happens with a filesystem on a GELI encrypted ZVOL.
The volume has 4K block size and that's the GELI's sector size as well.
FreeBSD is stable/13 from mid January.
Let me demonstrate:
# fsck -C -p /dev/zvol/.../vault.eli
/dev/zvol/.../vault.eli: NO WRITE ACCESS
/dev/zvol/.../vault.eli: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
# fsck /dev/zvol/.../vault.eli
** /dev/zvol/.../vault.eli
SAVE DATA TO FIND ALTERNATE SUPERBLOCKS? [yn] n
ADD CYLINDER GROUP CHECK-HASH PROTECTION? [yn] n
ADD SUPERBLOCK CHECK-HASH PROTECTION? [yn] n
ADD INODE CHECK-HASH PROTECTION? [yn] n
** Last Mounted on /usr/home/avg/secret
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames
** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
** Phase 5 - Check Cyl groups
136 files, 371 used, 253491 free (35 frags, 31682 blocks, 0.0% fragmentation)
***** FILE SYSTEM IS CLEAN *****
# fsck -C -p /dev/zvol/.../vault.eli
/dev/zvol/.../vault.eli: NO WRITE ACCESS
/dev/zvol/.../vault.eli: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
# mount /dev/zvol/.../vault.eli ~/secret
<success>
So, mount can mount the filesystem without problems.
Plain fsck suggests to enable some features, it does not find any problem and
sees the filesystem as clean.
But fsck -C -p fails with a rather cryptic error.
I don't think I saw this before.
If I try to allow fsck to enable some things that it wants, it actually does
fail to write:
# fsck /dev/zvol/.../vault.eli
** /dev/zvol/.../vault.eli
SAVE DATA TO FIND ALTERNATE SUPERBLOCKS? [yn] y
CANNOT WRITE BLK: 15
CONTINUE? [yn] n
With ktrace I see:
fsck_ufs CALL pwrite(0x4,0x800aa0000,0x1000,0x1e00)
fsck_ufs RET pwrite -1 errno 22 Invalid argument
So, a 4K write but the offset is not 4K aligned.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
A little bit more data:
# dumpfs -m /dev/zvol/.../vault.eli
# newfs command for /dev/zvol/.../vault.eli (/dev/zvol/.../vault.eli)
newfs -L vault -O 2 -a 4 -b 32768 -d 32768 -e 4096 -f 4096 -g 16384 -h 64 -i
8192 -k 2616 -m 8 -o time -s 2097144 /dev/zvol/.../vault.eli
# geli list
Geom name: zvol/.../vault.eli
State: ACTIVE
EncryptionAlgorithm: AES-XTS
KeyLength: 128
Crypto: software
Version: 7
UsedKey: 0
Flags: NONE
KeysAllocated: 1
KeysTotal: 1
Providers:
1. Name: zvol/.../vault.eli
Mediasize: 1073737728 (1.0G)
Sectorsize: 4096
Mode: r0w0e0
Consumers:
1. Name: zvol/.../vault
Mediasize: 1073741824 (1.0G)
Sectorsize: 512
Stripesize: 4096
Stripeoffset: 0
Mode: r1w1e1
# tunefs -p /dev/zvol/.../vault.eli
tunefs: POSIX.1e ACLs: (-a) disabled
tunefs: NFSv4 ACLs: (-N) disabled
tunefs: MAC multilabel: (-l) disabled
tunefs: soft updates: (-n) disabled
tunefs: soft update journaling: (-j) disabled
tunefs: gjournal: (-J) disabled
tunefs: trim: (-t) disabled
tunefs: maximum blocks per file in a cylinder group: (-e) 4096
tunefs: average file size: (-f) 16384
tunefs: average number of files in a directory: (-s) 64
tunefs: minimum percentage of free space: (-m) 8%
tunefs: space to hold for metadata blocks: (-k) 2616
tunefs: optimization preference: (-o) time
tunefs: volume label: (-L) vault
# dumpfs /dev/zvol/.../vault.eli
magic 19540119 (UFS2)
last mounted time Mon Feb 7 13:44:30 2022
last modified time Mon Feb 7 13:51:18 2022
superblock location 65536 id [ 581f28b7 8a03690e ]
ncg 4 size 262143 blocks 253862
bsize 32768 shift 15 mask 0xffff8000
fsize 4096 shift 12 mask 0xfffff000
frag 8 shift 3 fsbtodb 3
minfree 8% optim time symlinklen 120
maxbsize 32768 maxbpg 4096 maxcontig 4 contigsumsize 4
nbfree 31682 ndir 27 nifree 130934 nffree 35
bpg 8192 fpg 65536 ipg 32768 unrefs 0
nindir 4096 inopb 128 maxfilesize 2252349704110079
sbsize 4096 cgsize 16384 csaddr 2088 cssize 4096
sblkno 24 cblkno 32 iblkno 40 dblkno 2088
cgrotor 0 fmod 0 ronly 0 clean 1
metaspace 2616 avgfpdir 64 avgfilesize 16384
flags none
fsmnt /usr/home/avg/secret
volname vault swuid 0 providersize 262143
cs[].cs_(nbfree,ndir,nifree,nffree):
(7905,3,32748,1) (7920,17,32691,10) (7928,3,32755,21) (7929,4,32740,3)
blocks in last group 8191
cg 0:
magic 90255 tell 20000 time Fri Dec 17 00:25:26 2021
cgx 0 ndblk 65536 niblk 32768 initiblk 256 unrefs 0
nbfree 7905 ndir 3 nifree 32748 nffree 1
rotor 4872 irotor 19 frotor 4744
frsum 1 0 0 0 0 0 0
sum of frsum: 1
clusters 1-3: 0 0 0
clusters size 4 and over: 2
clusters free: 264-587, 611-8191
inodes used: 0-19
blks free: 2112-4703, 4751, 4888-65535
cg 1:
magic 90255 tell 10020000 time Thu Oct 21 10:39:16 2021
cgx 1 ndblk 65536 niblk 32768 initiblk 256 unrefs 0
nbfree 7920 ndir 17 nifree 32691 nffree 10
rotor 4776 irotor 20 frotor 4776
frsum 5 1 1 0 0 0 0
sum of frsum: 10
clusters 1-3: 0 0 1
clusters size 4 and over: 2
clusters free: 0-2, 265-587, 598-8191
inodes used: 0-19, 21-66, 68, 70, 74-82
blks free: 0-23, 2118, 2120-4703, 4726, 4743, 4745,
4757-4758, 4764, 4781-65535
cg 2:
magic 90255 tell 20020000 time Tue Dec 28 18:40:22 2021
cgx 2 ndblk 65536 niblk 32768 initiblk 256 unrefs 0
nbfree 7928 ndir 3 nifree 32755 nffree 21
rotor 4720 irotor 6 frotor 4712
frsum 0 0 0 2 0 1 1
sum of frsum: 21
clusters 1-3: 0 0 1
clusters size 4 and over: 3
clusters free: 0-2, 262-587, 590-641, 645-8191
inodes used: 0-5, 8-9, 12-14, 17, 19
blks free: 0-23, 2090-4703, 4716-5135, 5144-5147, 5153-65535
cg 3:
magic 90255 tell 30020000 time Wed Sep 1 12:52:02 2021
cgx 3 ndblk 65535 niblk 32768 initiblk 256 unrefs 0
nbfree 7929 ndir 4 nifree 32740 nffree 3
rotor 4736 irotor 26 frotor 4728
frsum 0 0 1 0 0 0 0
sum of frsum: 3
clusters 1-3: 0 0 1
clusters size 4 and over: 2
clusters free: 0-2, 261-587, 592-8190
inodes used: 0-25, 27-28
blks free: 0-23, 2088-4703, 4731-4733, 4736-65527
--
Andriy Gapon
