xfs_growfs "autodetects" the block device size. You can force re-read of the block device to refresh this info but might not do anything at all.
There are situations when block device size will not reflect reality - for example you can't (or at least couldn't) resize partition that is in use (mounted, mapped, used in LVM...) without serious hacks, and ioctls on this partition will return the old size until you reboot. The block device can also simply lie (like if you triggered a bug that made the rbd device visually larger). Device-mapper devices have their own issues. The only advice I can give is to never, ever shrink LUNs or block devices and to avoid partitions if you can. I usually set up a fairly large OS drive (with oversized partitions to be safe, assuming you have thin-provisioning it wastes no real space) and a separate data volume without any partitioning. This also works-around possible alignment issues.... Growing is always safe, shrinking destroys data. I am very surprised that "rbd resize" doesn't require something like "--i-really-really-know-what-i-am-doing --please-eatmydata" parameter to shrink the image (or does it ask for confirmation when shrinking at least? I can't try it now). Making a typo == instawipe? My bet would still be that the original image was larger and you shrunk it by mistake. The kernel client most probably never gets the capacity change notification and you end up creating filesystem that points outside the device. (not sure if mkfs.xfs actually tries seeking over the full sector range). This is the most plausible explanation I can think of, but anything is possible. I have other ideas if you want to investigate but I'd take it off-list... Jan P.S. Your image is not 2TB but rather 2000 GiB ;-) > On 12 Nov 2015, at 22:10, Bogdan SOLGA <bogdan.so...@gmail.com> wrote: > > Unfortunately I can no longer execute those commands for that rbd5, as I had > to delete it; I couldn't 'resurrect' it, at least not in a decent time. > > Here is the output for another image, which is 2TB big: > > ceph-admin@ceph-client-01:~$ sudo blockdev --getsz --getss --getbsz /dev/rbd1 > 4194304000 > 512 > 512 > > ceph-admin@ceph-client-01:~$ xfs_info /dev/rbd1 > meta-data=/dev/rbd2 isize=256 agcount=8127, agsize=64512 blks > = sectsz=512 attr=2 > data = bsize=4096 blocks=524288000, imaxpct=25 > = sunit=1024 swidth=1024 blks > naming =version 2 bsize=4096 ascii-ci=0 > log =internal bsize=4096 blocks=2560, version=2 > = sectsz=512 sunit=8 blks, lazy-count=1 > realtime =none extsz=4096 blocks=0, rtextents=0 > > > I know rbd can also shrink the image, but I'm sure I haven't shrunk it. What > I have tried, accidentally, was to resize the image to the same size it > previously had, and that operation has failed, after trying for some time. > Hmm... I think the failed resize was the culprit for it's malfunctioning, > then. > > Any (additional) advices on how to prevent this type of issues, in the > future? Should the resizing and the xfs_growfs be executed with some > parameters, for a better configuration of the image and / or filesystem? > > Thank you very much for your help! > > Regards, > Bogdan > > > On Thu, Nov 12, 2015 at 11:00 PM, Jan Schermer <j...@schermer.cz > <mailto:j...@schermer.cz>> wrote: > Can you post the output of: > > blockdev --getsz --getss --getbsz /dev/rbd5 > and > xfs_info /dev/rbd5 > > rbd resize can actually (?) shrink the image as well - is it possible that > the device was actually larger and you shrunk it? > > Jan > >> On 12 Nov 2015, at 21:46, Bogdan SOLGA <bogdan.so...@gmail.com >> <mailto:bogdan.so...@gmail.com>> wrote: >> >> By running rbd resize <http://docs.ceph.com/docs/master/rbd/rados-rbd-cmds/> >> and then 'xfs_growfs -d' on the filesystem. >> >> Is there a better way to resize an RBD image and the filesystem? >> >> On Thu, Nov 12, 2015 at 10:35 PM, Jan Schermer <j...@schermer.cz >> <mailto:j...@schermer.cz>> wrote: >> >>> On 12 Nov 2015, at 20:49, Bogdan SOLGA <bogdan.so...@gmail.com >>> <mailto:bogdan.so...@gmail.com>> wrote: >>> >>> Hello Jan! >>> >>> Thank you for your advices, first of all! >>> >>> The filesystem was created using mkfs.xfs, after creating the RBD block >>> device and mapping it on the Ceph client. I haven't specified any >>> parameters when I created the filesystem, I just ran mkfs.xfs on the image >>> name. >>> >>> As you mentioned the filesystem thinking the block device should be larger >>> than it is - I have initially created that image as a 2GB image, and then >>> resized it to be much bigger. Could this be the issue? >> >> Sounds more than likely :-) How exactly did you grow it? >> >> Jan >> >>> >>> There are several RBD images mounted on one Ceph client, but only one of >>> them had issues. I have made a clone, and I will try running fsck on it. >>> >>> Fortunately it's not important data, it's just testing data. If I won't >>> succeed repairing it I will trash and re-create it, of course. >>> >>> Thank you, once again! >>> >>> >>> >>> On Thu, Nov 12, 2015 at 9:28 PM, Jan Schermer <j...@schermer.cz >>> <mailto:j...@schermer.cz>> wrote: >>> How did you create filesystems and/or partitions on this RBD block device? >>> The obvious causes would be >>> 1) you partitioned it and the partition on which you ran mkfs points or >>> pointed during mkfs outside the block device size (happens if you for >>> example automate this and confuse sectors x cylinders, or if you copied the >>> partition table with dd or from some image) >>> or >>> 2) mkfs created the filesystem with pointers outside of the block device >>> for some other reason (bug?) >>> or >>> 3) this RBD device is a snapshot that got corrupted (or wasn't snapshotted >>> in crash-consistent state and you got "lucky") and some reference points to >>> a non-sensical block number (fsck could fix this, but I wouldn't trust the >>> data integrity anymore) >>> >>> Basically the filesystem thinks the block device should be larger than it >>> is and tries to reach beyond. >>> >>> Is this just one machine or RBD image or is there more? >>> >>> I'd first create a snapshot and then try running fsck on it, it should >>> hopefully tell you if there's a problem in setup or a corruption. >>> >>> If it's not important data and it's just one instance of this problem then >>> I'd just trash and recreate it. >>> >>> Jan >>> >>>> On 12 Nov 2015, at 20:14, Bogdan SOLGA <bogdan.so...@gmail.com >>>> <mailto:bogdan.so...@gmail.com>> wrote: >>>> >>>> Hello everyone! >>>> >>>> We have a recently installed Ceph cluster (v 0.94.5, Ubuntu 14.04), and >>>> today I noticed a lot of 'attempt to access beyond end of device' messages >>>> in the /var/log/syslog file. They are related to a mounted RBD image, and >>>> have the following format: >>>> >>>> Nov 12 21:06:44 ceph-client-01 kernel: [438507.952532] attempt to access >>>> beyond end of device >>>> Nov 12 21:06:44 ceph-client-01 kernel: [438507.952534] rbd5: rw=33, >>>> want=6193176, limit=4194304 >>>> >>>> After restarting that Ceph client, I see a lot of 'metadata I/O error' >>>> messages in the boot log: >>>> >>>> XFS (rbd5): metadata I/O error: block 0x46e001 >>>> ("xfs_buf_iodone_callbacks") error 5 numblks 1 >>>> >>>> Any idea on why these messages are shown? The health of the cluster shows >>>> as OK, and I can access that block device without (apparent) issues... >>>> >>>> Thank you! >>>> >>>> Regards, >>>> Bogdan >>>> _______________________________________________ >>>> ceph-users mailing list >>>> ceph-users@lists.ceph.com <mailto:ceph-users@lists.ceph.com> >>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>>> <http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com> >>> >>> >> >> > >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
