Hi Eugen
Thanks, we'll not worry about it then.
Another reboot of the host have fixed it for now but unfortunately it
didn't help anything with the main issue we are having:
https://www.mail-archive.com/ceph-users@ceph.io/msg29255.html
Mvh.
Torkil
On 29-03-2025 01:21, Eugen Block wrote:
I don’t really have an explanation, only that I’ve seen that for years.
On our own cluster I see it as well, still on latest Pacific. I also
noticed that removed devices are not cleaned up from the inventory. I
still see some drives in there which we removed months ago. I thought
maybe it would go away when we upgrade to Reef (waiting for 18.2.5), but
it doesn’t seem like that will happen in newer versions.
Zitat von Torkil Svensgaard <tor...@drcmr.dk>:
On 27/03/2025 10:10, Torkil Svensgaard wrote:
Hi
19.2.1
"
[root@franky ~]# ceph device ls | grep franky
ATA_HGST_HDN726060ALE614_K1GV9P4B franky:sda osd.579
now
ATA_HGST_HDS724040ALE640_PK1334PBH7PZ5P franky:sdn osd.577
now
ATA_HGST_HDS724040ALE640_PK2381P1GNV2UZ franky:sdg osd.580
ATA_Hitachi_HDS5C3030ALA630_MJ1311YNG6DDUA franky:sde osd.533
now
ATA_Hitachi_HDS5C3030ALA630_MJ1311YNG6DGZA franky:sdf osd.527
ATA_Hitachi_HDS5C3030ALA630_MJ1311YNG7R0ZA franky:sdc osd.538
now
ATA_Hitachi_HDS5C3030ALA630_MJ1311YNG7R45A franky:sdx osd.583
ATA_INTEL_SSDSC2BA200G4_BTHV6032026G200MGN franky:sdp osd.532
5% now
ATA_INTEL_SSDSC2KG480G7_PHYM743202DW480BGN franky:sdt osd.531
7% now
ATA_TOSHIBA_MG09ACA18TA_81P0A00PFJEH franky:sdh osd.576
now
ATA_TOSHIBA_MG09ACA18TA_81P0A01KFJEH franky:sdq osd.535
now
ATA_TOSHIBA_MG09ACA18TA_81P0A02RFJEH franky:sdv osd.574
now
ATA_TOSHIBA_MG09ACA18TA_81P0A02WFJEH franky:sdm osd.581
now
ATA_TOSHIBA_MG09ACA18TA_81Q0A00FFJEH franky:sdk osd.582
now
ATA_WDC_WD2002FYPS-01U1B1_WD-WCAVY2249800 franky:sdu osd.536
now
ATA_WDC_WD2002FYPS-01U1B1_WD-WCAVY2258167 franky:sdr osd.530
ATA_WDC_WD2002FYPS-02W3B0_WD-WCAVY6793009 franky:sdl osd.528
ATA_WDC_WD30EFRX-68AX9N0_WD-WMC1T2483048 franky:sdaa osd.529
ATA_WDC_WD30EFRX-68EUZN0_WD-WCC4N6TYT6KZ franky:sdd osd.537
HGST_HUS724040ALS640_PBG458BX franky:sdb osd.568
HGST_HUS724040ALS640_PCJ2VXRX franky:sdj osd.578
HGST_HUS724040ALS640_PCJN3LWX franky:sdi osd.539
HGST_HUS724040ALS640_PCKX3UWX franky:sdo osd.564
HGST_HUS724040ALS640_PCKX4KSX franky:sds osd.575
SAMSUNG_MZPLJ3T2HBJR-00007_S55HNC0R600134 franky:nvme0n1 osd.527
osd.536 osd.539 osd.576 osd.577 osd.581 osd.582 osd.583
0%
SAMSUNG_MZPLL1T6HAJQ-00005_S4CBNA0N601201 franky:nvme1n1 osd.530
osd.535 osd.538 osd.574 osd.575
4%
SAMSUNG_MZPLL1T6HAJQ-00005_S4CBNA0N601254 franky:nvme3n1 osd.528
osd.533 osd.578 osd.579 osd.580
4%
[root@franky ~]# ceph orch device ls | grep franky
franky /dev/md127 ssd 221G No 12m ago
Has a FileSystem
franky /dev/nvme0n1 ssd
SAMSUNG_MZPLJ3T2HBJR-00007_S55HNC0R600134 2980G No 12m
ago Has a FileSystem, LVM detected
franky /dev/nvme1n1 ssd
SAMSUNG_MZPLL1T6HAJQ-00005_S4CBNA0N601201 1490G No 12m
ago Has a FileSystem, LVM detected
franky /dev/nvme2n1 ssd
SAMSUNG_MZPLJ3T2HBJR-00007_S55HNC0R800346 2980G No 12m
ago Has a FileSystem, LVM detected
franky /dev/nvme3n1 ssd
SAMSUNG_MZPLL1T6HAJQ-00005_S4CBNA0N601254 1490G No 12m
ago Has a FileSystem, LVM detected
franky /dev/sda hdd ATA_HGST_HDN726060ALE614_K1GV9P4B
5589G No 12m ago Has a FileSystem, Insufficient
space (<10 extents) on vgs, LVM detected
franky /dev/sdaa hdd ATA_WDC_WD30EFRX-68AX9N0_WD-
WMC1T2483048 2794G No 12m ago Has a FileSystem,
Insufficient space (<10 extents) on vgs, LVM detected
franky /dev/sdb hdd HGST_HUS724040ALS640_PBG458BX
3726G No 12m ago Has a FileSystem, Insufficient space
(<10 extents) on vgs, LVM detected
franky /dev/sdc hdd
ATA_Hitachi_HDS5C3030ALA630_MJ1311YNG7R0ZA 2794G No 12m
ago Has a FileSystem, Insufficient space (<10 extents) on vgs,
LVM detected
franky /dev/sdd hdd ATA_WDC_WD30EFRX-68EUZN0_WD-
WCC4N6TYT6KZ 2794G No 12m ago Has a FileSystem,
Insufficient space (<10 extents) on vgs, LVM detected
franky /dev/sde hdd
ATA_Hitachi_HDS5C3030ALA630_MJ1311YNG6DDUA 2794G No 12m
ago Has a FileSystem, Insufficient space (<10 extents) on vgs,
LVM detected
franky /dev/sdf hdd
ATA_Hitachi_HDS5C3030ALA630_MJ1311YNG6DGZA 2794G No 12m
ago Has a FileSystem, Insufficient space (<10 extents) on vgs,
LVM detected
franky /dev/sdg hdd
ATA_HGST_HDS724040ALE640_PK2381P1GNV2UZ 3726G No 12m
ago Has a FileSystem, Insufficient space (<10 extents) on vgs,
LVM detected
franky /dev/sdh hdd
ATA_TOSHIBA_MG09ACA18TA_81P0A00PFJEH 16.3T No 12m
ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM
detected
franky /dev/sdi hdd HGST_HUS724040ALS640_PCJN3LWX
3726G No 12m ago Has a FileSystem, Insufficient space
(<10 extents) on vgs, LVM detected
franky /dev/sdj hdd HGST_HUS724040ALS640_PCJ2VXRX
3726G No 12m ago Has a FileSystem, Insufficient space
(<10 extents) on vgs, LVM detected
franky /dev/sdk hdd
ATA_TOSHIBA_MG09ACA18TA_81Q0A00FFJEH 16.3T No 12m
ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM
detected
franky /dev/sdl hdd ATA_WDC_WD2002FYPS-02W3B0_WD-
WCAVY6793009 1863G No 12m ago Has a FileSystem,
Insufficient space (<10 extents) on vgs, LVM detected
franky /dev/sdm hdd
ATA_TOSHIBA_MG09ACA18TA_81P0A02WFJEH 16.3T No 12m
ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM
detected
franky /dev/sdn hdd
ATA_HGST_HDS724040ALE640_PK1334PBH7PZ5P 3726G No 12m
ago Has a FileSystem, Insufficient space (<10 extents) on vgs,
LVM detected
franky /dev/sdo hdd HGST_HUS724040ALS640_PCKX3UWX
3726G No 12m ago Has a FileSystem, Insufficient space
(<10 extents) on vgs, LVM detected
franky /dev/sdp ssd
ATA_INTEL_SSDSC2BA200G4_BTHV6032026G200MGN 186G No 12m
ago Has a FileSystem, Insufficient space (<10 extents) on vgs,
LVM detected
franky /dev/sdq hdd
ATA_TOSHIBA_MG09ACA18TA_81P0A01KFJEH 16.3T No 12m
ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM
detected
franky /dev/sdr hdd ATA_WDC_WD2002FYPS-01U1B1_WD-
WCAVY2258167 1863G No 12m ago Has a FileSystem,
Insufficient space (<10 extents) on vgs, LVM detected
franky /dev/sds hdd HGST_HUS724040ALS640_PCKX4KSX
3726G No 12m ago Has a FileSystem, Insufficient space
(<10 extents) on vgs, LVM detected
franky /dev/sdt ssd
ATA_INTEL_SSDSC2KG480G7_PHYM743202DW480BGN 447G No 12m
ago Has a FileSystem, Insufficient space (<10 extents) on vgs,
LVM detected
franky /dev/sdu hdd ATA_WDC_WD2002FYPS-01U1B1_WD-
WCAVY2249800 1863G No 12m ago Has a FileSystem,
Insufficient space (<10 extents) on vgs, LVM detected
franky /dev/sdv hdd
ATA_TOSHIBA_MG09ACA18TA_81P0A02RFJEH 16.3T No 12m
ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM
detected
franky /dev/sdw hdd ATA_WDC_WD30EFRX-68AX9N0_WD-
WMC1T2273861 2794G Yes 12m ago
franky /dev/sdx hdd
ATA_Hitachi_HDS5C3030ALA630_MJ1311YNG7R45A 2794G No 12m
ago Has a FileSystem, Insufficient space (<10 extents) on vgs,
LVM detected
"
Any suggestions as to why a device (nvme2n1) might be missing from
"ceph device ls"?
Verbose output shows the device but with no device name:
"
submit {"prefix": "device ls-by-host", "host": "franky", "target":
["mon-mgr", ""]} to mon-mgr
DEVICE DEV DAEMONS
EXPECTED FAILURE
ATA_HGST_HDN726060ALE614_K1GV9P4B sda osd.579
now
ATA_HGST_HDS724040ALE640_PK1334PBH7PZ5P sdn osd.577
now
ATA_HGST_HDS724040ALE640_PK2381P1GNV2UZ sdg osd.580
ATA_Hitachi_HDS5C3030ALA630_MJ1311YNG6DDUA sde osd.533
now
ATA_Hitachi_HDS5C3030ALA630_MJ1311YNG6DGZA sdf osd.527
ATA_Hitachi_HDS5C3030ALA630_MJ1311YNG7R0ZA sdc osd.538
now
ATA_Hitachi_HDS5C3030ALA630_MJ1311YNG7R45A sdx osd.583
ATA_INTEL_SSDSC2BA200G4_BTHV6032026G200MGN sdp osd.532
now
ATA_INTEL_SSDSC2KG480G7_PHYM743202DW480BGN sdt osd.531
now
ATA_TOSHIBA_MG09ACA18TA_81P0A00PFJEH sdh osd.576
now
ATA_TOSHIBA_MG09ACA18TA_81P0A01KFJEH sdq osd.535
now
ATA_TOSHIBA_MG09ACA18TA_81P0A02RFJEH sdv osd.574
now
ATA_TOSHIBA_MG09ACA18TA_81P0A02WFJEH sdm osd.581
now
ATA_TOSHIBA_MG09ACA18TA_81Q0A00FFJEH sdk osd.582
now
ATA_WDC_WD2002FYPS-01U1B1_WD-WCAVY2249800 sdu osd.536
now
ATA_WDC_WD2002FYPS-01U1B1_WD-WCAVY2258167 sdr osd.530
ATA_WDC_WD2002FYPS-02W3B0_WD-WCAVY6793009 sdl osd.528
ATA_WDC_WD30EFRX-68AX9N0_WD-WMC1T2483048 sdaa osd.529
ATA_WDC_WD30EFRX-68EUZN0_WD-WCC4N6TYT6KZ sdd osd.537
HGST_HUS724040ALS640_PBG458BX sdb osd.568
HGST_HUS724040ALS640_PCJ2VXRX sdj osd.578
HGST_HUS724040ALS640_PCJN3LWX sdi osd.539
HGST_HUS724040ALS640_PCKX3UWX sdo osd.564
HGST_HUS724040ALS640_PCKX4KSX sds osd.575
SAMSUNG_MZPLJ3T2HBJR-00007_S55HNC0R600134 nvme0n1 osd.527 osd.536
osd.539 osd.576 osd.577 osd.581 osd.582 osd.583
SAMSUNG_MZPLJ3T2HBJR-00007_S55HNC0R800346 osd.529 osd.537
osd.564 osd.568
SAMSUNG_MZPLL1T6HAJQ-00005_S4CBNA0N601201 nvme1n1 osd.530 osd.535
osd.538 osd.574 osd.575
SAMSUNG_MZPLL1T6HAJQ-00005_S4CBNA0N601254 nvme3n1 osd.528 osd.533
osd.578 osd.579 osd.580
"
Mvh.
Torkil
It is being used as db device for 4 OSDs.
Mvh.
Torkil
--
Torkil Svensgaard
Sysadmin
MR-Forskningssektionen, afs. 714
DRCMR, Danish Research Centre for Magnetic Resonance
Hvidovre Hospital
Kettegård Allé 30
DK-2650 Hvidovre
Denmark
Tel: +45 386 22828
E-mail: tor...@drcmr.dk
_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
--
Torkil Svensgaard
Sysadmin
MR-Forskningssektionen, afs. 714
DRCMR, Danish Research Centre for Magnetic Resonance
Hvidovre Hospital
Kettegård Allé 30
DK-2650 Hvidovre
Denmark
Tel: +45 386 22828
E-mail: tor...@drcmr.dk
_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
