Thank you Alfredo
I did not have any reasons to keep volumes around.
I tried using ceph-volume to zap these stores, but none of the command
worked, including yours 'ceph-volume lvm zap
osvg-sdd-db/2ukzAx-g9pZ-IyxU-Sp9h-fHv2-INNY-1vTpvz'
I ended up manually removing LUKS volumes and then deleting LVM LV, VG, and
PV
cryptsetup remove /dev/mapper/AeV0iG-odWF-NRPE-1bVK-0mxH-OgHL-fneTzr
cryptsetup remove /dev/mapper/2ukzAx-g9pZ-IyxU-Sp9h-fHv2-INNY-1vTpvz
lvremove
/dev/ceph-f4efa78f-a467-4214-b550-81653da1c9bd/osd-block-097d59be-bbe6-493a-b785-48b259d2ff35
sgdisk -Z /dev/sdd
# ceph-volume lvm zap osvg-sdd-db/2ukzAx-g9pZ-IyxU-Sp9h-fHv2-INNY-1vTpvz
Running command: /usr/sbin/cryptsetup status /dev/mapper/
--> Zapping: osvg-sdd-db/2ukzAx-g9pZ-IyxU-Sp9h-fHv2-INNY-1vTpvz
Running command: /usr/sbin/wipefs --all
osvg-sdd-db/2ukzAx-g9pZ-IyxU-Sp9h-fHv2-INNY-1vTpvz
stderr: wipefs: error: osvg-sdd-db/2ukzAx-g9pZ-IyxU-Sp9h-fHv2-INNY-1vTpvz:
probing initialization failed: No such file or directory
--> RuntimeError: command returned non-zero exit status: 1
On Thu, Apr 18, 2019 at 10:10 AM Alfredo Deza <ad...@redhat.com> wrote:
> On Thu, Apr 18, 2019 at 10:55 AM Sergei Genchev <sgenc...@gmail.com>
> wrote:
> >
> > Hello,
> > I have a server with 18 disks, and 17 OSD daemons configured. One of the
> OSD daemons failed to deploy with ceph-deploy. The reason for failing is
> unimportant at this point, I believe it was race condition, as I was
> running ceph-deploy inside while loop for all disks in this server.
> > Now I have two left over LVM dmcrypded volumes that I am not sure how
> clean up. The command that failed and did not quite clean up after itself
> was:
> > ceph-deploy osd create --bluestore --dmcrypt --data /dev/sdd --block-db
> osvg/sdd-db ${SERVERNAME}
> >
> > # lsblk
> > .......
> > sdd 8:48 0 7.3T 0 disk
> >
> └─ceph--f4efa78f--a467--4214--b550--81653da1c9bd-osd--block--097d59be--bbe6--493a--b785--48b259d2ff35
> > 253:32 0 7.3T 0 lvm
> > └─AeV0iG-odWF-NRPE-1bVK-0mxH-OgHL-fneTzr 253:33 0 7.3T 0 crypt
> >
> > sds 65:32 0 223.5G 0 disk
> > ├─sds1 65:33 0 512M 0 part
> /boot
> > └─sds2 65:34 0 223G 0 part
> > .......
> > ├─osvg-sdd--db 253:8 0 8G 0 lvm
> > │ └─2ukzAx-g9pZ-IyxU-Sp9h-fHv2-INNY-1vTpvz 253:34 0 8G 0 crypt
> >
> > # ceph-volume inventory /dev/sdd
> >
> > ====== Device report /dev/sdd ======
> >
> > available False
> > rejected reasons locked
> > path /dev/sdd
> > scheduler mode deadline
> > rotational 1
> > vendor SEAGATE
> > human readable size 7.28 TB
> > sas address 0x5000c500a6b1d581
> > removable 0
> > model ST8000NM0185
> > ro 0
> > --- Logical Volume ---
> > cluster name ceph
> > name
> osd-block-097d59be-bbe6-493a-b785-48b259d2ff35
> > osd id 39
> > cluster fsid 8e7a3953-7647-4133-9b9a-7f4a2e2b7da7
> > type block
> > block uuid AeV0iG-odWF-NRPE-1bVK-0mxH-OgHL-fneTzr
> > osd fsid 097d59be-bbe6-493a-b785-48b259d2ff35
> >
> > I was trying to run
> > ceph-volume lvm zap --destroy /dev/sdd but it errored out. Osd id on
> this volume is the same as on next drive, /dev/sde, and osd.39 daemon is
> running. This command was trying to zap running osd.
> >
> > What is the proper way to clean both data and block db volumes, so I can
> rerun ceph-deploy again, and add them to the pool?
> >
>
> Do you want to keep the LVs around or you want to complete get rid of
> them? If you are passing /dev/sdd to 'zap' you are telling the tool to
> destroy everything that is in there, regardless of who owns it
> (including running
> OSDs).
>
> If you want to keep LVs around then you can omit the --destroy flag
> and pass the LVs as input, or if using a recent enough version you can
> use --osd-fsid to zap:
>
> ceph-volume lvm zap osvg-sdd-db/2ukzAx-g9pZ-IyxU-Sp9h-fHv2-INNY-1vTpvz
>
> If you don't want the LVs around you can add --destroy, but use the LV
> as input (not the device)
>
> > Thank you!
> >
> >
> >
> > _______________________________________________
> > ceph-users mailing list
> > ceph-users@lists.ceph.com
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
