On Fri, Apr 19, 2019 at 10:44 AM Varun Singh <varun.si...@gslab.com> wrote: > > On Thu, Apr 18, 2019 at 9:53 PM Siegfried Höllrigl > <siegfried.hoellr...@xidras.com> wrote: > > > > Hi ! > > > > I am not 100% sure, but i think, --net=host does not propagate /dev/ > > inside the conatiner. > > > > From the Error Message : > > > > 2019-04-18 07:30:06 /opt/ceph-container/bin/entrypoint.sh: ERROR- The > > device pointed by OSD_DEVICE (/dev/vdd) doesn't exist ! > > > > > > I whould say, you should add something like --device=/dev/vdd to the docker > > run command for the osd. > > > > Br > > > > > > Am 18.04.2019 um 14:46 schrieb Varun Singh: > > > Hi, > > > I am trying to setup Ceph through Docker inside a VM. My host machine > > > is Mac. My VM is an Ubuntu 18.04. Docker version is 18.09.5, build > > > e8ff056. > > > I am following the documentation present on ceph/daemon Docker Hub > > > page. The idea is, if I spawn docker containers as mentioned on the > > > page, I should get a ceph setup without KV store. I am not worried > > > about KV store as I just want to try it out. Following are the > > > commands I am firing to bring the containers up: > > > > > > Monitor: > > > docker run -d --net=host -v /etc/ceph:/etc/ceph -v > > > /var/lib/ceph/:/var/lib/ceph/ -e MON_IP=10.0.2.15 -e > > > CEPH_PUBLIC_NETWORK=10.0.2.0/24 ceph/daemon mon > > > > > > Manager: > > > docker run -d --net=host -v /etc/ceph:/etc/ceph -v > > > /var/lib/ceph/:/var/lib/ceph/ ceph/daemon mgr > > > > > > OSD: > > > docker run -d --net=host --pid=host --privileged=true -v > > > /etc/ceph:/etc/ceph -v /var/lib/ceph/:/var/lib/ceph/ -v /dev/:/dev/ -e > > > OSD_DEVICE=/dev/vdd ceph/daemon osd > > > > > > From the above commands I am able to spawn monitor and manager > > > properly. I verified this by firing this command on both monitor and > > > manager containers: > > > sudo docker exec d1ab985 ceph -s > > > > > > I get following outputs for both: > > > > > > cluster: > > > id: 14a6e40a-8e54-4851-a881-661a84b3441c > > > health: HEALTH_OK > > > > > > services: > > > mon: 1 daemons, quorum serverceph-VirtualBox (age 62m) > > > mgr: serverceph-VirtualBox(active, since 56m) > > > osd: 0 osds: 0 up, 0 in > > > > > > data: > > > pools: 0 pools, 0 pgs > > > objects: 0 objects, 0 B > > > usage: 0 B used, 0 B / 0 B avail > > > pgs: > > > > > > However when I try to bring up OSD using above command, it doesn't > > > work. Docker logs show this output: > > > 2019-04-18 07:30:06 /opt/ceph-container/bin/entrypoint.sh: static: > > > does not generate config > > > 2019-04-18 07:30:06 /opt/ceph-container/bin/entrypoint.sh: ERROR- The > > > device pointed by OSD_DEVICE (/dev/vdd) doesn't exist ! > > > > > > I am not sure why the doc asks to pass /dev/vdd to OSD_DEVICE env var. > > > I know there are five different ways to spawning the OSD, but I am not > > > able to figure out which one would be suitable for a simple > > > deployment. If you could please let me know how to spawn OSDs using > > > Docker, it would help a lot. > > > > > > > > Thanks Br, I will try this out today. > > -- > Regards, > Varun Singh
Hi, So following your suggestion I tried following two commands: 1. I added --device=/dev/vdd switch without removing OSD_DEVICE env var. This resulted in same error before docker run -d --net=host --pid=host --privileged=true --device=/dev/vdd -v /etc/ceph:/etc/ceph -v /var/lib/ceph/:/var/lib/ceph/ -v /dev/:/dev/ -e OSD_DEVICE=/dev/vdd ceph/daemon osd 2. Then I removed OSD_DEVICE env var and just added --device=/dev/vdd switch docker run -d --net=host --pid=host --privileged=true --device=/dev/vdd -v /etc/ceph:/etc/ceph -v /var/lib/ceph/:/var/lib/ceph/ -v /dev/:/dev/ ceph/daemon osd OSD_DEVICE related error went away and I think ceph created an OSD successfully. But it wasn't able to connect to cluster. Is it because I did not give and network related information? I get the following error now: 2019-04-18 08:30:47 /opt/ceph-container/bin/entrypoint.sh: static: does not generate config 2019-04-18 08:30:47 /opt/ceph-container/bin/entrypoint.sh: Bootstrapped OSD(s) found; using OSD directory 2019-04-18 08:30:47 /opt/ceph-container/bin/entrypoint.sh: Creating osd 2019-04-18 08:30:52.944 7f897ca6d700 -1 auth: unable to find a keyring on /var/lib/ceph/bootstrap-osd/ceph.keyring: (2) No such file or directory 2019-04-18 08:30:52.944 7f897ca6d700 -1 AuthRegistry(0x7f8978063e78) no keyring found at /var/lib/ceph/bootstrap-osd/ceph.keyring, disabling cephx 2019-04-18 08:30:52.964 7f897ca6d700 -1 auth: unable to find a keyring on /var/lib/ceph/bootstrap-osd/ceph.keyring: (2) No such file or directory 2019-04-18 08:30:52.964 7f897ca6d700 -1 AuthRegistry(0x7f89780bcad8) no keyring found at /var/lib/ceph/bootstrap-osd/ceph.keyring, disabling cephx 2019-04-18 08:30:52.964 7f897ca6d700 -1 auth: unable to find a keyring on /var/lib/ceph/bootstrap-osd/ceph.keyring: (2) No such file or directory 2019-04-18 08:30:52.964 7f897ca6d700 -1 AuthRegistry(0x7f897ca6be78) no keyring found at /var/lib/ceph/bootstrap-osd/ceph.keyring, disabling cephx 2019-04-18 08:30:52.976 7f897585d700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [1] 2019-04-18 08:30:52.976 7f897ca6d700 -1 monclient: authenticate NOTE: no keyring found; disabled cephx authentication [errno 95] error connecting to the cluster I tried googling the error and came across a couple of filed bugs but no solution. Any help in fixing this would be great. -- Regards, Varun Singh -- Confidentiality Notice and Disclaimer: This email (including any attachments) contains information that may be confidential, privileged and/or copyrighted. If you are not the intended recipient, please notify the sender immediately and destroy this email. Any unauthorized use of the contents of this email in any manner whatsoever, is strictly prohibited. If improper activity is suspected, all available information may be used by the sender for possible disciplinary action, prosecution, civil claim or any remedy or lawful purpose. Email transmission cannot be guaranteed to be secure or error-free, as information could be intercepted, lost, arrive late, or contain viruses. The sender is not liable whatsoever for damage resulting from the opening of this message and/or the use of the information contained in this message and/or attachments. Expressions in this email cannot be treated as opined by the sender company management – they are solely expressed by the sender unless authorized. _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
