Hi Travis, These binaries are hosted on Canonical servers and are only for Ubuntu. Until the latest FireFly patch release 0.80.9, everything worked fine. I just tried the hammer binaries, and they seem to be failing in loading up erasure coding libraries. I have now built my own binaries and I was able to get the cluster up and running using ceph-deploy. You just have to skip the ceph installation step with ceph-deploy and rather do a manual install from deb files. Rest worked fine.
Thanks Pankaj -----Original Message----- From: Travis Rhoden [mailto:trho...@gmail.com] Sent: Thursday, May 28, 2015 8:02 AM To: Garg, Pankaj Cc: ceph-users@lists.ceph.com Subject: Re: [ceph-users] ceph-deploy for Hammer Hi Pankaj, While there have been times in the past where ARM binaries were hosted on ceph.com, there is not currently any ARM hardware for builds. I don't think you will see any ARM binaries in http://ceph.com/debian-hammer/pool/main/c/ceph/, for example. Combine that with the fact that ceph-deploy is not intended to work with locally compiled binaries (only packages, as it relies on paths, conventions, and service definitions from the packages), and it is a very tricky combo to use ceph-deploy and ARM together. Your most recent error is indicative of the ceph-mon service not coming up successfully. when ceph-mon (the service, not the daemon) is started, it also calls ceph-create-keys, which waits for the monitor daemon to come up and the creates keys that are necessary for all clusters to run when using cephx (the admin key, the bootsraps keys). - Travis On Wed, May 27, 2015 at 8:27 PM, Garg, Pankaj <pankaj.g...@caviumnetworks.com> wrote: > Actually the ARM binaries do exist and I have been using for previous > releases. Somehow this library is the one that doesn’t load. > > Anyway I did compile my own Ceph for ARM, and now getting the > following > issue: > > > > [ceph_deploy.gatherkeys][WARNIN] Unable to find > /etc/ceph/ceph.client.admin.keyring on ceph1 > > [ceph_deploy][ERROR ] KeyNotFoundError: Could not find keyring file: > /etc/ceph/ceph.client.admin.keyring on host ceph1 > > > > > > From: Somnath Roy [mailto:somnath....@sandisk.com] > Sent: Wednesday, May 27, 2015 4:29 PM > To: Garg, Pankaj > > > Cc: ceph-users@lists.ceph.com > Subject: RE: ceph-deploy for Hammer > > > > If you are trying to install the ceph repo hammer binaries, I don’t > think it is built for ARM. Both binary and the .so needs to be built > in ARM to make this work I guess. > > Try to build hammer code base in your ARM server and then retry. > > > > Thanks & Regards > > Somnath > > > > From: Pankaj Garg [mailto:pankaj.g...@caviumnetworks.com] > Sent: Wednesday, May 27, 2015 4:17 PM > To: Somnath Roy > Cc: ceph-users@lists.ceph.com > Subject: RE: ceph-deploy for Hammer > > > > Yes I am on ARM. > > -Pankaj > > On May 27, 2015 3:58 PM, Somnath Roy <somnath....@sandisk.com> wrote: > > Are you running this on ARM ? > > If not, it should not go for loading this library. > > > > Thanks & Regards > > Somnath > > > > From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf > Of Garg, Pankaj > Sent: Wednesday, May 27, 2015 2:26 PM > To: Garg, Pankaj; ceph-users@lists.ceph.com > Subject: Re: [ceph-users] ceph-deploy for Hammer > > > > I seem to be getting these errors in the Monitor Log : > > 2015-05-27 21:17:41.908839 3ff907368e0 -1 > erasure_code_init(jerasure,/usr/lib/aarch64-linux-gnu/ceph/erasure-code): > (5) Input/output error > > 2015-05-27 21:17:41.978113 3ff969168e0 0 ceph version 0.94.1 > (e4bfad3a3c51054df7e537a724c8d0bf9be972ff), process ceph-mon, pid > 16592 > > 2015-05-27 21:17:41.984383 3ff969168e0 -1 ErasureCodePluginSelectJerasure: > load > dlopen(/usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so): > /usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so: > cannot open shared object file: No such file or directory > > 2015-05-27 21:17:41.984444 3ff969168e0 -1 > erasure_code_init(jerasure,/usr/lib/aarch64-linux-gnu/ceph/erasure-code): > (5) Input/output error > > 2015-05-27 21:17:42.052415 3ff90cf68e0 0 ceph version 0.94.1 > (e4bfad3a3c51054df7e537a724c8d0bf9be972ff), process ceph-mon, pid > 16604 > > 2015-05-27 21:17:42.058656 3ff90cf68e0 -1 ErasureCodePluginSelectJerasure: > load > dlopen(/usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so): > /usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so: > cannot open shared object file: No such file or directory > > 2015-05-27 21:17:42.058715 3ff90cf68e0 -1 > erasure_code_init(jerasure,/usr/lib/aarch64-linux-gnu/ceph/erasure-code): > (5) Input/output error > > 2015-05-27 21:17:42.125279 3ffac4368e0 0 ceph version 0.94.1 > (e4bfad3a3c51054df7e537a724c8d0bf9be972ff), process ceph-mon, pid > 16616 > > 2015-05-27 21:17:42.131666 3ffac4368e0 -1 ErasureCodePluginSelectJerasure: > load > dlopen(/usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so): > /usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so: > cannot open shared object file: No such file or directory > > 2015-05-27 21:17:42.131726 3ffac4368e0 -1 > erasure_code_init(jerasure,/usr/lib/aarch64-linux-gnu/ceph/erasure-code): > (5) Input/output error > > > > > > The lib file exists, so not sure why this is happening. Any help > appreciated. > > > > Thanks > > Pankaj > > > > From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf > Of Garg, Pankaj > Sent: Wednesday, May 27, 2015 1:37 PM > To: ceph-users@lists.ceph.com > Subject: [ceph-users] ceph-deploy for Hammer > > > > Hi, > > Is there a particular verion of Ceph-Deploy that should be used with > Hammer release? This is a brand new cluster. > > I’m getting the following error when running command : ceph-deploy mon > create-initial > > > > [ceph_deploy.conf][DEBUG ] found configuration file at: > /home/cephuser/.cephdeploy.conf > > [ceph_deploy.cli][INFO ] Invoked (1.5.25): /usr/local/bin/ceph-deploy > mon create-initial > > [ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts ceph1 > > [ceph_deploy.mon][DEBUG ] detecting platform for host ceph1 ... > > [ceph1][DEBUG ] connection detected need for sudo > > [ceph1][DEBUG ] connected to host: ceph1 > > [ceph1][DEBUG ] detect platform information from remote host > > [ceph1][DEBUG ] detect machine type > > [ceph_deploy.mon][INFO ] distro info: Ubuntu 14.04 trusty > > [ceph1][DEBUG ] determining if provided host has same hostname in > remote > > [ceph1][DEBUG ] get remote short hostname > > [ceph1][DEBUG ] deploying mon to ceph1 > > [ceph1][DEBUG ] get remote short hostname > > [ceph1][DEBUG ] remote hostname: ceph1 > > [ceph1][DEBUG ] write cluster configuration to > /etc/ceph/{cluster}.conf > > [ceph1][DEBUG ] create the mon path if it does not exist > > [ceph1][DEBUG ] checking for done path: > /var/lib/ceph/mon/ceph-ceph1/done > > [ceph1][DEBUG ] create a done file to avoid re-doing the mon > deployment > > [ceph1][DEBUG ] create the init path if it does not exist > > [ceph1][DEBUG ] locating the `service` executable... > > [ceph1][INFO ] Running command: sudo initctl emit ceph-mon > cluster=ceph > id=ceph1 > > [ceph1][INFO ] Running command: sudo ceph --cluster=ceph > --admin-daemon /var/run/ceph/ceph-mon.ceph1.asok mon_status > > [ceph1][ERROR ] admin_socket: exception getting command descriptions: > [Errno 2] No such file or directory > > [ceph1][WARNIN] monitor: mon.ceph1, might not be running yet > > [ceph1][INFO ] Running command: sudo ceph --cluster=ceph > --admin-daemon /var/run/ceph/ceph-mon.ceph1.asok mon_status > > [ceph1][ERROR ] admin_socket: exception getting command descriptions: > [Errno 2] No such file or directory > > [ceph1][WARNIN] monitor ceph1 does not exist in monmap > > [ceph_deploy.mon][INFO ] processing monitor mon.ceph1 > > [ceph1][DEBUG ] connection detected need for sudo > > [ceph1][DEBUG ] connected to host: ceph1 > > [ceph1][INFO ] Running command: sudo ceph --cluster=ceph > --admin-daemon /var/run/ceph/ceph-mon.ceph1.asok mon_status > > [ceph1][ERROR ] admin_socket: exception getting command descriptions: > [Errno 2] No such file or directory > > [ceph_deploy.mon][WARNIN] mon.ceph1 monitor is not yet in quorum, > tries > left: 5 > > [ceph_deploy.mon][WARNIN] waiting 5 seconds before retrying > > [ceph1][INFO ] Running command: sudo ceph --cluster=ceph > --admin-daemon /var/run/ceph/ceph-mon.ceph1.asok mon_status > > [ceph1][ERROR ] admin_socket: exception getting command descriptions: > [Errno 2] No such file or directory > > [ceph_deploy.mon][WARNIN] mon.ceph1 monitor is not yet in quorum, > tries > left: 4 > > [ceph_deploy.mon][WARNIN] waiting 10 seconds before retrying > > [ceph1][INFO ] Running command: sudo ceph --cluster=ceph > --admin-daemon /var/run/ceph/ceph-mon.ceph1.asok mon_status > > [ceph1][ERROR ] admin_socket: exception getting command descriptions: > [Errno 2] No such file or directory > > [ceph_deploy.mon][WARNIN] mon.ceph1 monitor is not yet in quorum, > tries > left: 3 > > [ceph_deploy.mon][WARNIN] waiting 10 seconds before retrying > > [ceph1][INFO ] Running command: sudo ceph --cluster=ceph > --admin-daemon /var/run/ceph/ceph-mon.ceph1.asok mon_status > > [ceph1][ERROR ] admin_socket: exception getting command descriptions: > [Errno 2] No such file or directory > > [ceph_deploy.mon][WARNIN] mon.ceph1 monitor is not yet in quorum, > tries > left: 2 > > [ceph_deploy.mon][WARNIN] waiting 15 seconds before retrying > > [ceph1][INFO ] Running command: sudo ceph --cluster=ceph > --admin-daemon /var/run/ceph/ceph-mon.ceph1.asok mon_status > > [ceph1][ERROR ] admin_socket: exception getting command descriptions: > [Errno 2] No such file or directory > > [ceph_deploy.mon][WARNIN] mon.ceph1 monitor is not yet in quorum, > tries > left: 1 > > [ceph_deploy.mon][WARNIN] waiting 20 seconds before retrying > > [ceph_deploy.mon][ERROR ] Some monitors have still not reached quorum: > > [ceph_deploy.mon][ERROR ] ceph1 > > > > Thanks > > Pankaj > > > > ________________________________ > > > PLEASE NOTE: The information contained in this electronic mail message > is intended only for the use of the designated recipient(s) named > above. If the reader of this message is not the intended recipient, > you are hereby notified that you have received this message in error > and that any review, dissemination, distribution, or copying of this > message is strictly prohibited. If you have received this > communication in error, please notify the sender by telephone or > e-mail (as shown above) immediately and destroy any and all copies of > this message in your possession (whether hard copies or electronically stored > copies). > > > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
