You should check that the owner of your ceph partitions (both journal and data) is 'ceph', otherwise the ceph user won't mount it.
You can simply do : chown ceph:disk /dev/sdc3 If this solve your issue you should set the GPT GUID [1] of the partitions with a tool like sgdisk to make this persistent across reboot. I think only your journal is affected as ceph-disk does not prepare the partition (WARNING:ceph-disk:Journal /dev/sdc3 was not prepared with ceph-disk. Symlinking directly) [1] https://en.wikipedia.org/wiki/GUID_Partition_Table#Partition_type_GUIDs On Thu, Dec 3, 2015 at 3:03 AM, Goncalo Borges <goncalo.bor...@sydney.edu.au > wrote: > Dear Cephers... > > 1./ I am currently deploying infernalis 9.2.0 in centos7. > > 2./ I am not using ceph-deploy (I prefer not to use it) and I prefer to > use the short form osd installation procedure described here: > > http://docs.ceph.com/docs/infernalis/install/manual-deployment/#short-form > > 3./ In hammer this instructions worked fine but in infernalis 9.2.0, which > the implementation of the ceph user running the ceph daemons, those > instructions do not seem to work properly anymore. > > 4./ Here is the behaviour: > > 4.1/ ceph-disk prepare runs fine: > > # ceph-disk prepare --cluster ceph --cluster-uuid > a9431bc6-3ee1-4b0a-8d21-0ad883a4d2ed --fs-type xfs /dev/sdh /dev/sdc3 > WARNING:ceph-disk:OSD will not be hot-swappable if journal is not the same > device as the osd data > WARNING:ceph-disk:Journal /dev/sdc3 was not prepared with ceph-disk. > Symlinking directly. > Creating new GPT entries. > The operation has completed successfully. > meta-data=/dev/sdh1 isize=2048 agcount=4, agsize=183107519 > blks > = sectsz=512 attr=2, projid32bit=1 > = crc=0 finobt=0 > data = bsize=4096 blocks=732430075, imaxpct=5 > = sunit=0 swidth=0 blks > naming =version 2 bsize=4096 ascii-ci=0 ftype=0 > log =internal log bsize=4096 blocks=357631, version=2 > = sectsz=512 sunit=0 blks, lazy-count=1 > realtime =none extsz=4096 blocks=0, rtextents=0 > The operation has completed successfully. > > 4.2/ ceph-disk activate fails with permission problems. It seems some > temporary directory us made by root which is then not accessible to ceph > user. > > # ceph-disk activate /dev/sdh1 > got monmap epoch 1 > 2015-12-03 01:56:41.406742 7fb050fa9900 -1 > filestore(/var/lib/ceph/tmp/mnt.dsi7zI) mkjournal error creating journal on > /var/lib/ceph/tmp/mnt.dsi7zI/journal: (13) Permission denied > 2015-12-03 01:56:41.406777 7fb050fa9900 -1 OSD::mkfs: ObjectStore::mkfs > failed with error -13 > 2015-12-03 01:56:41.406843 7fb050fa9900 -1 ** ERROR: error creating empty > object store in /var/lib/ceph/tmp/mnt.dsi7zI: (13) Permission denied > ERROR:ceph-disk:Failed to activate > Traceback (most recent call last): > File "/usr/sbin/ceph-disk", line 3576, in <module> > main(sys.argv[1:]) > File "/usr/sbin/ceph-disk", line 3532, in main > main_catch(args.func, args) > File "/usr/sbin/ceph-disk", line 3554, in main_catch > func(args) > File "/usr/sbin/ceph-disk", line 2424, in main_activate > dmcrypt_key_dir=args.dmcrypt_key_dir, > File "/usr/sbin/ceph-disk", line 2197, in mount_activate > (osd_id, cluster) = activate(path, activate_key_template, init) > File "/usr/sbin/ceph-disk", line 2360, in activate > keyring=keyring, > File "/usr/sbin/ceph-disk", line 1950, in mkfs > '--setgroup', get_ceph_user(), > File "/usr/sbin/ceph-disk", line 349, in command_check_call > return subprocess.check_call(arguments) > File "/usr/lib64/python2.7/subprocess.py", line 542, in check_call > raise CalledProcessError(retcode, cmd) > subprocess.CalledProcessError: Command '['/usr/bin/ceph-osd', '--cluster', > 'ceph', '--mkfs', '--mkkey', '-i', '4', '--monmap', > '/var/lib/ceph/tmp/mnt.dsi7zI/activate.monmap', '--osd-data', > '/var/lib/ceph/tmp/mnt.dsi7zI', '--osd-journal', > '/var/lib/ceph/tmp/mnt.dsi7zI/journal', '--osd-uuid', > '995e436f-dc6f-42ca-bbbf-3ef9f0bb4952', '--keyring', > '/var/lib/ceph/tmp/mnt.dsi7zI/keyring', '--setuser', 'ceph', '--setgroup', > 'ceph']' returned non-zero exit status 1 > > 4.3 If I run the the ceph activate command as a ceph user, it > complains it need root permissions to mount > > > # su ceph -s /bin/bash --session-command="ceph-disk activate /dev/sdh1" > Problem opening /dev/sdh for reading! Error is 13. > You must run this program as root or use sudo! > mount: only root can use "--options" option > ceph-disk: Mounting filesystem failed: Command '['/usr/bin/mount', '-t', > 'xfs', '-o', 'noatime,inode64', '--', '/dev/sdh1', > '/var/lib/ceph/tmp/mnt.0bUl5q']' returned non-zero exit status 1 > > 5./ So there is an incoherence on this set of instructions. > > Is there a way to solve / patch that solves this? > > Cheers > Goncalo > > -- > Goncalo Borges > Research Computing > ARC Centre of Excellence for Particle Physics at the Terascale > School of Physics A28 | University of Sydney, NSW 2006 > T: +61 2 93511937 > > > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > -- ----------------------------------------------------------------------------------------- Adrien GILLARD +33 (0)6 29 06 16 31 gillard.adr...@gmail.com
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
