[ceph-users] Mount Error
Hi there, I intend to install Ceph on two VMs (client - server) where both have Ubuntu 12.04 and for client 4G RAM and server 8G RAM. Walking through the 5 min quick installation i did relay on the default configurations with little changes: disable the cephx authentication, let rep value = 1 , and finally making the PG default = 200. now when trying to mount cephfs on the directory /mnt/cephfs it gave me an error *Mount error 12= cannot allocate memory * can you help me solving it !!! Thank you. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] Weird problem with mkcephfs
This is consistently repeatable on my system. This is the latest of two cluster builds I have done. This is a brand new deployment on hardware I haven't deployed on previously. Any help is much appreciated. root@mon:~# uname -a Linux mon.X.com 3.2.0-39-generic #62-Ubuntu SMP Thu Feb 28 00:28:53 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux root@mon:~# ceph -v ceph version 0.56.3 (6eb7e15a4783b122e9b0c85ea9ba064145958aa5) root@mon:~# ls -al /tmp/ total 8 drwxrwxrwx 2 root root 4096 Mar 23 12:15 . drwxr-xr-x 25 root root 4096 Mar 22 23:18 .. root@mon:~# ls -al / | grep tmp drwxrwxrwx 2 root root 4096 Mar 23 12:15 tmp root@mon:~# mkcephfs -d /tmp -a -c /etc/ceph/ceph.conf -k /etc/ceph/keyring preparing monmap in /tmp/monmap /usr/bin/monmaptool --create --clobber --add a 192.168.0.253:6789 --print /tmp/monmap /usr/bin/monmaptool: monmap file /tmp/monmap /usr/bin/monmaptool: generated fsid 68b9c724-21c0-4d54-8237-674ced7adbfe epoch 0 fsid 68b9c724-21c0-4d54-8237-674ced7adbfe last_changed 2013-03-23 12:17:03.087018 created 2013-03-23 12:17:03.087018 0: 192.168.0.253:6789/0 mon.a /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) Building generic osdmap from /tmp/conf /usr/bin/osdmaptool: osdmap file '/tmp/osdmap' /usr/bin/osdmaptool: writing epoch 1 to /tmp/osdmap Generating admin key at /tmp/keyring.admin creating /tmp/keyring.admin Building initial monitor keyring cat: /tmp/key.*: No such file or directory root@mon:~# ls -al /tmp/ total 32 drwxrwxrwx 2 root root 4096 Mar 23 12:17 . drwxr-xr-x 25 root root 4096 Mar 22 23:18 .. -rw-r--r-- 1 root root 695 Mar 23 12:17 conf -rw--- 1 root root 63 Mar 23 12:17 keyring.admin -rw--- 1 root root 192 Mar 23 12:17 keyring.mon -rw-r--r-- 1 root root 187 Mar 23 12:17 monmap -rw-r--r-- 1 root root 6886 Mar 23 12:17 osdmap ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] Weird problem with mkcephfs
This is consistently repeatable on my system. This is the latest of two cluster builds I have done. This is a brand new deployment on hardware I haven't deployed on previously. You see the error below is referencing /tmp/key.* and the keyring files are actually keyring.*. Any help is much appreciated. root@mon:~# uname -a Linux mon.X.com 3.2.0-39-generic #62-Ubuntu SMP Thu Feb 28 00:28:53 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux root@mon:~# ceph -v ceph version 0.56.3 (6eb7e15a4783b122e9b0c85ea9ba064145958aa5) root@mon:~# ls -al /tmp/ total 8 drwxrwxrwx 2 root root 4096 Mar 23 12:15 . drwxr-xr-x 25 root root 4096 Mar 22 23:18 .. root@mon:~# ls -al / | grep tmp drwxrwxrwx 2 root root 4096 Mar 23 12:15 tmp root@mon:~# mkcephfs -d /tmp -a -c /etc/ceph/ceph.conf -k /etc/ceph/keyring preparing monmap in /tmp/monmap /usr/bin/monmaptool --create --clobber --add a 192.168.0.253:6789 --print /tmp/monmap /usr/bin/monmaptool: monmap file /tmp/monmap /usr/bin/monmaptool: generated fsid 68b9c724-21c0-4d54-8237-674ced7adbfe epoch 0 fsid 68b9c724-21c0-4d54-8237-674ced7adbfe last_changed 2013-03-23 12:17:03.087018 created 2013-03-23 12:17:03.087018 0: 192.168.0.253:6789/0 mon.a /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) Building generic osdmap from /tmp/conf /usr/bin/osdmaptool: osdmap file '/tmp/osdmap' /usr/bin/osdmaptool: writing epoch 1 to /tmp/osdmap Generating admin key at /tmp/keyring.admin creating /tmp/keyring.admin Building initial monitor keyring cat: /tmp/key.*: No such file or directory root@mon:~# ls -al /tmp/ total 32 drwxrwxrwx 2 root root 4096 Mar 23 12:17 . drwxr-xr-x 25 root root 4096 Mar 22 23:18 .. -rw-r--r-- 1 root root 695 Mar 23 12:17 conf -rw--- 1 root root 63 Mar 23 12:17 keyring.admin -rw--- 1 root root 192 Mar 23 12:17 keyring.mon -rw-r--r-- 1 root root 187 Mar 23 12:17 monmap -rw-r--r-- 1 root root 6886 Mar 23 12:17 osdmap ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] Weird problem with mkcephfs
This is consistently repeatable on my system. This is the latest of two cluster builds I have done. This is a brand new deployment on hardware I haven't deployed on previously. You see the error below is referencing /tmp/key.* and the keyring files are actually keyring.*. Any help is much appreciated. root@mon:~# uname -a Linux mon.X.com 3.2.0-39-generic #62-Ubuntu SMP Thu Feb 28 00:28:53 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux root@mon:~# ceph -v ceph version 0.56.3 (6eb7e15a4783b122e9b0c85ea9ba064145958aa5) root@mon:~# ls -al /tmp/ total 8 drwxrwxrwx 2 root root 4096 Mar 23 12:15 . drwxr-xr-x 25 root root 4096 Mar 22 23:18 .. root@mon:~# ls -al / | grep tmp drwxrwxrwx 2 root root 4096 Mar 23 12:15 tmp root@mon:~# mkcephfs -d /tmp -a -c /etc/ceph/ceph.conf -k /etc/ceph/keyring preparing monmap in /tmp/monmap /usr/bin/monmaptool --create --clobber --add a 192.168.0.253:6789 --print /tmp/monmap /usr/bin/monmaptool: monmap file /tmp/monmap /usr/bin/monmaptool: generated fsid 68b9c724-21c0-4d54-8237-674ced7adbfe epoch 0 fsid 68b9c724-21c0-4d54-8237-674ced7adbfe last_changed 2013-03-23 12:17:03.087018 created 2013-03-23 12:17:03.087018 0: 192.168.0.253:6789/0 mon.a /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) Building generic osdmap from /tmp/conf /usr/bin/osdmaptool: osdmap file '/tmp/osdmap' /usr/bin/osdmaptool: writing epoch 1 to /tmp/osdmap Generating admin key at /tmp/keyring.admin creating /tmp/keyring.admin Building initial monitor keyring cat: /tmp/key.*: No such file or directory root@mon:~# ls -al /tmp/ total 32 drwxrwxrwx 2 root root 4096 Mar 23 12:17 . drwxr-xr-x 25 root root 4096 Mar 22 23:18 .. -rw-r--r-- 1 root root 695 Mar 23 12:17 conf -rw--- 1 root root 63 Mar 23 12:17 keyring.admin -rw--- 1 root root 192 Mar 23 12:17 keyring.mon -rw-r--r-- 1 root root 187 Mar 23 12:17 monmap -rw-r--r-- 1 root root 6886 Mar 23 12:17 osdmap ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] Can't start radosgw
Hi all,
I'm having some issues starting radosgw and I was wondering if anyone here
might have some tips for us. We setup a Ceph cluster with no issues (ceph -s
returns HEALTH_OK) but when we try to start radosgw all we get in the logs is
"Initialization timeout, failed to initialize". We turned debugging up (debug
ms = 1) and tried to run in the foreground (radosgw -d), but no useful logs are
produced... We've quadruple checked all the config, permissions, keyfiles, etc,
but have no idea what's wrong. In the log file (without debug) all we see is
this:
2013-03-23 18:29:18.064508 7fa93ed6f780 0 ceph version 0.56.3
(6eb7e15a4783b122e9b0c85ea9ba064145958aa5), process radosgw, pid 25755
2013-03-23 18:29:48.067629 7fa938a3b700 -1 Initialization timeout, failed to
initialize
If we turn on debug, we see this:
2013-03-23 18:30:05.666988 7fda3dd17780 0 ceph version 0.56.3
(6eb7e15a4783b122e9b0c85ea9ba064145958aa5), process radosgw, pid 25777
2013-03-23 18:30:05.692383 7fda3dd17780 1 -- :/0 messenger.start
2013-03-23 18:30:05.695468 7fda3dd17780 1 -- :/1025779 --> 10.10.10.100:6789/0
-- auth(proto 0 40 bytes epoch 0) v1 -- ?+0 0x14b0930 con 0x14b0520
2013-03-23 18:30:05.695895 7fda33a81700 1 -- 10.10.10.100:0/1025779 learned my
addr 10.10.10.100:0/1025779
2013-03-23 18:30:05.697127 7fda35a85700 1 -- 10.10.10.100:0/1025779 <== mon.0
10.10.10.100:6789/0 1 mon_map v1 473+0+0 (2926159256 0 0) 0x14b5090
con 0x14b0520
2013-03-23 18:30:05.697475 7fda35a85700 1 -- 10.10.10.100:0/1025779 <== mon.0
10.10.10.100:6789/0 2 auth_reply(proto 2 0 Success) v1 33+0+0
(4028035406 0 0) 0x14b5340 con 0x14b0520
2013-03-23 18:30:05.697987 7fda35a85700 1 -- 10.10.10.100:0/1025779 -->
10.10.10.100:6789/0 -- auth(proto 2 32 bytes epoch 0) v1 -- ?+0 0x14b5980 con
0x14b0520
2013-03-23 18:30:05.699032 7fda35a85700 1 -- 10.10.10.100:0/1025779 <== mon.0
10.10.10.100:6789/0 3 auth_reply(proto 2 0 Success) v1 222+0+0
(404397455 0 0) 0x14b5340 con 0x14b0520
2013-03-23 18:30:05.699478 7fda35a85700 1 -- 10.10.10.100:0/1025779 -->
10.10.10.100:6789/0 -- auth(proto 2 181 bytes epoch 0) v1 -- ?+0 0x14b1b70 con
0x14b0520
2013-03-23 18:30:05.700788 7fda35a85700 1 -- 10.10.10.100:0/1025779 <== mon.0
10.10.10.100:6789/0 4 auth_reply(proto 2 0 Success) v1 425+0+0
(1505282999 0 0) 0x14b1fe0 con 0x14b0520
2013-03-23 18:30:05.701088 7fda35a85700 1 -- 10.10.10.100:0/1025779 -->
10.10.10.100:6789/0 -- mon_subscribe({monmap=0+}) v2 -- ?+0 0x14b0d60 con
0x14b0520
2013-03-23 18:30:05.701627 7fda3dd17780 1 -- 10.10.10.100:0/1025779 -->
10.10.10.100:6789/0 -- mon_subscribe({monmap=3+,osdmap=0}) v2 -- ?+0 0x14b7a40
con 0x14b0520
2013-03-23 18:30:05.701791 7fda3dd17780 1 -- 10.10.10.100:0/1025779 -->
10.10.10.100:6789/0 -- mon_subscribe({monmap=3+,osdmap=0}) v2 -- ?+0 0x14b7fb0
con 0x14b0520
2013-03-23 18:30:05.702395 7fda35a85700 1 -- 10.10.10.100:0/1025779 <== mon.0
10.10.10.100:6789/0 5 mon_map v1 473+0+0 (2926159256 0 0) 0x14b7fb0
con 0x14b0520
2013-03-23 18:30:05.702648 7fda35a85700 1 -- 10.10.10.100:0/1025779 <== mon.0
10.10.10.100:6789/0 6 mon_subscribe_ack(300s) v1 20+0+0 (3392334090 0
0) 0x14b83b0 con 0x14b0520
2013-03-23 18:30:05.702757 7fda35a85700 1 -- 10.10.10.100:0/1025779 <== mon.0
10.10.10.100:6789/0 7 mon_subscribe_ack(300s) v1 20+0+0 (3392334090 0
0) 0x7fda2c000ce0 con 0x14b0520
2013-03-23 18:30:05.702814 7fda35a85700 1 -- 10.10.10.100:0/1025779 <== mon.0
10.10.10.100:6789/0 8 mon_subscribe_ack(300s) v1 20+0+0 (3392334090 0
0) 0x7fda2c000f10 con 0x14b0520
2013-03-23 18:30:35.669341 7fda379e3700 -1 Initialization timeout, failed to
initialize
ceph -s returns the following:
# ceph -s
health HEALTH_OK
monmap e2: 3 mons at
{a=10.10.10.100:6789/0,b=10.10.10.101:6789/0,c=10.10.10.102:6789/0}, election
epoch 22, quorum 0,1,2 a,b,c
osdmap e268: 16 osds: 16 up, 16 in
pgmap v8749: 3304 pgs: 3304 active+clean; 540 bytes data, 82134 MB used,
29713 GB / 29793 GB avail
mdsmap e1: 0/0/1 up
Any ideas??
Thanks!
Rom
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Weird problem with mkcephfs
The below part of the mkcephfs code seems responsible for this. specifically the 'for' loop below. I wonder if I installed from the wrong place? I installed from the ubuntu source rather than the ceph source. # admin keyring echo Generating admin key at $dir/keyring.admin $BINDIR/ceph-authtool --create-keyring --gen-key -n client.admin $dir/keyring.admin # mon keyring echo Building initial monitor keyring cp $dir/keyring.admin $dir/keyring.mon $BINDIR/ceph-authtool -n client.admin --set-uid=0 \ --cap mon 'allow *' \ --cap osd 'allow *' \ --cap mds 'allow' \ $dir/keyring.mon $BINDIR/ceph-authtool --gen-key -n mon. $dir/keyring.mon for k in $dir/key.* do kname=`echo $k | sed 's/.*key\.//'` ktype=`echo $kname | cut -c 1-3` kid=`echo $kname | cut -c 4- | sed 's/^\\.//'` kname="$ktype.$kid" secret=`cat $k` if [ "$ktype" = "osd" ]; then $BINDIR/ceph-authtool -n $kname --add-key $secret $dir/keyring.mon \ --cap mon 'allow rwx' \ --cap osd 'allow *' fi if [ "$ktype" = "mds" ]; then $BINDIR/ceph-authtool -n $kname --add-key $secret $dir/keyring.mon \ --cap mon "allow rwx" \ --cap osd 'allow *' \ --cap mds 'allow' fi done exit 0 fi On 03/23/2013 01:50 PM, Steve Carter wrote: This is consistently repeatable on my system. This is the latest of two cluster builds I have done. This is a brand new deployment on hardware I haven't deployed on previously. You see the error below is referencing /tmp/key.* and the keyring files are actually keyring.*. Any help is much appreciated. root@mon:~# uname -a Linux mon.X.com 3.2.0-39-generic #62-Ubuntu SMP Thu Feb 28 00:28:53 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux root@mon:~# ceph -v ceph version 0.56.3 (6eb7e15a4783b122e9b0c85ea9ba064145958aa5) root@mon:~# ls -al /tmp/ total 8 drwxrwxrwx 2 root root 4096 Mar 23 12:15 . drwxr-xr-x 25 root root 4096 Mar 22 23:18 .. root@mon:~# ls -al / | grep tmp drwxrwxrwx 2 root root 4096 Mar 23 12:15 tmp root@mon:~# mkcephfs -d /tmp -a -c /etc/ceph/ceph.conf -k /etc/ceph/keyring preparing monmap in /tmp/monmap /usr/bin/monmaptool --create --clobber --add a 192.168.0.253:6789 --print /tmp/monmap /usr/bin/monmaptool: monmap file /tmp/monmap /usr/bin/monmaptool: generated fsid 68b9c724-21c0-4d54-8237-674ced7adbfe epoch 0 fsid 68b9c724-21c0-4d54-8237-674ced7adbfe last_changed 2013-03-23 12:17:03.087018 created 2013-03-23 12:17:03.087018 0: 192.168.0.253:6789/0 mon.a /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) Building generic osdmap from /tmp/conf /usr/bin/osdmaptool: osdmap file '/tmp/osdmap' /usr/bin/osdmaptool: writing epoch 1 to /tmp/osdmap Generating admin key at /tmp/keyring.admin creating /tmp/keyring.admin Building initial monitor keyring cat: /tmp/key.*: No such file or directory root@mon:~# ls -al /tmp/ total 32 drwxrwxrwx 2 root root 4096 Mar 23 12:17 . drwxr-xr-x 25 root root 4096 Mar 22 23:18 .. -rw-r--r-- 1 root root 695 Mar 23 12:17 conf -rw--- 1 root root 63 Mar 23 12:17 keyring.admin -rw--- 1 root root 192 Mar 23 12:17 keyring.mon -rw-r--r-- 1 root root 187 Mar 23 12:17 monmap -rw-r--r-- 1 root root 6886 Mar 23 12:17 osdmap ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Weird problem with mkcephfs
I changed: for k in $dir/key.* to: for k in $dir/key* and it appeared to run correctly: root@smon:/etc/ceph# mkcephfs -a -c /etc/ceph/ceph.conf -d /tmp -k /etc/ceph/keyring preparing monmap in /tmp/monmap /usr/bin/monmaptool --create --clobber --add a 192.168.0.253:6789 --print /tmp/monmap /usr/bin/monmaptool: monmap file /tmp/monmap /usr/bin/monmaptool: generated fsid 46e4ae99-3df6-41ae-8d45-474c95b98852 epoch 0 fsid 46e4ae99-3df6-41ae-8d45-474c95b98852 last_changed 2013-03-23 22:33:26.254974 created 2013-03-23 22:33:26.254974 0: 192.168.0.253:6789/0 mon.a /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) Building generic osdmap from /tmp/conf /usr/bin/osdmaptool: osdmap file '/tmp/osdmap' /usr/bin/osdmaptool: writing epoch 1 to /tmp/osdmap Generating admin key at /tmp/keyring.admin creating /tmp/keyring.admin Building initial monitor keyring placing client.admin keyring in /etc/ceph/keyring On 03/23/2013 10:29 PM, Steve Carter wrote: The below part of the mkcephfs code seems responsible for this. specifically the 'for' loop below. I wonder if I installed from the wrong place? I installed from the ubuntu source rather than the ceph source. # admin keyring echo Generating admin key at $dir/keyring.admin $BINDIR/ceph-authtool --create-keyring --gen-key -n client.admin $dir/keyring.admin # mon keyring echo Building initial monitor keyring cp $dir/keyring.admin $dir/keyring.mon $BINDIR/ceph-authtool -n client.admin --set-uid=0 \ --cap mon 'allow *' \ --cap osd 'allow *' \ --cap mds 'allow' \ $dir/keyring.mon $BINDIR/ceph-authtool --gen-key -n mon. $dir/keyring.mon for k in $dir/key.* do kname=`echo $k | sed 's/.*key\.//'` ktype=`echo $kname | cut -c 1-3` kid=`echo $kname | cut -c 4- | sed 's/^\\.//'` kname="$ktype.$kid" secret=`cat $k` if [ "$ktype" = "osd" ]; then $BINDIR/ceph-authtool -n $kname --add-key $secret $dir/keyring.mon \ --cap mon 'allow rwx' \ --cap osd 'allow *' fi if [ "$ktype" = "mds" ]; then $BINDIR/ceph-authtool -n $kname --add-key $secret $dir/keyring.mon \ --cap mon "allow rwx" \ --cap osd 'allow *' \ --cap mds 'allow' fi done exit 0 fi On 03/23/2013 01:50 PM, Steve Carter wrote: This is consistently repeatable on my system. This is the latest of two cluster builds I have done. This is a brand new deployment on hardware I haven't deployed on previously. You see the error below is referencing /tmp/key.* and the keyring files are actually keyring.*. Any help is much appreciated. root@mon:~# uname -a Linux mon.X.com 3.2.0-39-generic #62-Ubuntu SMP Thu Feb 28 00:28:53 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux root@mon:~# ceph -v ceph version 0.56.3 (6eb7e15a4783b122e9b0c85ea9ba064145958aa5) root@mon:~# ls -al /tmp/ total 8 drwxrwxrwx 2 root root 4096 Mar 23 12:15 . drwxr-xr-x 25 root root 4096 Mar 22 23:18 .. root@mon:~# ls -al / | grep tmp drwxrwxrwx 2 root root 4096 Mar 23 12:15 tmp root@mon:~# mkcephfs -d /tmp -a -c /etc/ceph/ceph.conf -k /etc/ceph/keyring preparing monmap in /tmp/monmap /usr/bin/monmaptool --create --clobber --add a 192.168.0.253:6789 --print /tmp/monmap /usr/bin/monmaptool: monmap file /tmp/monmap /usr/bin/monmaptool: generated fsid 68b9c724-21c0-4d54-8237-674ced7adbfe epoch 0 fsid 68b9c724-21c0-4d54-8237-674ced7adbfe last_changed 2013-03-23 12:17:03.087018 created 2013-03-23 12:17:03.087018 0: 192.168.0.253:6789/0 mon.a /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) Building generic osdmap from /tmp/conf /usr/bin/osdmaptool: osdmap file '/tmp/osdmap' /usr/bin/osdmaptool: writing epoch 1 to /tmp/osdmap Generating admin key at /tmp/keyring.admin creating /tmp/keyring.admin Building initial monitor keyring cat: /tmp/key.*: No such file or directory root@mon:~# ls -al /tmp/ total 32 drwxrwxrwx 2 root root 4096 Mar 23 12:17 . drwxr-xr-x 25 root root 4096 Mar 22 23:18 .. -rw-r--r-- 1 root root 695 Mar 23 12:17 conf -rw--- 1 root root 63 Mar 23 12:17 keyring.admin -rw--- 1 root root 192 Mar 23 12:17 keyring.mon -rw-r--r-- 1 root root 187 Mar 23 12:17 monmap -rw-r--r-- 1 root root 6886 Mar 23 12:17 osdmap ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
